this post was submitted on 24 Jul 2024
133 points (95.2% liked)

Selfhosted

40296 readers
322 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

Context : Immich default map tile provider (which gets sent a bunch of PII every time you use the map feature) is a company that I see no reason to trust. This is a follow-up to this post, with the ~~permanent~~ temporary fix I came up with. I will also summarize the general opinion from the comments, as well as some interesting piece of knowledge that commenters shared.

Hacky fix

This will use Nginx proxy module to build a caching proxy in front of Open Street Map's tileserver and to serve a custom style.json for the maps.

This works well for me, since I already proxy all my services behind a single Nginx instance. It is probably possible to achieve similar results with other reverse proxies, but this would obviously need to be adapted.

Caching proxy

Inside Nginx's http config block (usually in /etc/nginx/nginx.conf), create a cache zone (a directory that will hold cached responses from OSM) :

http {
     # You should not need to edit existing lines in the http block, only add the line below
    proxy_cache_path /var/cache/nginx/osm levels=1:2 keys_zone=osm:100m max_size=5g inactive=180d;
}

You may need to manually create the /var/cache/nginx/osm directory and set its owner to Nginx's user (typically www-data on Debian based distros).

Customize the max_size parameter to change the maximum amount of cached data you want to store on your server. The inactive parameter will cause Nginx to discard cached data that's not been accessed in this duration (180d ~ 6months).

Then, inside the server block that serves your Immich instance, create a new location block :

server {
    listen 443 ssl;
    server_name immich.your-domain.tld;

    # You should not need to change your existing config, only add the location block below

    location /map_proxy/ {
        proxy_pass https://tile.openstreetmap.org/;
        proxy_cache osm;
        proxy_cache_valid 180d;
        proxy_ignore_headers Cache-Control Expires;
        proxy_ssl_server_name on;
        proxy_ssl_name tile.openstreetmap.org;
        proxy_set_header Host tile.openstreetmap.org;
        proxy_set_header User-Agent "Nginx Caching Tile Proxy for self-hosters";
        proxy_set_header Cookie "";
        proxy_set_header Referer "";
    }
}

Reload Nginx (sudo systemctl reload nginx). Confirm this works by visiting https://immich.your-domain.tld/map_proxy/0/0/0.png, which should now return a world map PNG (the one from https://tile.openstreetmap.org/0/0/0.png )

This config ignores cache control headers from OSM and sets its own cache validity duration (proxy_cache_valid parameter). After the specified duration, the proxy will re-fetch the tiles. 6 months seem reasonable to me for the use case, and it can probably be set to a few years without it causing issues.

Besides being lighter on OSM's servers, the caching proxy will improve privacy by only requesting tiles from upstream when loaded for the first time. This config also strips cookies and referrer before forwarding the queries to OSM, as well as set a user agent for the proxy following OSM foundation's guidelines (according to these guidelines, you should add a contact information to this user agent)

This can probably be made to work on a different domain than the one serving your Immich instance, but this probably requires to add the appropriate headers for CORS.

Custom style.json

I came up with the following mapstyle :

{
  "version": 8,
  "name": "Immich Map",
  "sources": {
    "immich-map": {
      "type": "raster",
      "tileSize": 256,
      "tiles": [
        "https://immich.your-domain.tld/map_proxy/{z}/{x}/{y}.png"
      ]
    }
  },
  "sprite": "https://maputnik.github.io/osm-liberty/sprites/osm-liberty",
  "glyphs": "https://fonts.openmaptiles.org/{fontstack}/{range}.pbf",
  "layers": [
    {
      "id": "raster-tiles",
      "type": "raster",
      "source": "immich-map",
      "minzoom": 0,
      "maxzoom": 22
    }
  ],
  "id": "immich-map-dark"
}

Replace immich.your-domain.tld with your actual Immich domain, and remember the absolute path you save this at.

One last update to nginx's config

Since Immich currently does not provide a way to manually edit style.json, we need to serve it from http(s). Add one more location block below the previous one :

location /map_style.json {
    alias /srv/immich/mapstyle.json;
}

Replace the alias parameter with the location where you saved the json mapstyle. After reloading nginx, your json style will be available at https://immich.your-domain.tld/map_style.json

Configure Immich to use this

For this last part, follow steps 8, 9, 10 from this guide (use the link to map_style.json for both light and dark themes). After clearing the browser or app's cache, the map should now be loaded from your caching proxy. You can confirm this by tailing Nginx's logs while you zoom and move around the map in Immich

Summary of comments from previous post

Self-hosting a tile server is not realistic in most cases

People who have previously worked with maps seem to confirm that there are no tile server solution lightweight enough to be self hosted by hobbyists. There is maybe some hope with generating tiles on demand, but someone with deep knowledge of the file formats involved in the process should confirm this.

Some interesting links were shared, which seem to confirm this is not realistically self-hostable with the available software :

General sentiment about this issue

In all this part, I want to emphasize that while there seems to be a consensus, this is only based on the few comments from the previous post and may be biased by the fact that we're discussing it on a non-mainstream platform. If you disagree with anything below, please comment this post and explain your point of view.

  • Nobody declared that they had noticed the requests to a third-party server before
  • A non-negligible fraction of Immich users are interested in the privacy benefits over other solutions such as Google photos. These users do not like their self-hosted services to send requests to third-party servers without warning them first
  • The fix should consist of the following :
    • Clearly document the implications of enabling the map, and any feature that sends requests to third parties
    • Disable by default features that send requests to third parties (especially if it contains any form of geolocated data)
    • Provide a way to easily change the tile provider. A select menu with a few pre-configured style.json would be nice, along with a way to manually edit style.json (or at least some of its fields) directly from the Immich config page
all 30 comments
sorted by: hot top controversial new old
[–] HybridSarcasm@lemmy.world 22 points 4 months ago (1 children)

This is good stuff. Has it been posted to the project’s GitHub (issue, discussion, etc.)?

[–] pcouy@lemmy.pierre-couy.fr 20 points 4 months ago

Not yet, but I will probably submit a PR to include this guide in the docs

[–] Churbleyimyam@lemm.ee 18 points 3 months ago (1 children)

Man, thank you for looking into this.

I don't even use Immich but it's a relief to know that people like you are actively monitoring these kinds of applications. I have a VPS running a few things and there is no way I would be happy with an application on it behaving this way without my consent.

[–] pcouy@lemmy.pierre-couy.fr 5 points 3 months ago* (last edited 3 months ago)

I used to wonder what kind of nerd notices this kind of thing, now I'm one of them

Edit : If you want to join us :

  • you can run Pi-hole which is a self-hosted DNS server that allow monitoring/blocking DNS requests from devices configured to use it. In its default configuration, it acts as a network wide ad/tracker blocker.
  • On Android, you can install Rethink DNS. This will configure itself as a VPN on your device, forcing all traffic to go through it. This allows it to act as an on-device firewall that allow monitoring/blocking DNS requests and TCP/UDP connections. This is similar to the features of Pi-hole, but the fact that it's on-device allows it to be app aware : the logs will detail which app is responsible for which connection, and the allow/block rules can be app-dependent. The app honestly goes beyond all my expectations :
    • it does a good job at being easy to use by default
    • it is very configurable which gives you a lot of control if you want/need/can handle it
    • You can configure it to route traffic (after applying firewall rules) to a Wireguard VPN or through Orbot. (Apps that act as VPNs are not compatible with each other : you can only have one active at a time)
    • You can even configure several Wireguard interfaces at the same time, and route specific apps through specific tunnels
[–] catloaf@lemm.ee 11 points 3 months ago (1 children)
[–] Scipitie@lemmy.dbzer0.com 37 points 3 months ago

The first link goes into amazing detail on that. In short: all your information concerning location as well as current IP and some other metadata gets send to a basically unknown company with no transparency on how that data is handled.

I highly recommend reading the first, linked post though!

[–] Retro_unlimited@lemmy.world 9 points 4 months ago (2 children)

Can I just block the 3rd party company URL on my router?

[–] pcouy@lemmy.pierre-couy.fr 23 points 4 months ago (1 children)

You can, but you would not be able to display the map. Might as well disable the map server-wide

[–] Retro_unlimited@lemmy.world 1 points 3 months ago

I did not know you can disable the map, I will need to do that.

I only used the map once to see which photos may have accidentally had GPS Embedded, but turns out it was only the photos that other people took and sent me. lol

[–] MangoPenguin@lemmy.blahaj.zone 4 points 3 months ago

You could, but it's easier to just disable the map feature in Immich if you don't want to use it.

[–] Decronym@lemmy.decronym.xyz 4 points 4 months ago* (last edited 3 months ago) (1 children)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
DNS Domain Name Service/System
HTTP Hypertext Transfer Protocol, the Web
IP Internet Protocol
TCP Transmission Control Protocol, most often over IP
UDP User Datagram Protocol, for real-time communications
VPN Virtual Private Network
VPS Virtual Private Server (opposed to shared hosting)
nginx Popular HTTP server

6 acronyms in this thread; the most compressed thread commented on today has 10 acronyms.

[Thread #883 for this sub, first seen 24th Jul 2024, 15:15] [FAQ] [Full list] [Contact] [Source code]

[–] Caboose12000@lemmy.world 2 points 3 months ago
[–] N0x0n@lemmy.ml 4 points 3 months ago (1 children)

Wow perfect write-up ! Thanks for your hard work !

I hope someone will be motivated enough to translate it to Treafik ! I'm not versed enough to actually mess with the config files on such level... Let alone to understand Nginx's syntax 🫠 but very cool for Nginx users !!!

[–] pcouy@lemmy.pierre-couy.fr 3 points 3 months ago* (last edited 3 months ago)

I don't use Traefik myself, but this documentation page seems to suggest that Traefik only allows in-memory cache (which would eat RAM and not persist across reboots). You can probably run Nginx with this config inside a container for the caching, then use Traefik to handle requests to immich.your-domain.tld/map_proxy/* with the caching proxy container.

[–] TedZanzibar@feddit.uk 2 points 3 months ago (1 children)

This is excellent but alas I can't get it to work in nginx-proxy-manager. Keen to see if anyone else can figure it out.

[–] pcouy@lemmy.pierre-couy.fr 2 points 3 months ago

I don't use nginx-proxy-manager, but if you want to share what you tried, I will try to help you figure what's not working