A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
You could always just get the entire list of AWS public ranges and use that to identify interesting traffic.... Of course that would vpn all AWS stuff.
If you have a list of domains, you could schedule a script to resolve each to IPs and rebuild your VPN definition every night.
What exactly are you trying to split out? Generally, routing is done at the IP level, not by name.
There's a few apps I need to split out. Top priority is the signiant app which according to their documentation requires various AWS subdomains as well as their own. Specific subdomains are not specified and are implied to change regularly/on demand.
In an ideal world I would do my split tunneling on the device itself, but I don't trust Windows and thus I run my VPN at the router level.
This isn't a problem for most things, but I need to utilize my full bandwidth to transfer large files to clients in a timely manner, and a VPN becomes a massive bottleneck.
Pfsense lets you alias by domain name (I believe it regularly resolves down to an IP and uses that for filtering), but again, you need to supply the exact subdomain.
Just wondering if there's an alternative solution to this issue. If it's external to pfsense that's not the end of the world.
Worst case scenario, I would set up a dedicated Linux box or maybe even a VM which could share access to the file transfer NAS and split tunnel the entire box around the VPN. Definitely less convenient.
Can you configure those apps to use a proxy? You could set up a tiny little VM or container that has direct access, and upload through there.
I think the way people do it is by making a script that gets the hostnames and updates the alias, then just schedule it in pfsense. I've also seen ASN based routing using a script, but that'll only work on large services that use their own AS. If the service is large enough, they might predictably use IPs from the same CIDR, so if you spend some time collecting the relevant IPs, you might find that even when the hostnames are new and random, they always go to the same pool of IPs, that's the lazy way I did selective routing to GitHub since it was always the same subnet.
Your first suggestion is a clever one.
I can imagine writing a small script on the host machine to listen for subdomains, forward them to pfsense to update the aliases, and possibly set them to expire after a few days for security reasons.
Surely something like this exists. How to find it...?
If you search for pfsense alias script, you'll find some examples on updating aliases from a script, so you'll only need to write the part that gets the hostnames. Since it sounds like the hostnames are unpredictable, it might be hard as the only way to get them on the fly is to listen for what hostnames are being resolved by clients on the LAN, probably by hooking into unbound or whatever. If you can share what the service is it would make it easier to determine if there's a shortcut, like the example I gave where all the subdomains are always in the same CIDR and if one of the hostnames is predictable (or if the subdomains are always in the same CIDR as the main domain for example, then you can have the script just look up the main domain's cidr). Another possibly easier alternative would be to find an API that lets you search the certificate transparency logs for the main domain which would reveal all subdomains that have SSL certificates. You could then just load all those subdomains into the alias and let pfsense look up the IPs.
I would investigate whether the IPs of each subdomain follow a pattern of a particular CIDR or unique ASN because reacting to DNS lookups in realtime will probably mean some lag between first request and the routing being updated, compared to a solution that's able to proactively route all relevant CIDRs or all CIDRs assigned to an ASN.