homelab

Thank you for that detailed description. I see two things which are of concern: the first is the IPv6 network unreachable. The second is the lost IPv4 connection, as opposed to a rejection.

So staring in order, the machine on the external network that you're running curl on, does it have a working IPv6 stack? As in, if you opened a web browser to https://test-ipv6.com/ , does it pass all or most tests? An immediate "network is unreachable" suggests that external machine doesn't have IPv6 connectivity, which doesn't help debug what's going on with the services.

Also, you said that all services that aren't on port 80 or 443 are working when viewed externally, but do you know if that was with IPv4 or IPv6? I use a browser extension called IPvFoo to display which protocol the page has loaded with, available for Chrome and Firefox. I would check that your services are working over IPv6 equally well as IPv4.

Now for the second issue. Since you said all services except those on port 80, 443 are reachable externally, that would mean the IP address -- v4 or v6, whichever one worked -- is reachable but specifically ports 80 and 443 did not.

On a local network, the norm (for properly administered networks) is for OS firewalls to REJECT unwanted traffic -- I'm using all-caps simply because that's what I learned from Linux IP tables. A REJECT means that the packet was discarded by the firewall, and then an ICMP notification is sent back to the original sender, indicating that the firewall didn't want it and the sender can stop waiting for a reply.

For WANs, though, the norm is for an external-facing firewall to DROP unwanted traffic. The distinction is that DROPping is silent, whereas REJECT sends the notification. For port forwarding to work, both the firewall on your router and the firewall on your server must permit ports 80 and 443 through. It is a very rare network that blocks outbound ICMP messages from a LAN device to the Internet.

With all that said, I'm led to believe that your router's firewall is not honoring your port-forward setting. Because if it did and your server's firewall discarded the packet, it probably would have been a REJECT, not a silent drop. But curl showed your connection timed out, which usually means no notifications was received.

This is merely circumstantial, since there are some OS's that will DROP even on the LAN, based on misguided and improper threat modeling. But you will want to focus on the router's firewall, as one thing routers often do is intercept ports 80 and 443 for the router's own web UI. Thus, you have to make sure there aren't such hidden rules that preempt the port-forwarding table.