Oddly I think the only cases I ever used it where I was connecting to my home computer from outside my house was when I needed to connect to my router's webpage. SSH to my home computer and then pull up the browser to open a port on my DMZ or other such nonsense.
When at home and just using LAN bandwidth it was to run lesser programs.
I think the guy you're thinking of Fox Mulder