I'd be willing to bet you bought at least a few years ago, and probably couldn't afford the house you're in now if you had to buy it today. I'm in a similar spot. It definitely feels wrong. The rapid increase in prices in the housing market in the past few years is ridiculous. I think it's a lot more complicated than "landlords" though. I think a lot of the issue stems from restrictive zoning that prevents the construction of small homes in dense neighborhoods. A lack of respect for trade jobs also contributes, with massive shortages of skilled construction workers driving prices up.

Granted, I live in a relatively affordable smaller city. If I were in a city with a lot of real estate speculation like LA or Toronto I might feel differently. But speculators aren't landlords. I have a much bigger beef with a speculator who let's a house sit empty than a landlord renting out apartments.