Have you considered buying a used thin client / mini PC?
Since the Orange Pi 5 sounded interesting, I checked it out and it's rather expensive. Well at least for me.
I found it from 180 to 200 Euro for the 8GB version. An used thin client or mini PC can be bought for half the price and most of the time it comes with 256gb storage included.
The extra cost for energy should be less than what you might be paying extra for the Orange Pi plus storage.
Modern Java isn't that bad, and with new developments like the graalvm and cloud native builds, or what they are called, the footprint of a modern Java app can be comparable to an golang app.
Modern Java kinda has the same image problem as modern PHP. Not saying is all great, but it sure has seen quite the improvements in the last years