There are quite a few foams. There’s latex, polyfoam, memory foam, hybrid, and different variations of polyfoams and memory foams. Low density polyfoam and memory foam are what im not a fan of so i get where youre coming from on that. Only in certain configurations, such as with the glacier apex i bought back when i did not know as much about beds. It did offgas for a little while, but i got lucky on that bed since it actually feels amazing. Somehow it’s memory foam but doesnt give the same sunk in feel of a typical foam bed and is still quite firm and supportive.
Anyway, you are correct in your thoughts about firmness being necessary. What’s important is the nature of the firmness and how far down into the mattress the stabilization point occurs between your body and the materials. Ive laid on firm beds and just felt “blegh” because i think somehow your body can feel when something is made cheaply.
My favorite base is a hybrid spring base, the quadcoil system by texas pocket springs specifically. The 15.5g should work perfect for you. It does compress, but because the coils are packed into pockets of 4 each, they dont fall over or distort like a typical coil system might. it’s very supportive, but not hard.
The spring layer is 8", and you definitely dont need anything more than a 12" mattress, which is considered a medium height. If you get a 12" encasement, that leaves you 4" of room for your transition layer and comfort layer. If your encasement is either a touch taller, or a small touch shorter than your total layer height, that is generally okay. Though i wouldnt recommend going any more than .5" over the encasement height.
If you go under, that’s more acceptable because the encasement is just for covering the mattress. Going over too much will firm up the whole bed because itll become too tight.
Latex is a good option, you could put a 2" layer of medium and then a 2" layer of soft on top of the springs
Or high density poly foam is actually good in my opinion. Foambymail sells HD36-HQ which is 35 ild. You could put a 2" layer of that and then either a 1.5" layer or a 2.5" layer of their convoluted HD36-HQ on top for the comfort layer, which comes out to about 20ild because its convoluted, so that gives you both the comfort layer and the transition layer with the same foam.
This is why DIY is great. You can replace layers and shift things around to find what’s perfect for you.