Hi joemokins,
I also like the “feel” of latex over memory foam but any topper will reduce the amount of heat that reaches the memory foam underneath it and can make it firmer than it would be if you were sleeping directly on the memory foam.
If you are “close to perfect” that means you are very close and I would look at some of the things that can fine tune your sleeping system frst.
One of these is any layers that are over your mattress (such as a mattress protector, mattress pad, or even your sheets and bedding) that may interfere with the ability of the foams below it to “allow” your shoulders to sink in. A pillow that is too thin can also contribute to shoulder pressure. Post #2 here would also be worth reading.
If these suggestions aren’t effective then a slightly softer latex topper may give you the improvement you need but this would mean that you would need to make a new purchase so it’s not particularly cost effective because most toppers aren’t returnable.
If you need “just a touch to a little” more softness then an additional inch or so of softer materials or foam may be just enough to make the difference. There are some guidelines in post #2 here about making the most suitable topper choices.
Phoenix