Hi LgEnder,
The effect of adding a memory foam layer underneath another material can be very unpredictable because the layer above it will isolate the memory foam from your body heat which will make it firmer than it would be if you were sleeping directly on the memory foam. It will also depend on the thickness and on the type of memory foam and its specific properties (see post #9 here and post #8 here about the many differences between different types of memory foam).
I don’t know anything about your body type or sleeping positions or preferences but post #2 here and the topper guidelines in post #8 here may also be useful. If you can exchange your latex topper for a softer version it may be a less risky and less costly approach to softening up your mattress than buying new layers.
Phoenix