Hi wheezer,
Out of the three types of foam … memory foam is the most problematic for people who are sensitive. I can also tell you from personal experience that the symptoms that come from a sensitivity to the chemicals in some memory foams are not pleasant and can be quite scary. While I’ve only had the unfortunate experience with one memory foam product (a topper from Novaform), the burning throat, dry mouth, muscle aches, fatigue, palpitations, headaches, nausea, strange dreams, and general listlessness were not fun to go through. They subsided within 48 hours after removing the topper. Both of us experienced similar symptoms so this was an indication of a “bad” memory foam product but some people are sensitive enough that it seems that all (or at least most) memory foam products affect them. Neither one of us has had similar experiences with other memory foam mattresses we have slept on.
Polyfoam is less of an issue (although some people are very sensitive to this as well) while latex is generally considered to be the “safest” of the foams with the least likelihood of sensitivities.
Post #4 here along with post #2 here and post #2 here has much more information and links to resources about the “safety” of different materials that you will find in a mattress including memory foam.
Hope this helps.
Phoenix