To answer this question we must know something about temperature physics. A common idea is that heat and temperature are the same. This is not the case. Heat is (a form of) energy or, more specifically, a measurement of some of the energy in a substance. Temperature is a number related to the (kinetic) energy of the particles in a substance, but it is not energy itself.
We use water as an example. By adding energy in the form of heat to water, the average kinetic energy of the particles increases (which translates into an increase of the speed at which the particles travel). This increase in kinetic energy is measured by a number we call temperature and which changes proportionally with the kinetic energy. So by adding heat, the temperature rises.
However, if we add heat to ice something different happens: it changes state from solid to liquid, but the temperature stays the same. The energy added in the form of heat is absorbed by the change of state (by changing the bonding of molecules), but it does not increase the kinetic energy of the particles. So melting ice does not rise in temperature if heat is added.
The second aspect we have to understand is how heat (thermal energy) transfers: it flows from regions with higher temperature to regions of lower temperature. If the temperatures of two regions are the same, no heat is transferred between them. Also, the speed at which heat flows depends on the conductive properties of the medium: a high conductivity means heat will travel faster from a higher temperature region to a lower temperature region, while a low conductivity means heat will travel relatively slow. Second, a high temperature difference leads to a faster transfer of heat than a low temperature difference.
Now we know these aspects of heat and temperature, we look at the human body as a temperature measuring instrument. First, while the average body temperature is 37 °C, the temperature differs between organs. Normal skin temperature is 33 °C, liver temperature is 39 °C. One could say that if an object doesn’t feel warm or cold, its temperature is 33 °C. However, our body does not measure temperature, but it detects the flow of heat into or from our body. So an object with a temperature lower than 33 °C feels cold, because it extracts heat from our body. An object with a temperature higher than 33 °C feels warm, because heat flows from it into our body.
As explained above, the bigger the temperature difference and/or the higher the conductivity of the material, the greater the heat flow. An iron fence in winter feels much colder than a wooden fence next to it, while the temperature is the same; heat from our skin transfers faster into iron than into wood because of its higher heat conductivity. It works the other way around as well: when using a metal kitchen spatula heat travels faster from the frying pan through the iron into our body than when using a wooden kitchen spatula. We feel the transfer of heat, not the difference in temperature. An extreme example is the thermal protection of the Space Shuttle. You can safely hold a glowing 1260 °C piece of the material in your hand. Its heat conductivity is so low that thermal energy travels extremely slow into or from the material.
Also, the bigger the difference in temperature between the skin and the fences, the bigger the difference in heat transfer we feel between each fence. If both fences have a temperature of 30 °C, the iron feels colder than the wood but only a little bit. If the fences have a temperature of 5 °C, the difference we feel between the iron and wood is much larger.
Another reason why our body is not a good temperature measuring instrument, is because our body feels heat or cold by using itself as a comparison. It does not have an accurate objective standard to refer to. This is illustrated by the following example. Person A holds his hand in water with a temperature of 5 °C, while person B holds his hand in water with a temperature of 40 °C. If after a while they both put their hand in water of 20 °C, that water will feel warm to person A and cold to person B.
Three other aspects reduce accurateness in temperature measurement by our body. Body parts such as eyelids and cheeks are more sensitive to changes in temperature than e.g. hands (which have a better developed tactile sensitivity). Second, accuracy in detecting changes in temperature increases if a larger part of the body is exposed to the temperature change. When testing the temperature of a warm bath you get a better judgement when putting in your whole arm than just a fingertip. Last, heat is detected by special nerve endings. The nerve fibres themselves cannot detect differences in heat. They are affected by heat and cold, but can only sense pain caused by it. Pain travels much faster to the brain and pain signals are much stronger than temperature impressions (especially in the hands, where temperature sensitivity is less developed than tactile sensitivity). So in cases of extreme hot or cold, it is difficult to say whether it is either hot or cold. In both cases pain is the dominant (or only) sensation experienced.
In conclusion: while the human body is very good at recognising changes in temperature (or detecting heat flow), it is very poor at determining the absolute degree of temperature.