Imbedded beneath the outside walls of the refrigerator are little heaters called 'mullion heaters' which are designed to prevent condensate from forming on the exterior surfaces (sweating). If the humidity in your kitchen is low enough that there is little danger of your refrigerator sweating, simply toggle the 'energy saver' switch off. That will turn off the 'mullion heaters' and save a little electricity.

If you can't find an 'energy saver' switch on your refrigerator's control panel, don't sweat it: some refrigerators have them and some don't.

At any rate, warm refrigerator exterior panel surfaces are usually not a problem.