A fever is your body’s attempt to help kill a pathogen by raising your body heat.
The thermostat in your house is set to keep your home in a certain heat range. Mine is set to 65-72 degrees. When it falls below that heat range it kicks on the heater. When it gets too far above that heat range it kicks on the AC. I could if I wanted set it to 75-85. Then it will kick on the heater when it is 73 even when previously that temperature was considered too hot.
When something that would normally feel room temperature feels too cold when you have a fever, that’s your body’s way of raising your body heat to the new hotter temperature it’s looking for.
Yes, the room temperature is neutralizing the heat. That’s the problem as far as your body is concerned. It’s telling you “That’s too cold!” to motivate you to get out of that room temperature water and some place more warm so it can continue to try and cook off whatever is infecting you.