We don’t necessarily, it depends on the extent and type of bacteria causing the mould, your personal susceptibility to that bacteria, and indeed how your body handles exposure – such as digestion.
It is a bit of a misconception that mould is bad in absolute, but for the ease of removing obvious mould from food – such as cutting the mouldy bit off some generally un mouldy cheese – you may as well do it.
Latest Answers