Burning it all is arguably much worse than having landfills. Burning random garbage will generate an assortment of toxic gases, and also a large amount of CO2.
Now some garbage is actually burnt in incineration plants, but those have filters and such to stop the worst of the resulting gases from getting into the atmosphere.
Latest Answers