Each color of light has a particular energy. Some ingredients in some substances have the ability to absorb energy that is in the range of certain colors. Which energy levels these are is very complicated and would be hard to predict just from knowing the ingredients. The same ingredient in different substances can absorb different energies (and therefore different colors).
(That is my ELI5 version. If this were r/askscience, I would talk about non-bonding d-orbital electrons and such)
Source: used to teach this stuff to Mineralogy students.
Latest Answers