Eli5: Why and how does our brains fill in colours into images that are not there at all?

234 viewsOther

This picture only has black, white and cyan.
How is the [can](https://i.imgur.com/DfRNh0z.png) red? Zoom in to check.

In: Other

3 Answers

Anonymous 0 Comments

This image is using CYMK colour, basically – take a look at the [CYMK](https://plumgroveinc.com/wp-content/uploads/featured-cmyk-color-versus-rgb-color-1280×730-thegem-blog-default.jpg) colours here, and notice how opposite cyan is the red segment.

The inverse of cyan is red; in that part of the image, there is no cyan present, so we assume that there must be red there that’s cancelling it out. Zoom in on the hand and you can see that when cyan is present, the whites look reddish. We don’t just use basic visual information gathered through our eyes to assign colour, but also contextual clues from the image – that’s what caused the issue with the gold and white/black and blue dress years ago – [link](https://upload.wikimedia.org/wikipedia/en/2/21/The_dress_blueblackwhitegold.jpg). Depending on how you visualised the way the overall image was lit, it affected your perception of what colour the dress would be under that light.

You can see a good example with this [Rubik’s cube illusion](https://www.researchgate.net/publication/370213289/figure/fig1/AS:11431281153068848@1682322633045/The-color-cube-illusion.jpg). The two middle squares on the top and facing face you can see are actually the same colour; however, because we think the one on the front is in shadow, we perceived it as being much brighter and more vivid in comparison to all the squares around it, and therefore to the brown square on top too.

Think of how when your room gets dimmer in the evening, you can still tell what colours things are, as you can use the information that the ambient light is lower to work it out.

Anonymous 0 Comments

The brain (and technically, even before that, the nervous system on the way to the brain) does a huge amount of preprocessing and heuristics in order to turn the vast amount of raw data you get from your eyes into actually useful information.

*Most* of what you think you see is not literally what you see. It is a post-processed representation.

In this context, one of the big things that the brain relies on is contrast and extrapolation. It doesn’t tell you “this is color code A and that is color code B”. It sees a region that’s roughly kind of one color, sees another with a high contrast, and uses that to infer the color of the other region. (ETA: specifically, in this case, we see “red” because that is an inferred contrast color to “cyan”.)

The immediate reason it does this is that this is how brains evolved.

The detailed reason of why we might have evolved this way is what is beneficial in a natural environment. In particular, as relevant to this, a lot of the “optical illusions” that we experience are useful heuristics when dealing with natural colors in *changing light conditions* – particularly daylight vs. shadows vs. night-time. Often, shadows create areas where you can’t see all the relevant colors, but you can see contrast; and it is beneficial to be able to make good “guesses” based on that contrast.

Anonymous 0 Comments

Vision isn’t what photons hit your retinas, that’s just the start of it. Your brain has to interpret all of these signals coming in through your optic nerves. We see different colors based partially on the wavelengths of lights triggering different photoreceptors in the eyes, but that’s only part of it. If we saw purely based on the wavelengths, things would look like dramatically different colors based on the type of lighting, whether it’s in shadow, how much light is hitting it, etc. While this is a little noticeable (think about something looking a bit different outside vs in fluorescent lighting), colors don’t look drastically different if you put a shadow in front of something.

When these signals reach the brain, they start in the thalamus, then the information is moved to the visual cortex. From there, it is split up in to different areas that process different information, such as processing color, motion, depth and form, although no area does one of these exclusively. This all gets combined to give you an idea of the shape and color that you are seeing. We don’t know exactly how it all comes together, though.

One of the things that can happen during this is that color perception can be affected by borders – basically, you get a bleeding of a color difference as your brain is working out the borders of different areas. This can go one of two ways, either the opposite (contrast), or similar (assimilation). In your example, the broken border is creating a contrast effect. Your brain, while working out all of those pixel edges, is filling in color that is the opposite of the color fill in the rest of the picture. If you concentrate and focus on just the area with the illusory red, it goes away – it’s when you try to process the picture as a whole that it makes your brain fill in those colors.