For example, cropping off a person in a picture and you see a wall that was hidden behind them in the original picture.
In: Other
They guess based on the surroundings of the crop because there is no actual data for what’s behind it.
The _how_ is very complicated and these days involves lots of machine learning, but the general idea is that they look for patterns near the area being removed and infer details based on the patterns.
Old algorithms would try to continue nearby patterns across the removed area. This worked pretty well when the background was a simple repeating pattern, like brickwork or grass, but did really poorly when important things were obviously changing behind the removed object.
With machine learning, the model can infer the existence of more complicated backgrounds given clues. For example, if it sees a doorframe surrounding a person that you want to remove, it can infer the existence of a door behind the person and fill in details accordingly. It “knows” about doors and doorframes because it was trained on many pictures with doors in doorframes and has stored that information in some format during its training.
Machine learning models do not “understand” photos the same way that humans do, so the exact way that this happens is poorly understood and often unreliable. It generally creates better results than earlier “copy a pattern” algorithms did.
Latest Answers