I’ve wondered this for a while and a quick Google hasn’t yielded great results.
When films, particularly full CGI films, are made, do they truly make it frame by frame? Or do they desgin the full 3d world and put cameras into the software to “film” the scene? And how extensive to they get?
For example, in the car chase ending for Toy Story, would they have designed and modeled the entire neighborhood and used “cameras” in the scene to film it? Would they fully 3d model each house and “play” the scene to film it? Or would they design and create small pieces and put them together like traditional cartoon?
I always imagined it was similar to the StarCraft map editor. I could build the map, add in units and then assign them AI cues to move through routes or take actions based on programmed cues. Very curious how it works on big budget films.
In: 331
Your suspicion is correct. They do indeed create the “world” and place the characters, lights, and virtual cameras inside it.
The camera can be moved similarly to a real camera, however since it takes up no physical space, it can achieve shots that a real camera never could.
It’s a combination of the two.
You start with some generated scene using models that are put through a series of motions, and then you go in frame by frame as needed and add detail that you can’t have in that model generated world.
There are a few different technologies that go into CGI. Usually, what you see on screen is a mix of them.
So you might have a program that specifically simulates destruction physics, or a program that specifically does water/cloth. You might also do simulation-based animations – or maybe you’ll do motion capture, and then use that data to drive an animation.
Sometimes the simulators allow you to apply textures on top of them. Sometimes they just export raw data, and you’ll have to start applying shaders, textures – basically everything that makes it pretty. Usually, when working on them, they’ll look pretty bad because making photorealistic images is VERY resource-intensive, and it’s faster to do work with simplified visuals and only apply the fancy pictures at the very end.
Then once everything’s done, you’ll throw it into a renderer that basically does all that. You’ll take the simple shapes, and start applying textures so that it looks real.
Just like a Hollywood set, they model out the area that will be in the shot, the further away the lower the detail. Animation is keyed, or key framed, you set an objects location, rotation, and scale, then set it again further in time. The software calculates what happens in between, and the animator adjusts and readjusts.
There are camera objects in each scene, they operate/simulate real world cameras, you can use all the same settings, lenses, etc.
AI, procedural mapping, simulations, physics, can do a ton of the work. You don’t animate a crowd, you more set the number of people, adjust locations, and tweak. All the background motion is calculated based on user settings, you don’t animate leaves on trees individually, or even model the trees themselves, it’s procedurally generated based on settings.
The short answer is that we create a ‘world’ but the long answer is that it really depends on a lot of factors!
Since you’re interested I’d be happy to explain the entire process as maybe this will help you and other people to understand how VFX works.
First we start off with the Plate – this is what we call the individual Shot in the film. It’s what gets shot in camera. For our example let’s say it’s two dudes standing on a street looking across the road at a building… but there’s no real building there.
So the plate for our example is two guys on the close side of the road looking at a big bluescreen thing, which we need to make some cool building that’s never existed.
The first thing the VFX team will do once they get the plate is to Track the Camera. This means we make a digital copy of the camera in a 3D program, and we’ll also add things like a 3D floor and some simple landmarks of other buildings and objects in the plate. The software used for this is pretty smart and does a lot of calculations to figure out the camera and locations geometry. But it can be a tricky process but the result of a 3D camera in a program like Maya, Houdini or Blender, which is a scale accurate copy of the real world camera. Once tracking is done we can add CG objects that stick to the real world plate … so if I add a ball onto the ground in my 3D program then this object could be rendered and put into the Plate and it would stick to the real world even if the camera moves.
The next step is to do some quick concept art to show the director what we’ll put into the shot. This is usually someone drawing a single frame in Photoshop or something like that. The idea is to make a quick mock up so we can make changes quickly and cheaply – doing quick draw overs in Photoshop is much faster than doing all the 3D stuff we need to make something real so this step helps us all agree on the scope of the work. I will add that often concept work is actually done before shoot, and before tracking, but I wanted to list it here cause the main thing is we try do things quick broad strokes idea work on single frames. This isn’t a “whole world” it’s just a drawing related to the single frame. But it’s fast so we don’t mind at this stage.
Once the director approves the concept (which can take some revisions) we have a Modeller make a 3d Model of the building, based on the concept.
Modelling can be simple or complicated depending on how complex the asset you need to make is. We also have libraries to help with common objects to help and programs that an build things like trees and bushes. Some objects require detailed sculpting which is high res detailed modelling done in a program like zBrush. There’s a lot of specialisation within modelling, some people just do creatures and humans while others only do Hard Surface modelling like cars and buildings. In either case the end result is the ‘Mesh’ of a CG object.
Everything gets modelled and then we begin to put all the models together into the tracked world.
We also might want to add People or Animated things, maybe some cars driving past on the road or a dog or hell, maybe the. building is supposed to explode! These new objects all need to be modelled too. Once we’ve modelled things which need to move, they get Rigged. This is a process where we add controls so you can move stuff – often for creatures and people this looks like adding a skeleton and muscles, all in CG, which control the character. The people who do this are referred to as Riggers or Character Technical Directors (Character TDs). And then some basic blocking animation is done by Animators.
This step is sort of called Layout.
The key thing with layout is some simple animation and simple models are all put into the scene with the Camera track and suddenly you have CG in a real world scene. It doesn’t look good yet but it’s all kinda there.
We get approval from the director on this, and make changes as necessary.
While that’s happening we are concurrently making all the objects look pretty, because up until now they are only a ‘mesh’ but have no colour.
So next up is texturing artist and/or surfacing artists who add textures to the building. Things like the concrete, the smudges on the windows, all that kind of thing. It’s the surface of all the materials. This gets pretty detailed too and we need to think about the whole movie and if we get really close to the object or only see it from far away as that determines how much detail we need to add. Surfacing needs to happen to every object in the scene so that it reacts to light in the correct way. This process is usually referred to as Look Dev, although the nomenclature gets a bit confusing for reasons I won’t go into.
Anyway … now we have Layout and the Look Dev approved so we can head into Animation and Lighting.
During this step animation is refined with small details, the cars now move beautiflly with people inside them actually steering. CG characters walk around. If we do ned an explosion that happens by specialists called CGFX Artists who do special simulation work to explode and break things. All those things now get signed off.
With animation done we add lighting to the building based on some pretty clever data we captured during the shoot of the plate – this helps everything feel like it’s matches. This is a whole complex field but in the end we have a CG building that looks like it really exists in the shot.
The lighting team takes the lit 3D scene and they render it – the output of this is the lit CG objects with a bunch of special layers that are used by artists called Compositors, and their job is to combine the Plate and the CG into a single, seamless, shot. We often use something called Rotoscoping (just Roto in the biz) which helps us cut out the foreground people from the background, so we can put things behind the foreground. In this case we might get roto of our guys standing on the street, and the compositor might use a technique called chroma-keying to make the bluescreen disappear. We often use both these methods, but the result is we cut parts of the image out so we can slot things like the CG building into the image.
This same process is repeated for every shot of the building in our scene.
If the building is seen from multiple angles we might plan it out carefully BEFORE the shoot. We also tend to tryand work on multiple shots at the same time to make sure the building looks good from all the angles – it’s a 3D asset so it’s kinda real in a digital sense, and we want to make sure if the building is seen from the side and front and back that all those things make sense in the new real combined CG + Real world we’ve created.
Sometimes we have to tweak the building for specific angles, or variations. So assets like the building can have a lot of complexity and change from shot to shot. But we have a lot of tools to make handling these situations easier. Shooting films can take place over multiple days so lighting conditions and weather conditions might change, among other things.
We also split the assets up. So the chairs and tables outside the building might get reused in another locations CG. We try to reuse stuff when we can but it’s a little complicated.
Where possible we try to work on Scenes rather than individual shots. So we will track all the shots in a scene, do layout for all the scene, then animation and then lighting at the scene level. This helps ensure continuity is accurate across all the shots and the whole team is working in the same way. We try to watch the Scene as a whole to make sure everything feels natural and invisible and hopefully the audience never knows we did anything.
Finally I’ll add that some of the tasks above are definitely done on a per-shot level, while others are done at the asset or scene level. Per-shot work is usually: tracking, animation, roto, lightning and comp. Asset level would be things like modelling and surfacing, usually that’s not changed shot to shot, although sometimes tweaks are required.
And that’s basically how it works!
Feel free to ask me anything about it – I love explaining this stuff to people! I’m a VFX supervisor and mod over on r/VFX and you can find out heaps of info about this over there too 🙂
There’s a Youtube channel / website of some people who do VFX breakdowns they also tell you how certain things are done and invite people from the industry [Corridor Digital](https://www.youtube.com/channel/UCSpFnDQr88xCZ80N-X7t0nQ)
You should watch the documentary series Light & Magic on Disney+. Amazing doc on the history of Industrial Light & Magic that covers their history and how special effects have evolved from shooting with miniatures and against matte paintings to the CGI revolution. An absolute must-see for those who love finding out how movies are made and are addicted to the extras on DVDs that go into all the making-of and behind the scenes stuff with filmmaking.
Even at age five, I think you would LOVE watching [Corridor Crew](https://www.youtube.com/@CorridorCrew) on YouTube for an hour. They’re digital artists who review scenes from TV and movies and discuss how they’re made, sometimes with artists and stunt performers from the actual movie as a guest. You really, really ought to just watch a couple episodes or some of their YT shorts.
Sometimes it’s an entire world like a video game. There was a 14-yr-old boy who recreated the entire [Spider-Man: Into The Spiderverse](https://youtube.com/shorts/ikme2icAU50?si=vu_CbOmWyoU5jqAP) trailer in computer software, in less than a month. Sometimes it’s real people and things in front of a background that’s all CGI. Sometimes it’s real people in front of a projection on a huge screen that wraps around the set. Sometimes it’s [people in green bodysuits operating puppets](https://youtu.be/UZl6uAu1-fY?si=_KlXDbBSNtyQXwQS) that touch human actors and move physical things on set, processed with color-replacement. It’s a great channel.
These days they build a world and render the frames one at a time. They used to glue bits together but it was a pain in the arse and computers got heaps bigger.
Latest Answers