I’ve wondered this for a while and a quick Google hasn’t yielded great results.
When films, particularly full CGI films, are made, do they truly make it frame by frame? Or do they desgin the full 3d world and put cameras into the software to “film” the scene? And how extensive to they get?
For example, in the car chase ending for Toy Story, would they have designed and modeled the entire neighborhood and used “cameras” in the scene to film it? Would they fully 3d model each house and “play” the scene to film it? Or would they design and create small pieces and put them together like traditional cartoon?
I always imagined it was similar to the StarCraft map editor. I could build the map, add in units and then assign them AI cues to move through routes or take actions based on programmed cues. Very curious how it works on big budget films.
In: 331
The short answer is that we create a ‘world’ but the long answer is that it really depends on a lot of factors!
Since you’re interested I’d be happy to explain the entire process as maybe this will help you and other people to understand how VFX works.
First we start off with the Plate – this is what we call the individual Shot in the film. It’s what gets shot in camera. For our example let’s say it’s two dudes standing on a street looking across the road at a building… but there’s no real building there.
So the plate for our example is two guys on the close side of the road looking at a big bluescreen thing, which we need to make some cool building that’s never existed.
The first thing the VFX team will do once they get the plate is to Track the Camera. This means we make a digital copy of the camera in a 3D program, and we’ll also add things like a 3D floor and some simple landmarks of other buildings and objects in the plate. The software used for this is pretty smart and does a lot of calculations to figure out the camera and locations geometry. But it can be a tricky process but the result of a 3D camera in a program like Maya, Houdini or Blender, which is a scale accurate copy of the real world camera. Once tracking is done we can add CG objects that stick to the real world plate … so if I add a ball onto the ground in my 3D program then this object could be rendered and put into the Plate and it would stick to the real world even if the camera moves.
The next step is to do some quick concept art to show the director what we’ll put into the shot. This is usually someone drawing a single frame in Photoshop or something like that. The idea is to make a quick mock up so we can make changes quickly and cheaply – doing quick draw overs in Photoshop is much faster than doing all the 3D stuff we need to make something real so this step helps us all agree on the scope of the work. I will add that often concept work is actually done before shoot, and before tracking, but I wanted to list it here cause the main thing is we try do things quick broad strokes idea work on single frames. This isn’t a “whole world” it’s just a drawing related to the single frame. But it’s fast so we don’t mind at this stage.
Once the director approves the concept (which can take some revisions) we have a Modeller make a 3d Model of the building, based on the concept.
Modelling can be simple or complicated depending on how complex the asset you need to make is. We also have libraries to help with common objects to help and programs that an build things like trees and bushes. Some objects require detailed sculpting which is high res detailed modelling done in a program like zBrush. There’s a lot of specialisation within modelling, some people just do creatures and humans while others only do Hard Surface modelling like cars and buildings. In either case the end result is the ‘Mesh’ of a CG object.
Everything gets modelled and then we begin to put all the models together into the tracked world.
We also might want to add People or Animated things, maybe some cars driving past on the road or a dog or hell, maybe the. building is supposed to explode! These new objects all need to be modelled too. Once we’ve modelled things which need to move, they get Rigged. This is a process where we add controls so you can move stuff – often for creatures and people this looks like adding a skeleton and muscles, all in CG, which control the character. The people who do this are referred to as Riggers or Character Technical Directors (Character TDs). And then some basic blocking animation is done by Animators.
This step is sort of called Layout.
The key thing with layout is some simple animation and simple models are all put into the scene with the Camera track and suddenly you have CG in a real world scene. It doesn’t look good yet but it’s all kinda there.
We get approval from the director on this, and make changes as necessary.
While that’s happening we are concurrently making all the objects look pretty, because up until now they are only a ‘mesh’ but have no colour.
So next up is texturing artist and/or surfacing artists who add textures to the building. Things like the concrete, the smudges on the windows, all that kind of thing. It’s the surface of all the materials. This gets pretty detailed too and we need to think about the whole movie and if we get really close to the object or only see it from far away as that determines how much detail we need to add. Surfacing needs to happen to every object in the scene so that it reacts to light in the correct way. This process is usually referred to as Look Dev, although the nomenclature gets a bit confusing for reasons I won’t go into.
Anyway … now we have Layout and the Look Dev approved so we can head into Animation and Lighting.
During this step animation is refined with small details, the cars now move beautiflly with people inside them actually steering. CG characters walk around. If we do ned an explosion that happens by specialists called CGFX Artists who do special simulation work to explode and break things. All those things now get signed off.
With animation done we add lighting to the building based on some pretty clever data we captured during the shoot of the plate – this helps everything feel like it’s matches. This is a whole complex field but in the end we have a CG building that looks like it really exists in the shot.
The lighting team takes the lit 3D scene and they render it – the output of this is the lit CG objects with a bunch of special layers that are used by artists called Compositors, and their job is to combine the Plate and the CG into a single, seamless, shot. We often use something called Rotoscoping (just Roto in the biz) which helps us cut out the foreground people from the background, so we can put things behind the foreground. In this case we might get roto of our guys standing on the street, and the compositor might use a technique called chroma-keying to make the bluescreen disappear. We often use both these methods, but the result is we cut parts of the image out so we can slot things like the CG building into the image.
This same process is repeated for every shot of the building in our scene.
If the building is seen from multiple angles we might plan it out carefully BEFORE the shoot. We also tend to tryand work on multiple shots at the same time to make sure the building looks good from all the angles – it’s a 3D asset so it’s kinda real in a digital sense, and we want to make sure if the building is seen from the side and front and back that all those things make sense in the new real combined CG + Real world we’ve created.
Sometimes we have to tweak the building for specific angles, or variations. So assets like the building can have a lot of complexity and change from shot to shot. But we have a lot of tools to make handling these situations easier. Shooting films can take place over multiple days so lighting conditions and weather conditions might change, among other things.
We also split the assets up. So the chairs and tables outside the building might get reused in another locations CG. We try to reuse stuff when we can but it’s a little complicated.
Where possible we try to work on Scenes rather than individual shots. So we will track all the shots in a scene, do layout for all the scene, then animation and then lighting at the scene level. This helps ensure continuity is accurate across all the shots and the whole team is working in the same way. We try to watch the Scene as a whole to make sure everything feels natural and invisible and hopefully the audience never knows we did anything.
Finally I’ll add that some of the tasks above are definitely done on a per-shot level, while others are done at the asset or scene level. Per-shot work is usually: tracking, animation, roto, lightning and comp. Asset level would be things like modelling and surfacing, usually that’s not changed shot to shot, although sometimes tweaks are required.
And that’s basically how it works!
Feel free to ask me anything about it – I love explaining this stuff to people! I’m a VFX supervisor and mod over on r/VFX and you can find out heaps of info about this over there too 🙂
Latest Answers