CG Elements

With the virtual environment created and the footage shot and tracked, it then came time to populate the scene with CG elements.
The over all look that I am going for with this shot is a kind of fantasy futurist vision of Bristol. So, naturally there will be flying cars and large buildings. The composition of the original shot allowed for a number CG elements to be discretely placed within the scene. Here is a small example of a few of the elements I am currently working on and how they will fit into the final shot.

The above image is a tower that I modelled a while ago, but for this project there have been a few issues with texturing which I am currently working through.

The tower is situated amongst other sky scrapers at the end of the street and into the distance.

All of these elements have been lit using the Mental Ray Physical sun and sky illumination. This gives a very realistic representation of where the sun is placed and how it illuminates the geometry. All elements are matted making the final comping job possible.

As promised, a flying car. All elements reflect the environmental geometry, with the environments primary visibility hidden.

Once all these elements were in place I did a quick playblast to check the track and how it all fit together.

Nelson St. Rebuild

Once the shoot had taken place and we got the shot we wanted in the can, it was then onto the arduous process of post production. As the hard work had been put in during pre production, the tracking process was relatively easy. The techniques I used can be seen on one of my earlier posts, although for this particular shot I actually ended up using the Nuke X camera tracker instead of Autodesk Matchmover. For some reason Matchmover was having a hard time solving. There were a few small issues with exporting the camera track from Nuke to Maya, but luckily I happened upon a neat little python script that did a good job of exporting it for me. That particular script can be found here: http://www.nukepedia.com/gizmos/python-scripts/import-export/fromnuke2maya-export-nuke-cameras/

So, with the footage tracked and the camera imported into Maya, I set about the rather large job of rebuilding the Nelson street environment. As outlined in an earlier post about Environmental Rebuild, I did this in order to give realistic reflections and shadows onto any geo I placed within the scene.

Using the original footage as a guide, I built the entire area out of polygons and textured then via a combination of camera projection and textures captured during the shoot by my lovely lady Elaine.

This screen grab shows the environment visible to the camera rebuilt within Maya. The green static cameras on the walk way chart the path of the tracked camera and have been used to project frames from the original footage onto the geometry.

This screen grab is the view from the actual camera track, looking at the rebuilt and matched geometry with a mixture of projections (lo res) and textures (hi res).

One this environment was finished it was then a matter of checking the track and the geometry to ensure everything lined up okay. In order to cut down on render time, I added some metallic spheres into the scene and hid all the environmental geometry apart from in the sphere's reflection. This meant that all the computer was needing to render was the sphere's and the reflections in the spheres. Much less computationally intensive.
Once the spheres were rendered it was just a matter of importing the footage into Nuke and comping it back over the original footage.

The screen were a last minute addition within Nuke as I still had the original camera track inside of Nuke. Having the same camera in both programs really saves a lot of time.
All in all I'm pretty happy with how this came out and especially happy with the track within Nuke. Seems the Nuke tracker is far superior to Matchmover.

Camera Projection

Yesterday was supposed to be the day when this shot came together, it was supposed to be a shinning moment, and then the track didn't work and blew my whole operation. I ended up doing a blog entry about reflections and shadows and some other hot air to try and cover up the fact my track didn't work.

Well, today, even though I'm ashamed of it I'm going to post the track that looks like arse, but I'm also going to post a little bit about camera projection, which in this case has been a bit of a get out of jail free card.

As I said in my last post, there was a number of benefits in recreating the physical environment within Maya, accurate lighting and shadows to name a few, but the ability to generate a believable camera projection in place of texturing every surface is another.

This image that I'm reusing again because I'm to lazy to render another shot out show's the camera projection at work. In my last post I spoke about how camera projection works to generate reflections onto the ball, this time however I am using the projection to actually texture the geometry.

This Maya view window animation gives a graphic representation of the geometry, the renderable camera and the projection camera.

This is my track that unfortunately didn't work out, which is always a shame. In this particular render, I hid the virtual geometry and just rendered out the ball with it's reflections (last blog entry) and then added the actual footage back in at the comp stage. This helped cut down render times although as you can see. Because the ball  movement didn't match the footage it looks horrible and there is next to nothing that can be done to fix it apart from retracking the footage.

So, what I decided to do was re-render the scene, only this time I would only render the virtual environment with 1 single frame of the footage projected onto it. This image to be precise:

As the original footage now appears nowhere in these sequence there can be no conflict between track and footage. The following shot is the virtual geometry with the one singe frame being projected into it by a locked off projection camera and the match-moved camera moving through the scene.

As you can see there are a few anomalies towards the end as the renderable camera moves out of the range of the projected frame, although if I had the inclination to fix it, this could be rectified with multiple projection cameras throughout the scene stitching together the whole footage from 4 or 5 frames. Also notice that the ball now fits perfectly in the sequence with no slippage at all due to there being no original footage, it is 100% virtual.

All in all I'm pretty impressed with the way it came out and it just goes to show that even when things fuck up badly, there's still a positive to come out of it!

Shadows and Environmental Reflections

So, after creating a virtual environment from a live action plate I thought I'd take a look at what it was possible to achieve within this environment.
Unfortunately for this exercise when I rendered the scene out it soon became obvious that the 3D track was not up to par and as such the geometry I placed into the scene slipped and jumped about the place, thus making the whole shot look like pure arse.
This was a bit of a shame because all the work up to then looked promising. This however is another good lesson to learn and just re-enforces the practice of getting things right at the input, or initial stage of production. That age old allegory; "you can't polish a dog turd" still holds true today and is something I will be especially mindful of when it comes to producing the money shots.

So, as the sequence looked rubbish that doesn't mean that the still frames looked bad, so here's a little look at what I did.

Back at the old location again, I decided in true 3D CG style to put a sphere right in the middle of the scene. Why I hear you ask? Because I can. But, this isn't just any sphere, this is a sphere with a true to life shadow and environmental reflections!

If you saw my previous blog entry on 3D environments, you'll soon notice that from this match moved footage I recreated this environment within Maya. As this environment very closely matched the actual physical space and the camera move was (roughly) taken from the physical camera, it allowed me to use a technique known as Camera Projection to map the reflections onto the sphere. Here's how it works:

This mash of a rendering above is the geometry of the match modelled space, however instead of texturing this space with UV co-ordinates, I duplicated the 3D match-moved camera, movement, translation and all and used this second identical camera as a sort of film projector if you will. This projection camera projected the original footage that I filmed back onto the blank 3D geometry (which you can see above), thus creating perfectly matched reflections for the sphere to pick up on. Once this was achieved it was just a matter of rendering all the elements out separately for comp.
This is done via the Maya render layers where it is possible in render to turn off the primary visibility of geometry but still have it picked up in reflections.

 
This image is an example of that, as all the other geometry has had it's primary visibility disabled but is still casting reflections.
The final element in this shot was the shadow which again, was rendered in the same way. Using the 3D geo to help with the shadow, I rendered out a simple shadow pass on the alpha channel for use in comp.

So there you have it. All the elements of placing 3D geo into a match moved scene. There were a few final touches such as a colour and grade correction and a defocus on the ball to fit it in the scene but I'll cover them properly down the line. And maybe next time I can get track to work so there's some proper footage to look at!

3D Environments and plate stabilization.

Having looked at pretty basic match moving in previous posts I thought Id take it a step further today and try and Match Move some footage and then recreate the environment within Maya to match the camera.

After much trial and error of footage, also outlined in previous posts, I managed to find a piece of footage that was appropriate for this exercise. The main issue that I was coming up against was the jerkiness of the handheld footage, however on this take I managed to feed the footage into Nuke first, track the footage and smooth it out using Nukes de-jitter transform option. I might also add that the de-jitter option in Nuke is something that AE lacks and is one of the most useful plugs out there for film makers.

Above footage before the De-Jitter tracker was added

Footage after De-Jitter

Once the footage was stabilized I outputted it into Autodesk Matchmover once again and tracked the footage, creating a bunch of 3D track points. With the footage stabilized and less movement in the X and Y axis then previous shots, the solve worked out very well. Although there were a few focal adorations due to the fact it was my girl friends hand held stills camera on auto focus... 

Once the shot had been solved I imported to to Maya and got onto digitally recreating the environment. As I didn't have my tape measure handy when I shot this footage a lot of the geometry placement came down to guess work and trial and error. This was a good lesson for next time however, it is vitally important to take measurements of your environment!

The above frame is a snapshot of the very basic scene recreation. There is no detail as such and is all basic geometry, but with the match moved camera and the now virtual environment, this was the result of the render.

As you can see, the movement exactly matches that of the physical camera and the environment more or less lines up with its physical counterpart (apart from a few slips here and there!)
The real power of this technique is not in the recreating an already existing physical space, but more as a great frame of reference for adding 3D elements to the original footage. With the ground plane and walls added to the Maya scene, this ultimately helps to achieve realistic shadows, lighting and reflections for anything else I may want to add to this scene. Although this one is very rough, it is very promising for things to come!

Mental Ray Rendering

Today I was having a quick look at some of the render options within Mental Ray in Maya, and I have to say, it shits all over the Maya Software renderer. As a part of the course we were asked to design a character in Maya that would eventually get rigged and animated, I assume using Maya's IK and bone animation tools...or whatever, I haven't really looked to much into animating yet, that's still to come.


Anyway, as I'm looking at placing CG into real world environments, I thought I'd create a robot...which seems to be becoming a recurring theme on this blog... But this robot wasn't a large robot that turns into a car and destroys everything within close proximity, I was thinking a little more realistically about this one. I've always been a bit of a sci-fi fan, but believable sci-fi. So I was looking more at a design of a robot along the lines of the Isaac Asimov/Will Smith type genre. I was also influenced by the visual style of the Chris Cunningham video for Bjork's all is full of love. These to me seems to represent a more realistic representation of what robots will one day represent. This film clip and Asimov style narrative also raises a lot of philosophical and sociological questions, but I'm not getting into that right now. Needless to say it's a bad arse clip with some equally bad-arse CG and model making.

Chris Cunningham and Bjork's all is full of love.

Sonny the NS-5. IRobot.

But anyway, I've digressed into talking about robots again...Damn! So, as I was saying and as is the name of this post, I was looking at Maya's mental Ray renderer, specifically Global Illumination and Final Gather. Up until now whenever I had modelled, textured and rendered any object, I had strictly used Maya's ray trace shadows and reflections and be done with it. Although high quality ray trace shadows can look very effective, there is still a something intrinsically CG them. This is where Global Illumination and Final gather come in. 

When rendering with just ray trace shadows, the geometry receives light and casts shadow as it would in the real world, but it doesn't reflect light onto other objects or ambient lighting. This ambient lighting is such an important part of rendering and can take a good render and make it look a hell of a lot more realistic.

The first rendering here of my robot here was done using just ray trace shadows. Although the shadows are mathematically correct they don't look realistic.

The second render here is without Ray Trace shadows, but instead has used Global Illumination and Final gather to calculate the lighting. The Global Illumination function causes the light source to emit virtual photons into the scene and them map them as they decay and interact with the geometry in the scene. I have initially had a few issues with getting this to work but the final result gives a much more true to life representation of the objects.

This light bouncing and ambient lighting effect can been better demonstrated when getting up close to the geometry to see its effects. Again, the first rendering is with Ray Trace shadows turned on, without Global Illumination or Final Gather and the second is the opposite.

Note the darkness of the shadows in the top render as opposed to the ambient light in the shadows in the bottom render. The effect is quite subtle but I think makes a big difference.

So there you have it, a brief look at the finer point of Mental Ray and of course more robots. I'll be looking into how to actually animate this guy in coming weeks and eventually try and drop him into some match moved footage. Perhaps creature comforts style talking about what it was like being raised in Bristol...

3D Modelling

So, the last week or so I've spent locked away in a small cupboard learning the finer points of Maya, thanks to the online Escape Studios Maya Fundamentals course. This has already made my coming back to uni for my final year worth while as alone these courses are around 4k. So kudos to UWE for getting on-board with the industry leading software trainers.

Anyway, as I've just finished the extensive modelling and texture parts of the introducing Maya part (a small part of the entire course) I thought I'd do some very basic modelling to put what I've learnt into practice.
This exercise works on a few different levels. As my ultimate goal is to create live action VFX, learning to model objects to then place into match moved scenes in Maya is good practice for projects further down the road when I look into full 3D matchmoving.

What I've discovered thus far is that modelling makes up only one aspect of the whole process to create photo realistic sequences. Texturing, lighting and animation are as important as the models themselves. A film that stuck with me as having amazing modelling, texturing and realistic lighting was the latest Transformers film, The Dark of the Moon.

Optimus Prime from Transformers: The Dark of the Moon.

The level of detail in the textures and the detail in the models themselves was something to behold. I don't care that the story was non existent, the film had large robots, fast cars, explosions and hot chicks. What more do you want?

So, here comes the anticlimax when I show you what I've been working on....I decided to model a simple car to begin with, with some simple metallic textures. The idea behind this is that it is something that I can later build upon and drop into some live action tests.

My lame ass blob car

At the moment it's pretty low detail and needs a lot more of the subtitles added, as well as some more complex textures, but for the moment it will suffice and is a good start for further explorations when I get onto doing proper shots.