UPDATE: This is a workaround for something that has now been implemented in PFTrack 2016. You can read about that here: http://www.thepixelfarm.co.uk/pftrack-st-maps/. Its works in the same way but is now A LOT easier! 🙂
So a while back I did a blog about lens distortion using PFTrack. And even though still valid I think a better way of doing things now will be to render distorted in the first place.
Why render distorted?
There are three main reasons:
- Keeps your distorted corners crisp.
- Simplifies comp.
- No plugins!
The issue is, when you re-distort post render, you are basically stretching single pixels to distort the frame to match the original plate. This causes softening around the edges which is horrible when it comes to set extensions, especially if its anamorphic! So when you render distorted, its as if your rendering through a physical glass lens, meaning your image is distorted but with the benefit of perfectly sharp edges. This gives you control in comp to soften however you like.
Now also, when your in comp it can get very confusing (especially for junior artists, and even senior artist) having to deal with undistorting and distorting. Sometimes the resolution changes on a per shot basis and things get even more crazy! So this way the matchmover deals with all the distortion and all comp get is a render thats the same resolution as the back plate, distorted, ready to slap on top of the original plate. As an added bonus this means you don’t have to deal with installing plugins for various tracking software, awesome!
Whats the process?
I’ve figured this out with both 3dequalizer and also PFTrack and both work basically the same way, which I’ll explain. 3dequalizer is a lot simpler but both work. My guess is you can probably do this with most tracking software, I’ve only tested it with these two so far.
Create Identity UV Map – Tool (from 3de guys)
Once installed (read in the notes below) this is pretty much all there is too it:
create_identity_uv_map (this calls up the script) -out uvmap.exr (this specifies the name of the file) -size “1920 1080” (this is the output resolution)
All you need to do is navigate to where you want the file to be output before running the command.
- Solve your camera, along with your lens distortion
- Open up warp4
- Export out your undistorted plate for 3d
- Generate your UV map with the ‘Create Identity Tool’ at the original plate resolution (this is the part thats different to pftrack) using 3dequalizers software (I’ll explain below)
- Replace your input in warp4 with the UV map and change it from ‘Remove distortion’ to ‘Apply distortion’
- Export this out as a 32bit EXR
*Sometimes the UV map looks just like a black and white gradient, think this is a bug in 3de but it makes no difference to us, just a display thing.
- Track and solve your camera however you like
- Export out your camera and undistorted backplate like normal
- Note down the undistorted resolution, it will be slightly different
- At the end of your tracking tree, add an ‘Apply Distortion’ node
- Now generate a UV map with the ‘Create Identity Tool’ the same way as with 3dequalizer, using their software, but at the undistorted resolution noted earlier
- Bring in this UV map and plug it into the second input of your Apply Distortion node
- You should now have a UV map distorted back to the original resolution
- Add a Export node after this and write out the UV map as a 32bit EXR
- Now in 3d (tested with Maya/Max) import in that tracking scene along with your undistorted backplate
- Add the vray physical camera attribute to your camera
- In the extra attributes find the distortion type menu and change it to texture
- Load in your UV map
- Now when you render your camera will be distorted.
*You can check this is working correctly by rendering your undistorted backplate and comparing it to the original, if nothing changed (apart from some softening where the image is getting resampled) then all is good!
- In nuke simply bring in your render and A over B!
If you want to use the UV map to distort in Nuke instead of in the render then just use the STMap node and it will distort the image. This is the normal method so you will still get softening but its still a better way of doing things as it means you don’t need plugins. Also if you are using the “apply distortion in comp” method then render at the overscanned resolution for pftrack and the normal plate resolution for 3dequalzer, its just how they deal with it.
A thing I found, the UV map exported from PFTrack sometimes crashed my Nuke when connecting it to the STMap, all I did was shuffle out the channels leaving just the red and green and it seemed to fix the issue, as below:
Things to note
- Zoom lenses work exactly the same, you just render a sequence of UV maps instead of just one, as your distortion is going to vary throughout the zoom.
- DO NOT CREATE YOUR UV MAP WITH NUKE. This is something that tripped me up, if you do this you will find there is a half pixel offset in all directions of distortion. I didn’t read too much into it but its something to do with Nuke reading pixel information from the bottom left corner, whereas we need to read from the centre of the pixel, the ‘Lens distortion plugin kit’ from 3dequalizer is free to download here: http://www.3dequalizer.com/index.php#?site=tech_docs&id=110216_01 for windows, linux and mac. Its a shell command thats simple to use if you read the help that it displays in the terminal.
- I’ve also heard of issues in Nuke with sampling when post distorting with the UV map, causing stepping in the image, I haven’t witnessed this myself but will update if I find the reason.