This technical article defines rotoscoping and outlines my attempt at mastering this important aspect of compositing for visual effects.
Origin of rotoscoping
Wikipedia defines rotoscoping as: “an animation technique in which animators trace over footage, frame by frame, for use in live-action and animated films. Originally, recorded live-action film images were projected onto a frosted glass panel and re-drawn by an animator. This projection equipment is called a rotoscope, although this device has been replaced by computers in recent years. In the visual effects industry, the term rotoscoping refers to the technique of manually creating a matte for an element on a live-action plate so it may be composited over another background.”
The technique was invented by animator Max Fleischer around 1915 and patented in 1917. Fleischer used rotoscoping extensively in many of his cartoon features. The Superman cartoon series is considered to be the most effective use of rotoscoping by Fleischer because of the realistic motion of the animated characters.
Need to know
Several years ago I was involved in the freelance graphics field of DVD authoring. I used the Adobe Production Studio Premium bundle. This software package included After Effects—powerful animation and compositing software, Premiere Pro—non–linear video editing software, Photoshop—the industry standard for graphics editing, Illustrator—high powered vector based drawing software, Encore—reliable DVD authoring software and Audition—sophisticated sound editing software.
I honed my skills until I was able to work efficiently and deliver high quality projects. One day I lost a bid because I wasn’t able to fill an order that required a substantial amount of rotoscoping. I was determined to master this aspect of the trade.
The fan film connection
The earliest attempts at rotoscoping that I can remember would probably be the creation of flip book animation. Simple figures were drawn in the outer margins of pages; these drawings changed slightly from one page to the next. When the pages were turned rapidly a primitive simulation of motion was created.
In a later attempt I was involved in making an 8mm movie with some friends. One scene required a character to fire an air rifle at a small aluminum boat. The bullet would strike the boat just below the waterline and cause it to sink. The crew would then abandon the vessel. After the shot was finished and cut; I meticulously hand painted a series of frames: a muzzle flash extending from the rifle barrel, a puncture hole and splash at the waterline of the boat.
I had a firm grasp of the concept of rotoscoping and considerable vector drawing skills acquired in the monument industry. I just needed practical application of frame by frame roto work.
One day I was browsing around a freelance site and I happened to notice a request for rotoscoping work to be done on a volunteer basis. It was a Star Wars fan film that had been produced and directed by an aeronautical engineer. A retired visual effects artist from Hollywood was the effects supervisor. This was a perfect opportunity to gain experience and learn from a professional.
About three days after giving my contact info to the director of the film I received an e-mail asking me to attend a Skype conference call.
“What studio are you affiliated with and how long have you been doing roto?” the effects supervisor asked.
“I’m a freelancer and I have don’t have any practical rotoscoping experience,” I replied.
“What can you possibly bring to this project?” the supervisor asked.
“I have a strong background in motion graphics and an even stronger desire to master rotoscoping,” I answered.
“Sorry, but I don’t think we can use you,” he said.
“Hold on a second,” the director said, “would you mind answering a few questions?” he added.
The effects supervisor asked me a series of questions related to animation and compositing. I was able to answer correctly on each point.
“Welcome aboard,” said the director.
I was given a login to the server that contained the source footage for the film and I was instructed to familiarize myself with the rotoscoping department procedures for execution and delivery of the shots. The rotoscoping requirements for the film were directly related to the laser saber battle scenes of actors wielding sawed–off broomsticks that had been jammed into the hilts of Star Wars sword handles. Third party plugin software that had been designed for laser saber rotoscoping was provided as well. The laser saber plugin was a tubular shaped graphic with control points at the top and bottom of the graphic. These control points could be moved to match the current position of the broomsticks.
My first roto assignment was a two second shot of a single actor with a minimum of saber and handle motion. The footage was recorded at approximately 30 frames per second. The process seemed straightforward and the manipulation of the plugin control points for this particular shot went fairly smoothly. After a considerable amount of tweaking and adjusting of the control points of the plugin to precisely match the movement of the footage I had a solid playback of the saber moving with the stick.
I rendered a sample for review and uploaded it to the server. In a few days I received an email message from the supervisor indicating that my submitted footage had been accepted. In film industry terminology it was called a “hero”.
The second roto assignment was three seconds long with two actors or 90 frames with two sabers—a total of 360 control points that had to be painstakingly moved to match the movement of the sticks. The process was tedious and mind numbing; however, I was determined to master rotoscoping and I pressed on until I was able to submit a hero.
The third assignment was a six second shot of four actors in a fast-moving sword fight. I began the roto process and noticed that the built-in motion blur feature of the saber plugin produced unexpected results when trying to match movements that switched directions rapidly. The motion blur feature would fan out into a glob of glowing jello. I was forced to repair individual frames.
I struggled through two more similar assignments.
Advanced technique application
The fourth assignment I received was 28 seconds long with eight actors. At this point I seriously considered giving up. I wasn’t satisfied with the amount of time it was taking to get the results I wanted—there had to be a better way.
I researched rotoscoping techniques and found an educational DVD entitled advanced Rotoscoping Techniques in After Affects by Pete O’Connell. I watched the sample video and decided it would be worth the investment.
The technique O’Connell used combined motion tracking, java script and traditional rotoscoping. I repeated the video exercises until the concepts and procedures were hammered in.
I adapted the technique of using the motion tracker to follow the coordinates of the tip and base of the sabers. I applied this data to the control points of the plugin and the swords fell right into place—as long as the tracking data was accurate. This workflow cut the production time by 50%.
The problem of the wacked out motion blur created by the plugin still needed to be addressed. I experimented with the Beam effect in After Affects. It was similar to the plugin in many respects; it had starting and ending control points and perspective adjustments that increased or decreased the thickness of the beam. I added different blur effects to the beam and built a slider interface for easier control. I was able to complete the 28 second battle scene in three weeks and I was assigned other effects work on the same scene.
My fifth and final assignment was a three-minute scene. The first shot of the scene had some serious camera movement problems. I stabilized the footage and started to remove the green screen walls and floor. The type of footage being used on the film was not ideal for green screen. It was impossible to pull a clean key—industry terminology for successful background removal. I used Photoshop to meticulously select and paint the edges of the foreground characters of each frame. This method provided the best results; however, it was time-consuming and required a strong working knowledge of Photoshop. The film had over 80 minutes of green screen footage. It would take two–three years to complete using that method.
The director announced that if the film wasn’t at least 75% complete in three months that he was going to kill the project. 25–30 volunteer compositors from around the world were struggling with the green screen footage. I’d mastered the technique of rotoscoping and I really didn’t want to put any more effort into a film that might not be released so I left the project.