FE417: Motion Capture for Artists

marey2red

“Progress is not possible without deviation” –Frank Zappa

Evolving Class Schedule for 2014:

January 13
First class meeting.  Browse some of the key books related to the course.  View several approaches to motion capture starting with a brilliantly cynical warning of the type of mocap abuse we hope to avoid.  The three following videos include scenes created via the PhaseSpace mocap system at CalArts.  The remainder of the videos are selected to serve as examples of other approaches to motion capture that extend the possibilities beyond the conventional.  Some of them are mere motion graphic exercises and others are much more.  All are useful examples for contemplating the possibilities of what we might do.  After the screening we will discuss each student’s interest in motion capture with some projections on directions the course might take to best serve them.

.

The California Institute of Motion Capture, CalArts Producers Show Intro 2007
We can do this the easy way or the hard way –or not do it at all.
Let’s push the boundaries of irony and do the dance of the mushroom cloud anyway!

.


Ke Jiang made Taxi while he was a student at CalArts.  He used The PhaseSpace mocap system to create a quirky performance by taking advantage of the artifacts that occur at the edge of the capture volume:

.


Visiting Artist Max Hattler conducted a workshop during the Program in Experimental Animation interim sessions in 2011.  The goal was to produce one or more short works using abstracted motion capture.  Forms I (Taekwondo) is one of those works:

.


Shimbe used the PhaseSpace motion capture system in a unique way for the making of this film. He rigged a Bunraku puppet with active markers and directed Danielle Ash as the puppeteer. The natural floppiness of the puppet provided an extraordinary quality to the targeted motion.

“The Wonder Hospital, a 3D & puppet animated film, is a surreal journey of oddity and empty illusion. In a mysterious hospital, modification of physical beauty is not what you would expect. A girl’s desire for superficial beauty leads her to chase after the luring ‘After’ images on a path of advertisements throughout the hospital. But in the end she finds something unimaginable and irreversible.”

.

January 20
Martin Luther King Holiday

.

January 27
Second class meeting.  View and discuss Sara Pocock’s mocap test and the documentation promo from Robert Abel and Associates historic 1985 production, The Making of Brilliance.  View and discuss video documentation of the 2013 Mocap for Artists Course Exhibition. Examine the components of the PhaseSpace Impulse system and learn to propagate the spandex suit with markers.  Suit up a student and explore several ways the live data may be viewed in the PhaseSpace Impulse software.  First and last assigned reading of the semester:  Maureen Furness’s comprehensive paper, Motion Capture.

.


A Maya playblast from 18 March 2010 of Sara Pocock’s little girl character animated via a simple mocap T-pose test. The T-pose test was performed in class by Justin Leon in order to double check that we had setup the MotionBuilder marker mapping process correctly before moving on to a directed capture session. We came close do doing a brief capture session but ran out of time and had to postpone the session until the upcoming class. The realtime data from the T-pose test is all that we used in this test. No clean-up, filtering, retargeting, or other adjustments were done. Justin’s simple casual movements gave the character an unintended sense of presence. In subsequent class meetings Justin and Sara worked on directed performance tests in order to gain more experience with that form of mocap –even though Sara’s goal was to keyframe all of the animation in the final film.

.


This promo documents several aspects of an early marker based motion capture system. Markers were hand tracked from a 2D screen and then plotted into the 3D character’s motion channels. The software used in this project grew out of the code used to drive motion control cameras. That code was soon incorporated into the first off-the-shelf 3D CG package created and marketed by Wavefront, Inc. Robert Able and Associates created the ground breaking Brilliance commercial for a single airing during Super Bowl XIX. Notice the subtle and not so subtle interplay of hype and confidence building between studio, agency, and client.

.

February 3
Third class meeting.  Postpone originally planned demo of Oculus Rift and Unity until later in the semester.  Suit up a student. Discuss ideal marker placement positions vs. positions on actual suit available (especially noting the disadvantages of our older suit design where the leg marker velcro pads are located on the front of the leg rather than preferred position on the side). View markers in the PhaseSpace Master program and demonstrate how to do a live data performance with the Master program.  Observe the ranges of marker jitter and data loss.  View the the marker grid in the Tracking Window to confirm marker drop out (flickers from green to white) and complete marker loss (turns from green to red).  Have fun with using the suit data with the path traces of the PhaseSpace Painter program. Discuss the advantages found in the creation of rigid bodies formed from a selection of markers, for example, how a rigid body created from four markers on the head allows for rotational data as well as translation data and the added reliability of having any three visible markers maintain the positional data for the head.  After the class break jump into using Master with MotionBuilder without a prepared and rehearsed demo. Fun with incorrect mappings, etc.

.

February 10
Fourth class meeting.  Suit up a student.  View markers in the PhaseSpace Master program and discuss how to do a direct data capture from Master.  Open Autodesk MotionBuilder and import PhaseSpace generated marker data.  Assign markers to the MotionBuilder “Actor”.  Create rigid body marker sets.  Import a MotionBuilder “Character” and assign the “Actor” to drive the motion of the “Character”.  Work in real-time to view issues with the data mapping onto the “Character”.  Use the “Plot” function to bake all the motion data onto the skeletal structure of the “Character” to save the rotations and translations as key-frames (every frame is considered a key-frame in this case).  Export the key-framed scene as a .fbx  file.  Import the .fbx file into Autodesk Maya.  Discover how  to actually see the imported character and observe issues with scaling etc. from within Maya.
MB_suit_config_guide

February 17
President’s Day Holiday.

.

February 24
Fifth class meeting. Presentation: Chronophotography: The Seminal Work of Étienne-Jules Marey and Eadweard Muybridge in Relation to the Development of Digital Motion Capture.  View and discuss sections of Norman McLaren’s 1968 film, Pas de Deux as an example of the influence of Marey and Muybridge on McLaren and other artists, such as Butch Rovan.  Suit up a volunteer student for a capture session.  Have students log into the F105 lab workstations, launch Parallels,  start Windows,  and open the PhaseSpace program Master.  Work with the performer to capture a T-pose and some motion data.  Students open Autodesk MotionBuilder and import the PhaseSpace generated marker data.  Assign markers to the MotionBuilder “Actor”.  Create rigid body marker sets.  Import a MotionBuilder “Character” and assign the “Actor” to drive the motion of the “Character”.  Work in real-time to view issues with the data mapping onto the “Character”.  Use the “Plot” function to bake all the motion data onto the skeletal structure of the “Character” to save the rotations and translations as key-frames (every frame is considered a key-frame in this case).  Export the key-framed scene as a .fbx  file.  Import the .fbx file into Autodesk Maya and discover ways it my be used.

As with Marcel Duchamp’s seminal painting Nude Descending a Staircase, No.2, Norman McLaren’s Pas de Deux was inspired by the chronophotography of Étienne-Jules Marey. McLaren’s studied use of the optical printer for creating dynamic temporal and spatial offsets can be seen to share similarities with the compositional devices later used by John Whitney Sr. and Larry Cuba in films such as Arabesque and Calculated Movements. Similar approaches to structuring and revealing image flow may also be seen in many contemporary real-time digital works.

.

Butch Rovan has this to say about his interactive installation, Let us imagine a straight line:
Let us imagine a straight line is an interactive work about movement, the first installment in my ongoing project for dancer, video, music, and live electronics called Studies in Movement. I take these titles from two French thinkers of the late 19th century: physiologist Etienne-Jules Marey and philosopher Henri Bergson. Marey conceived the apparatus for the modern scientific study of movement. He invented instruments to measure human and animal locomotion—a beating heart, a bird in flight—and developed technologies that eventually led to the modern cinema. Bergson responded to these advances with a philosophy that rethought the relation between space and time, matter and memory, physical and psychical movement.”

.

March 3
Fifth class meeting.   Michael McNeff  added the OWL plugin to MotionBuilder on the F105 systems so that students could follow along with demonstrations on running a PhaseSpace capture session with MotionBuilder.  Gina volunteered to suit up in the newly modified capture suit (a stopgap suit until we can purchase one of the new and improved versions).  Students learned to add the PhaseSpace server IP address to MotionBuilder, record a T-Pose  and walk around, load the “actor” model into the scene, create a marker set, add select markers to the “actor” nodes, create appropriate rigid body marker sets, activate the actor,  add a “character” to the scene, set the “actor” to drive the “character”, and record a take.

.

March 10
Sixth class meeting.  Demonstrate use of Autodesk Character Creator for creating modified character designs from templates for use in MotionBuilder and Maya.  Have students log in to Autodesk and set up a free account in order to access Character Generator.  Create “favorite paths” for the “clip art” stock character folder as well as the newly created characters folder.  Load student created characters into Motion Builder and apply motion from previous capture session to the character via the actor module.

 .

March 17
Seventh class meeting.  Quick review of the use of Autodesk Character Creator for generating modified character designs from templates for use in MotionBuilder and Maya.  Have students who missed last weeks class due the visiting artist workshop by Chris Sullivan log in to Autodesk and set up a free account in order to access Character Generator.  Demonstrate how to import marker data into Maya from MotionBuilder so that each marker shows up as a locator (null object).   Dissect the technical process used in making the abstract animation project, Handle, that Alexander Cruz created in the class  last yearSuit up a Gina so that Cameron and can direct motion capture sessions for their respective projects.   Discuss the unique issues and challenges that each project presents.  Have the whole class work with loading the live data data into MotionBuilder and applying it to their character designs.

.

March 24
Spring Break

.

March 31
Eighth class meeting. Meet in the Black and White  Studio (A404) to discuss  Rachel Ho’s upcoming project (slated to premiere on Thursday).  Visiting mocap expert John Brennan will demonstrate best practices with PhaseSpace and MotionBuilder.  The use of  low cost alternatives to professional mocap systems such as the Kinect will be discussed and/or demonstrated.  Continue work on student projects in preparation for the following week’s course exhibition in the B&W Studio Gallery A404.

.

April 7
Ninth class meeting.  Set up and run student projects in the A404 B&W Studio in preparation for the Thursday afternoon and evening course exhibition.

.

April 14
Tenth class meeting. Reset the PhaseSpace Impulse mocap system in F105.  Review documentation of the course exhibition and critique the event.

.

April 21
Eleventh class meeting.  Show and discuss examples of mouse based gestural motion capture with a focus on correlations with certain aspects of musical composition.  View a few excerpts of my absolute videographic animation from the eighties that serve as the point of reference in my proposal to extend that work into immersive Virtual Reality.  Set up the Oculus Rift stereoscopic HMD and PhaseSpace stylus.  Demonstrate one of the experimental “instruments”  that Elijah Kleeman has developed for his 3D drawing program .  Invite students to take a turn wearing the HMD and inscribing space with the stylus.  Discuss the positive and negative aspects encountered in these initial experiences with gestural marking in immersive 3D space.  Work with the PhaseSpace Impulse system to explore particular students interests. Suggested reading:  My paper, Absolute Animation and Immersive VR, and if time permits, Golan Levin’s comprehensive thesis, Painterly Interfaces for Audiovisual Performance
 .

.

.

April 28
Twelfth class meeting.  Meet in the CalArts exbox teleport for a a GoToMeeting teleconference and demonstration arranged by Alex Czarowicz,  Vice President of Sales, Organic Motion, Inc.  His team demonstrated various aspects of their markerless motion capture system which relies on an array of multiple video cameras viewing a white cube  with green floor stage.  Students were able to ask questions and request particular performances by the team conducting the live motion capture demo session. A soon to be released 120Hz system was demonstrated and its properties discussed. Alex provided us with a link to a zip file of some current data samples that can be loaded into MotionBuilder.

May 5
Thirteenth class meeting.  Final class session and party!

.

May 12
The School of Film/Video Bijou Festival.

.

.