3D Drawing and Painting

The stereoscopic still image pairs embedded on this page are formatted for cross-eye free-viewing.

Screen shot 2013-11-04 at 2.26.15 AM

This is not a pipe.  This is not a hand painted stereoscopic picture.  This is a copy of Jim Long’s  2D to 3D conversion of Rene Magritte’s 1966 version of his seminal 1929 painting, “The Treachery of Images“.  2D to 3D conversion via a greyscale image used as a pixel displacing depth map is probably the most common contemporary method for creating 3D drawings and paintings, however many other approaches are possible.  Jim utilizes the ‘select and shift’ method which allows for a precision as fine as single horizontal pixel shifts in sliced regions of the image.

While freehand drawing and painting in stereoscopic 3D is difficult it can be done –and in fact for many years it was the only approach possible.

.

.

WheatstoneDrawings

The very first stereoscopic drawings were created and exhibited in 1838 by Charles Wheatstone in order to demonstrate the binocular basis of  depth perception utilizing his new invention, the stereoscope.  Wheatstone used geometric figures for clarity and the relative ease of constructing them through projective geometry techniques.  He was also concerned with convincing people that it was his discovery of the principle of stereopsis that created the sense of depth experienced with his stereoscope. To this end he included the following explanation in his 1838 presentation to the Royal Society of London, “Contributions to the Physiology of Vision. Part the First.  On Some Remarkable, and Hitherto Unobserved, Phenomena of Binocular Vision.“:

“For the purposes of illustration I have employed only outline figures, for had either shading or colouring been introduced it might be supposed that the effect was wholly or in part due to these circumstances, whereas by leaving them out of consideration no room is left to doubt that the entire effect of relief is owing to the simultaneous perception of the two monocular projections, one on each retina.”

The difficulties encountered in hand drawing stereoscopic pairs with a pencil, straight edge, and protractor lead to a search for a mechanical drafting solution that would allow an assisted form of freehand drawing where both drawings could be made with a single stroke.

.

.

RulePatentDrawing_US2171894-0

In 1939 J.T. Rule received a patent for his “Apparatus for Producing Stereographic Drawings

.

.

pantograph3DdrawingTool

In his 1970 book, “The Intelligent Eye“, R. L. Gregory describes his development of a pantograph arm that allowed for vertical and horizontal motion across the XY axis of the drawing surface.  Attached to the end of the pantograph arm was a head assembly that remained perpendicular to the top and bottom of the drawing surface as it pivoted with the arm.  The drawing head contained two pens (one red, one cyan or green) that could be moved closer together or farther apart by twisting the large knob gripped in the artist’s hand thus allowing the artist to draw in a variable parallax that delineated the Z axis .

.

.

3DD_1982_vladimirTamari

Vladimir Tamari is a Palestinian artist, inventor, type designer, and physicist who has lived in Japan since 1970. He studied at the American University of Beirut where he met and was inspired by Buckminster Fuller. He invented the 3DD stereoscopic drawing instrument in Palestine in 1964, a new version in Japan in 1973, and received a patent on the version shown above in 1982.

The search for systems to allow an artist to draw or paint direct in stereoscopic 3D has continued into the era of computer graphics.  Real-time digital image processing is enabling new paradigms in stereoscopic drawing and painting.

.

.

Magnetic_Drawing_In_SpacewithLogoWEB

Roman Kroiter initiated a project at the IMAX corporation to develop a real-time stereoscopic hand drawing and painting system.  The project resulted in the commercial release of the  SANDDE software system that is imaginatively illustrated above.

.

WYSIWYG_3Dpaint

The 2013 paper,  “WYSIWYG  Stereo Painting“, provides a good look at continued development (even though the authors fail to decrypt the common acronym for those who may be unfamiliar with it. WYSIWYG = What You See Is What You Get).

.

.

In 2014 Drew Skillman and Patrick Hackett developed the stereoscopic VR drawing program Tilt Brush which has gone on to become one of the most compelling creator tools in VR.  Drew Skillman was attending a meeting of the Visual Effects Society in San Francisco where Glen Keane was presenting his Google Spotlight Stories project Duet (hand drawn animation executed at 60fps rather than 24fps) which had been sponsored by Google’s Motorola division to promote their Moto-X phone.  Skillman approached Keane after the presentation and offered to give him a demo of an early version of Tilt Brush at the Skillman & Hackett studio.  Keane accepted and was so impressed with the tool that he arranged to use it to create the promotional video Step into the Page for the 2015 Future of Story Telling conference.  Google acquired the Skillman & Hackett company that same year and put them to work with a team of other programmers and designers to continue the development of Tilt Brush.

.

.

portrait-jan16-1944-2_cropcross

Norman McLaren was 32 years old when he enthusiastically wrote the following opening to his paper, Stereographic Art, dated June, 1946 (coincidentally the month and year of my birth):

“In 1944 I began to explore, what, to my knowledge, is a new form of graphic expression, namely , stereoscopic drawing, painting, mobiles, and sculpture. As a result, I have come to the conclusion that the the development of Stereographics —a method introducing optical principals new to the visual arts— may have as much consequence for the future of these arts, as have the application of new optical laws and perspective for European painting of the fourteenth and fifteenth centuries.”

San Francisco State University Professor Harold Layer corresponded with Norman McLaren for decades. During that time Norman Mclaren provided him with unpublished archival materials (including the drawing above) that Professor Layer was accumulating as research for a book on stereoscopic drawing.  That book was never completed and so he has published some of his material online along with donating his entire collection to the McLaren Archive at the University of Sterling Library in Scotland.

.

.

Stereo-Portrait-of-Kim

In the 2012 version of this course, Josh Shaffner painted a freehand stereoscopic portrait in oil on a single 9″ x 12″ panel with the left and right eye images reversed for cross-eye free viewing.  Shaffner based the painting (his first in stereo) on a pair of photographs taken at slightly differing angles.

.

.

click to view larger image

Alan Ammann used a similar technique in 2009 to create a pair of oil paintings for this stereoscopic portrait of Andrew Leibs. The image above reverses the left and right paintings in order to free-view them with the cross-eye method.

.

.

3210572129_b3f7c424b4_z

Ammann made use of a folded mirror version of Wheatstone’s stereoscope to make gallery viewing accessible to people unfamiliar or uncomfortable with the technique of free viewing. More images are available from the project’s  Flickr stream

.

.

Oskar Fischinger made a number of stunning stereoscopic paintings.  His extreme parallax painting “Triangular Planes” dates to 1949.  I did not have the good fortune to meet Oskar, however I had found deep inspiration in his films and did have the good fortune to be introduced to his widow Elfriede by my friend Bill Moritz.  I loved to visit with Elfriede in the home that she and Oskar had built in Laurel Canyon.  The house was filled with Oskar’s paintings and I was especially entranced by the stereoscopic pairs.  Oskar’s paintings also served as studies for animation projects and he wrote the following about his plans (learn more at the Fischinger Trust):

Out of my recent Stereoscopic Paintings and out of my recent film “Motion Painting Number One”, the necessity for combining both creative efforts into one new “Stereo-Motion Painting” presents itself to me.

The technique developed in “Motion Painting Number One” will be combined with the deep-space creation developed in the Stereo Paintings, with the idea in mind of bringing this new creation to the largest possible number of people at present and in the future. To bring them to a closer participation, understanding and love toward Art.

…In a Stereoscopic presentation (which does not eliminate a non-stereoscopic presentation) the almost fantastic precision and space reality brings about a completely new field of Art expression into existence. Future Artists will have a new field of expression. Space itself frees them from the two dimensions of their present day canvases. There is a new infinity without limitations given to use through Space Paintings in Motion and color and sound.

.

.

click to view larger image

Salvador Dalí had a lifelong interest in stereoscopic perception, however it was not until the 1970’s that he produced a series of stereoscopic paintings.   One of the aspects of stereoscopic imagery that fascinated Dalí was the way in which the retinal rivalry (also known as binocular rivalry) caused by tonal variations in the left and right images could create the effect of luster.  He made great use of it in many of his 3D paintings.  In this stereoscopic pair of paintings from the unfinished “Dalí from the Back Painting Gala from the Back Eternalized by Six Virtual Corneas Provisionally Reflected in Six Real Mirrors” The colors of the drapes and of Dalí’s shirt are swapped in the left and right paintings.  This provides a lustrous sheen reminiscent of silk or velvet.  There are other variations as well on the canvas, the walls, mirror frame, and of course that unintended by the unpainted faces in the right image (appearing on the left in this pair as they are swapped for cross eye free viewing).

.

.

daliNoLuster_640

Jim Long‘s excellent 2D to 3D conversion of the painting is created from the more complete side of the pair and he does not add the retinal rivalry color shifts that create luster.  Long’s work  provides for a more readily fused image since even a skilled painter like Dalí could not maintain perfectly homologous points between the right and left paintings.   I purchased a set of foldable paper 3D viewers of some of Dalí’s work at the Dalí Theater-Museum in Figueres and was later surprised to discover that some of these are actually 2D to 3D conversions rather than left and right stereoscopic pairs hand painted by Dalí.

.

.

click to view larger image

The use of depth maps to convert 2D images to 3D has developed rapidly.  The control that the conversion artist has over the depth parameters of the stereoscopic image has become very refined.  Dan Jacob‘s 2D to 3D conversion of the painting “Burro and the Bad Men” by the talented illustrator Andy Thomas is a great example of this craft.

.

.

sleepingWoman_14

I created this simple 2D to 3D conversion of Pablo Picasso‘s Sleeping Woman in order to demonstrate to my stereoscopy class how 2D line drawings could be made into 3D line drawings for use in frame-by-frame animation.  I demonstrated how to segment the image into discrete layers within which varied grey values could be painted to serve as a depth map.   The shaded values of the depth map were used to horizontally displace the lines of the drawing. The lighter shades in the depth map served to create a horizontal disparity between the left and right eye images that caused the lines to appear to emerge forward of the zero parallax plane.  The darker regions of the depth map caused the lines to appear to recede behind the zero parallax plane For the purposes of the demo I created an anaglyph image, however it is a simple matter to create separate left image and right image files for other modes of 3D viewing.    Image displacement via depth map is currently the most prevalent  technique used for 2D to 3D conversion.  The technique is being utilized on low budget projects by independent artists with access to off the shelf tools such and Photoshop and After Effects as well as high budget block-buster film project work by large visual effects companies using proprietary tools.

.

.

click to view larger image

Mike Seymour authored an excellent article for “fxguide” in May of 2012 providing a comprehensive overview of what were then the current the state-of-the-art approaches being adopted in the feature film industry:  “The Art of Stereo Conversion 2D to 3D – 2012″.  Seymour’s article,  “The Art of Roto: 2011”, provides an in depth look at the development of digital tools for rotoscoping and includes a section on 2D to 3D conversion techniques.

.

.

click to view larger image

Like many people of my generation, I first experienced 2D to 3D conversion in the form of 3D comic books.  I was seven years old in 1953 and used to love walking down the hill to the little neighborhood market where I lived in Manhattan Beach.  The mission was to buy candy bars and hang out with my friends.  The market proprietor allowed us to sit on the floor and read the comic books (we would occasionally purchase a few).  My strongest recollection is of a 3D Mighty Mouse flying through the magic depth of those anaglyph panels.  Ray Zone –who was born a year after I was– was also impressed with 3D Mighty Mouse comics that year.  Ray was a visiting artist in the class a couple of years ago, and among the many things he shared with us were comic art paste-up boards that he had used in his work converting  2D drawings for 3D comics.  The camera ready boards were photostat copies of the original inked panels.  Ray had used the physical cut and paste process to make a second card.  When pasting up the second view he would shift elements on the horizontal axis the appropriate amount to create a sense of placement in depth in the final anaglyph printing of the comic book page.  The simple lateral move of that process created layers of flat images stacked in depth resulting in what is often described as the “cardboard cutout” look.  One of the most common 3D comic book production techniques used in the 1950’s involved the layering of transparent acetates inked with copied flat drawings.  The layers containing specific elements were horizontally offset to create the appropriate horizontal disparity in the left and right image when printed in anaglyph.    Occasionally an artist would create a perspective shift drawing for the left or right image that caused it to actually slant into and out of the page.  Harvey Comics promoted the power of the drawn 3D perspective look in their True 3-D process comics (a much more labor intensive process than the simple horizontal shifting of left and right elements).  After sharing his old school boards with us, Ray gave us a demo of how to use the digital images tools in Photoshop to cut and paste elements for quickly creating stacked planes of 2D artwork. He demonstrated how Photoshop transformation tools could also make perspective foreshortening 3D possible.

.

.


Andrew Murchie demonstrates the basic concepts of 2D to 3D conversion of comic book art in his “3D Photoshop Tutorial 01 – Create Stereo 3D from 2D Comic Images” tutorial.

Murchie has also created a wonderful set of spacey tutorials describing simple, fast , and imperfect 2D to 3D conversion techniques using After Effects:

•    ‪3D Tutorial – Convert 2D to 3D with Adobe After Effects – Part One – Simple Channel Shift‬
•    3D Tutorial – Convert 2D to 3D with Adobe After Effects – Part Two – The Pulfrich Effect
•    3D Tutorial – Convert 2D to 3D with Adobe After Effects – Part Three A – Displacement Mapping
•    3D Tutorial – Convert 2D to 3D with Adobe After Effects – Part Three B
•    3D Tutorial – 2d to 3d Adobe After Effects Tutorial Four – Motion Mattes

.

.

YUVsoftDepthMap

SFX veteran Steve Wright provides an overview of a 2D to 3D workflow using YUVsoft, a third party program for After Effects (Win only) and NUKE (OSX and Win) originally developed by researchers at Moscow State University.

.

.

Eric Maslowski of the University of Michigan 3D Lab created this long, slow, and carefully detailed introductory level demo on creating depth maps for 2D to 3D stereoscopic conversion using Mocha and After Effects (with references to Steve Wright’s demos).  Mocha AE is bundled with our After Effects CC licenses in the CalArts computer animation labs thus making 2D to 3D stereoscopic conversion of drawn, painted, and other forms of animation readily accessible.

.

.

chu_3d_anaglyph

And finally, a return to hand painted 3D in the form of anaglyph line art by the British graffiti artist Chu who received  a commission to create a 3D spray painting on the inside of a cubic chamber.  Chu created geometric projection warps on the six surfaces of the cube so that the corners seem to disappear. He then hand painted the resulting drawings with spray cans in red/cyan anaglyph stereo.  When a viewer enters the cube and stands in the central sweet spot, the immersive space of the painting appears to extend in all directions.  You can get some sense of this via Chu’s 360° interactive presentation located in the third panel of his project documentation page.  Can you find the door out?

.

.