It is probable that prehistoric peoples experienced drawing with light in space by quickly waving the glowing ember on the burning tip of a wooden stick and observing the curvilinear trail caused by the eye/brain phenomenon of visual persistence.
Observations of visual persistence have appeared throughout recorded history. The brilliant 11th Century scientist Ibn al-Haytham has been quoted as describing the visual impression of a rapidly moving light appearing as a curvilinear streak thus: “Let someone take a stick aflame at one of its ends, and let him quickly move it right and left in a dark night: looking at such a flame will find it extended through the interval along which is moves.”
The perceptual process of visual persistence has often been confused with the process of retinal afterimages. Visual persistence and retinal afterimages are separate phenomena, supported by different mechanisms within the visual system. Visual persistence occurs primarily in the visual cortex and is responsible for constructing the image blend we experience as motion blur or light streaks.
In 1889 chonophotographer Étienne-Jules Marey’s colleague Georges Demeny attached incandescent bulbs to the joints of an assistant and created the first known light streak photograph, Pathological Walk From in Front.
Man Ray has been credited as the first artist to explore the technique of light drawing. In 1935 he made a set of long exposure photographs using a small flashlight to create a series he called Space Writing
Gjon Mili created dynamic light drawings in the tradition of Étienne-Jules Marey by attaching lights to people in motion. in 1949 Life magazine sent him on an assignment to photograph Pablo Picasso in his studio. When Mili showed his light drawings to Picasso, the artist was immediately inspired to make a now famous set of light drawings photographed by Mili. The image above is from that session and is one of a series of light drawings widely published in Life.
The immersive temporal/spatial medium of Virtual Reality provides a powerful set of new opportunities for drawing in space.
In art making, the act of drawing typically refers to a composition constructed from lines drawn out upon a surface, however the act of drawing has broader definitions which can expand our sense of what drawing in Virtual Reality can be. The following definitions are taken from dictionary.com:
verb (used with object), drew, drawn, drawing.
- to cause to move in a particular direction by or as if by a pullingforce; pull; drag (often followed by along, away, in, out, or off).
- to bring, take, or pull out, as from a receptacle or source:to draw water from a well.
- to bring toward oneself or itself, as by inherent force or influence;attract:The concert drew a large audience.
- to sketch (someone or something) in lines or words; delineate;depict:to draw a vase with charcoal; to draw the comedy’s characters with skill.
- to compose or create (a picture) in lines.
- to mark or lay out; trace: to draw perpendicular lines.
- to frame or formulate: to draw a distinction.
Credit for creating the first viewpoint dependent computer imaging system is given to computer graphics research pioneer, Ivan Sutherland. Sutherland’s viewer, developed in 1966, came to be called The Sword of Damocles. The device was mounted on the spectator’s head in a manner that provided data on the spectator’s changing point of view to a computer that then calculated and recalculated transformations to the stereoscopic image pair supplied to the displays. The resultant coupling of viewpoint to displaypoint created a marked sense of the spectators immersion within the space of the 3D image. This was the genesis of the technique that Jaron Lanier is credited with popularizing in the 1980′ as Virtual Reality.
Earlier Sutherland had created Sketchpad, the first interactive computer drawing tool. The following description is from the Wikipedia entry on that tool:
Sketchpad (a.k.a. Robot Draftsman) was a revolutionary computer program written by Ivan Sutherland in 1963 in the course of his PhD thesis, for which he received the Turing Award in 1988, and the Kyoto Prize in 2012. It pioneered the way for human–computer interaction (HCI). Sketchpad is considered to be the ancestor of modern computer-aided design (CAD) programs as well as a major breakthrough in the development of computer graphics in general. For example, the graphical user interface (GUI) was derived from the Sketchpad as well as modern object oriented programming. Ivan Sutherland demonstrated with it that computer graphics could be used for both artistic and technical purposes in addition to showing a novel method of human-computer interaction.
Roman Kroiter founder of the IMAX corporation initiated a project at IMAX in the mid-1990’s to develop a real-time stereoscopic hand drawing and painting system. The project eventually resulted in the release of the SANDDE software which was used for a number of film projects at the National Film Board of Canada. SANDDE is an acronym for “Stereoscopic ANimation Drawing DEvice” and is a play on the Japanese term for “3D”, which is pronounced “San-D”. The concept of SANDDE is to enable artists to draw and animate in three-dimensional space. It is intended to be intuitively usable, like a pencil. As an art form, SANDDE incorporates aspects of drawing, painting, sculpture and puppetry. The main input device is a “wand” which allows the user to create drawings in the air.
“SANDDE features some unique and powerful animation methods that allow you to draw and define action timing with live gestural recording–simply by moving the Wand in space. Because you use your own hand, there is an organic feel to the action. You can produce a sense of weight and emotion in your work, precisely as you feel it. You can even animate “live” to a sound track, or in slow-motion, to create action timing that is just the way you want it. Of course, SANDDE also supports frame-by-frame animation and in-betweening. In addition to stereoscopic 3D animation, SANDDE can be used for 2D animation, as a 3D storyboard or sketching application, or to create 3D artwork.”
Steven Schkolne developed the innovative project Surface Drawing while a student at CalTech and filed for a patent on the process along with his advisor Peter Schroeder in 1999. Surface Drawing become the topic of his Ph.D dissertation, 3D Interfaces for Spatial Construction. Surface Drawing went on to be utilised by BMW’s Designworks in Newbury Park, California. I was fortunate to be able arrange a field trip to the Designworks for my CalArts class Alternative Approaches in 3D Computer Graphic Animation where we all had the extraordinary opportunity to have our first experience with drawing and editing forms in 3D space.
J-Walt Adamczyk graduated from the CalArts Program in Experimental Animation in 1988. His 16mm thesis film, Recurrents, was created on the Cubicomp system in the Computer Animation Lab located in F105. J-Walt programmed the images for the film as fractal geometries and used color map animation to give them movement. After graduating J-Walt worked in the film industry in LA where, among other things, he was a developer at Walt Disney Imagineering and became a founding member of the Disney VR Studio which created a state-of-the-art virtual reality experience for Disney’s EPCOT center in 1994. In 2006, he won a Technical Academy Award for his development of the J-Viz Pre-Visualization System.
I was fortunate to attend a Los Angeles Abstract Movie Workshop meeting where he demonstrated an early version of his real-time gestural animation system. This system had the ability to realize some of the absolute animation performance gestures that I had long imagined composing with in VR. At the end of his presentation J-Walt graciously allowed me to experiment with playing the system. I was delighted to be able to perform the simple flowing serpentine lines that up until then I had only imagined. In 2004 J-Walt began giving public performances with his visual instrument under the label Spontaneous Fantasia. He has continued to improve the instrument and develop new compositional structures. He has performed in a wide range of venues around the world from small salons to large immersive domes. In the spring of 2017 J-Walt performed live digital animation effects and and played the role of God in a five week run of Paradise Lost: Reclaiming Destiny, an adaptation of Milton’s epic poem Paradise Lost, staged at the Greenway Court Theater in LA’s Fairfax District.
“With my performances, I take literally the meaning of the word ‘animation’: making images come to life. I use my drawing skills to give directions to computer programs I’ve written. I aim to capture the sensitivity of gesture, but I also use techniques which amplify and augment the gestures. The wave of the arm or the stroke of the pen is my foundation, but the effect is much more. My creative process for these pieces is to continually shift between painting, programming, composing music, and performing. My goal is to create a wholly integrated experience for the eye, ear, and mind.”
It was not until 2011 that we were able to put together a rudimentary VR system in the Computer Animation Lab (F105) at CalArts. Elijah Kleeman, a particularly brilliant and talented BFA student in the CalArts Program in Experimental Animation took an interest in my long held ideas for creating a gestural system for absolute animation performance in VR. He began developing a project that realized the fundamental functionality I had envisioned and we worked together to create the foundation of what was to become his extraordinary BFA Thesis project, Traces. Elijah developed and coded his project in Unity to incorporate a PhaseSpace motion tracking system and the eMagin Z800 HMD. The development presented many novel technical and aesthetic challenges and the amount of time Elijah dedicated to working through these issues paid off in a successful presentation of his BFA final project in CalArts B&W Gallery A404 on April 18, 2013.
In 2014 Drew Skillman and Patrick Hackett developed the stereoscopic VR drawing program Tilt Brush which has gone on to become one of the most compelling creator tools in VR. Drew Skillman was attending a meeting of the Visual Effects Society in San Francisco where Glen Keane was presenting his Google Spotlight Stories project Duet (hand drawn animation executed at 60fps rather than 24fps) which had been sponsored by Google’s Motorola division to promote their Moto-X phone. Skillman approached Keane after the presentation and offered to give him a demo of an early version of Tilt Brush at the Skillman & Hackett studio. Keane accepted and was so impressed with the tool that he arranged to use it to create the promo video Step into the Page for the 2015 Future of Story Telling conference. Google acquired the Skillman & Hackett company that same year and put them to work with a team of other programmers and designers to continue the development of Tilt Brush. I used to show an early video from Skillman & Hackett showing an early version of the program including the drawing on a tiltable plane that was at the core of the early process and gave rise to the name Tilt Brush, unfortunately it is no longer online. Google devoted many resources to Tilt Brush including the Tilt Brush Artist in Residence Program at their headquarters and residencies for invited artists in Paris at The Lab of the Google Cultural Institute. On January 26, 2021 Google announced that it would no longer be developing Tilt Brush stating: “As we continue to build helpful and immersive AR experiences, we want to continue supporting the artists using Tilt Brush by putting it in your hands. This means open sourcing Tilt Brush, allowing everyone to learn how we built the project, and encouraging them to take it in directions that are near and dear to them.” Back in 2017 a young Dani Bittman used Tilt Brush to create the video Scribble, an exuberant dance performance which went viral. Since that time she has developed a career using Tilt Brush (and other VR drawing programs). She has released a comprehensive set of video tutorials which she has made freely available.
Tilt Brush was essentially the first public release of a VR application for drawing in space, however other approaches have been developed. Oculus Quill takes a unique approach to the medium –which among other things includes the ability to retouch drawn lines. These developments can be seen to neutralize the hilariously snarky jabs at Oculus made in Bob Ross VR: The Joy of Tiltbrush
From the Oculus Blog, November 17, 2016:
“Quill was born out of the creative needs of Dear Angelica’s Writer/Director Saschka Unseld and Art Director Wesley Allsbrook. The unique style and story of Dear Angelica required it to be painted and shaped entirely inside of VR—something that had never been attempted.
In October of 2015, during a 48-hour hack-a-thon, Inigo [Quilez]—the lead engineer on Dear Angelica and a mad genius with code—created the first version of what would eventually become Quill.
From the moment Wesley tried Quill, she said it gave her something she’d wished for all her life: to no longer be limited by the edge of the page and to be able to create an endless world around her.”
In addition to developing the VR drawing and painting tool Quill, teams at Oculus have been working on the sculpting tool Medium. In a live onstage demo at the Oculus Connect 3 conference Oculus Art Director Goro Fujita introduced the audience to the history of these tools and his own involvement in using them to paint and sculpt in VR.
Adobe’s Project Dali takes yet another approach to drawing in VR space. At the time of this writing information about the project is scant, but it is another example of a large software corporation seeing the value in devoting time and money to the development of creative tools for artists working in VR.
Milan Grajetzki along with his colleague Dario Seyb is developing ANIMVR a tool for drawing and key frame animating those drawings in VR via the HTC Vive. They have done an excellent job of creating a well structured intuitive Spatial User Interface (SUI) with their Beta 1.0 release. They have also been developing Creations enabling artists to sculpt inside Unity 3D via Unity’s experimental EditorVR.
I had the good fortune to meet Android Jones at the VRLA Summer Expo 2016 where is film Samskara was playing on the interior surface of a 12 meter diameter dome. There was 5 meter dome adjacent to the larger one and Android was set up inside with an HTC Vive showing people how to play the abstract gestural animation program Microdose VR which he and his colleagues Anson Phong, and Scott Hedstrom had been developing since before it was called FlowMotion. He allowed me to spend a great deal of time directly experiencing the power of this intuitive visual instrument. I was excited to be able to realize the sweeping gestures of flowing instanced geometry particles that I had long imagined working with —and had wanted to incorporate into my ongoing realtime gestural absolute animation VR instrument Anaphorium. Having a chance to discuss all this with Android was inspiring. My background in performing psychedelic liquids in light shows for rock concert dances in the late 1960’s and Android’s background in VJing for EDM concerts in this decade provided another interesting point of intersection.