Publication TypeMiscellaneous
AuthorsDeWitt, Tom; Edelstein, Phil
Source (1977)
Full Text: 

by Tom DeWitt

Illustrated by Vibeke Sorensen

Thomas Jefferson was fond of his pantograph (fig. 1). He used it to copy manuscripts as they were being written. Carbon paper and the Xerox have since eclipsed this function, but the pantograph does have its redeeming graces. Bucky Fuller must appreciate how this simple machine takes advantage of the innate weaknesses of the rectangle and converts them into a useful function. One wonders if this arcane instrument can instruct modern minds.

In the light of pantomime, the lesson of pantography becomes clear: imitation. Mr. Jefferson moves his quill up and down, and the automated quill follows every motion. The abstract movement of "writing" has been separated from Tom's immediate space and can be observed for itself. When a motion becomes independent of other bodily gestures, it is on the level of dance which motivates the mime.

Instruments which follow motion exist. Motion picture photography has this quality as does tracking radar. This article describes a new device, the Electronic Pantograph. It uses the eye of a television camera and the brain of a computer to keep track of moving points in space and writes a record of this movement in the abstract language of a computer-choreographer. It can be used to quantize the space of reflected light, and it can do this in real time. Dancers and musicians, artists of the dimension of time, must appreciate the waveform generated by the Electronic Pantograph: position versus time.

In fact, pantomation was first conceived in a music studio. It was seen as a convenient way to notate electronic music. Light pens and "Rand" tablets are also useful in this capacity, but they must be used at the computer site. How inconvenient for the composer who has a brainstorm on a boat in Bavaria (as suggested in the cover illustration) or for some impoverished genius who does not own a computer. As originally designed, the Electronic Pantograph uses a television camera to "read" handwritten musical scores. In this system, waveforms are loaded into digital oscillators and are routed to analogue processors by crosspoints closed according to a hand-drawn matrix chart. The resulting score looks strikingly like classical guitar notation (fig. 2). By December 1976 an actual machine had been developed which could load a hand-drawn waveform into a computer. The temptation to turn this modified camera on a human figure was too great to resist.

2. Pantomation

The first application of pantomation occurred during the production of Outta Space, a pantomime which makes use of video synthesis to simulate gravitationless space. Indeed, there is virtually no gravity in a cathode ray tube, so dancers will understand why a potentially serious development in their act was first used as a source of levity. In one scene, the mime James Snafu plays a magnetic robot who lifts the hapless Zierot by the scruff of his spacesuit. Snafu is not noted for his bulging biceps. The illusion was created by attaching a bright yellow dot to Snafu's hand. The dot was followed by the Electronic Pantograph looking through the eye of a chroma key camera tuned to the dot's particular hue. The derived position in space was used to deflect the image of Zierot as displayed on the video synthesizer. When the dot was hidden from the camera, which the mime could easily do by merely turning his wrist, Zierot was left suspended, as if floating, in the last position marked by the dot (fig. 3).

The sleight of sight which produced the pantomime hand gag can be extended to facilitate other performing arts. For example, the colored dot on the tip of a conductor's baton could be interpreted by a computer to shape the timbre, modulate the amplitude, alter the tempo or direct the balance of an orchestral sound.(fig. 4). To this musical purist, who disdains the frenzied expressions of contemporary conductors and places their gyrations in a category just one level above horse whipping; pantomation could provide a set of visual cues to help in the appreciation of the music while giving the performer better control over it.

The skill of movement which characterizes the conductor has its next nexus in the world of dance, and pantomation could prove extremely valuable in linking this ancient art with the modern media. The conductor may be able to generate music by movement, but the dancer-uses movement to generate more movement. Two dancers working together can create a larger sense of spatial change than the individual dancers working alone. Indeed, unlike mime artists, dancers invariably come in groups. The whole is greater than the sum of the parts. With dance in television, the whole should be greater than the sum of the arts.

It has been axiomatic in televising dance that the little screen narrows the scale of the performance. Cutaways to close-ups are essential if the viewer is to witness much rehearsed detail, but this lowers access to the performance as a whole. Such a dilution of intent becomes inevitable when

3. Pantomation

videotaping dance works meant for the stage. Of course, one should be thankful for the historical record afforded by such achievements as Dance in America, but it pains this observer to witness contemporary creative groups such as the Merce Cunningham Dance Company march in front of the camera like circus animals. The dancer ought to have some form of direct physical control over the image while it is being created. This is not to denigrate the abstract communication that makes possible collaborations between dancers and video artists, but rather a call to enhance the abstraction by giving it a visual representation. If the dancers' movement controlled a synthetic space, the link to video might be easier to forge.

For those whose imaginations might request a concrete example, Duchamp's Nude Descending a Staircase is an excellent reference. This skeletal abstraction shows the parameters of movement in discrete steps (fig. 6). The foresight of this painting has since been underscored by its frequent imitation in photographic studies. Pantomation could imitate this idea at a rate of sixty times a second. Indeed, such related creations as Boccioni's futurist sculpture, Unique Forms of Continuity in Space, are within reach of the Electronic Pantograph, because the third dimension can be readily detected by a two camera version of the same device (fig. 5). Additionally, a movement may be magnified or contracted, superimposed or offset to the side, created immediately or replayed from the past. Movements can become visual structures that serve as "sets" for subsequent events; other visual abstractions can replace the dancers' costumes.

These ideas of dance in television are currently being explored by Electronic Body Arts (EBA), the Air Farce Mime Troupe, and the Electronic Music Studio of the University at Albany with equipment support from WMHT TV, Schenectady, N.Y. The projects are guided by Joel Chadabe, Composer-in-Residence at SUNYA, Tom DeWitt, an NEA and AFI fellow; Maude Baum, Phil Edelstein and George Kindler of EBA. A pantomation system is under construction at WMHT TV under a grant from the New York State Council on the Arts. A basic demonstration will be made at the NEA sponsored Design in Electronic Art Conference on March 10 at Media Study Inc in Buffalo, N.Y.


Figure One. Jefferson's pantograph was a tabletop contraption which moved two quills in parallel.




Figure Two. Graphic Notation.

2a. Matrix designating ten rows of
inputs and ten rows of outputs.

2b. A sequence of notes scored
according to "skyline" notation.



Figure Three.

December 31, 1976, the first videotape using Pantomation. The live mime is in the outline costume, and he is using the Electronic Pantograph to manipulate a photograph of a man in a spacesuit.


Figure Four. The Electronic Pantograph interprets baton position to distribute a signal to four loudspeakers.




Figure Five. A simple algorithm in the computer uses the offset in the x dimension to calculate depth (z) -- just like our eyes.


Figure Six. If Duchamp's nude had worn chromakey dots she could have descended in the Pantomation video studio...

P. Edelstein


The techniques and hardware presented here should be viewed as exploration into a relatively new field of electronic technology as an independent medium with its own peculiar aesthetics and styles. While the actual hardware has applications in the visual arts, music, dance, and theatre, I hope that the specific environment does not overshadow the appreciation of this form. In coming to grips with this baby beast of a new medium, one gathers insight into the nature of sculpting with electronics by studying how the same hardware reveals various aspects when examined from different media.

The hardware and techniques described here are useful for deriving information from video images based on the position of a key signal relative to the video frame. By selecting and developing man and machine intelligible images, the signals derived from this system become highly articulate medium for processes involving the generation and modification of electronic images and sounds. Many of the applications of the basic hardware offers users a highly interactive environment for these generative and manipulative processes. There are many options available to the individual image maker/composer/performer/ choreographer/designer/ patch cord plugging user of this system. These basic techniques can be applied to a variety of approaches to making sounds and pictures. The environment that these techniques can be applied to range from live performance of sounds and visuals to real-time, interactive studio situations and the more traditional compositional approaches to making music and video. As a performance device the key positioner can free a performer who has to manipulate electronic signals to use visual cues as input to performance devices. In any situation, the system can be configured to reflect the specific interests of the user.

There are two elemental possibilities available by implementing the key positioner. The first involves tracking the position of one point per video frame by deriving binary counts proportional to the horizontal and vertical position of the point. The other mode utilizes the systems ability to derive the horizontal position of a key signal on each line while still keeping track of vertical counts.

The once-per-frame mode is most useful for manipulations based on the movement of a key object in video or real space. In conjunction with other image synthesis equipment or even a simple wipe generator, the position and movement of an electronic image can be controlled by the movement of the original keyed object or pattern. Where the original image is fed to a rescan system such as the-Rutt/Ettra synthesizer, it is possible to adjust the DAC outputs of the system such that the keyed point will remain static as referenced to the rescan frame although it is moving about the input frame. This is useful for keeping an object in a fixed position in a video frame although it was originally moving in the real world. We can now not only move fixed objects but freeze moving ones. The processor can also retrace the route the object described by storing a sequence of horizontal and vertical positions and dumping the list to the DAC's. Another aspect of the once-per-frame mode uses an external clock to control the output rate.

Instead of immediately outputting the position of the key, the position can be stored in computer random acces memory. This sequence can be clocked out thru the DAC's at a rate determined by a real-time clock. If the sequence is repeated quickly enough, the DAC's will draw a series of points corresponding

to the shape outlined by the path of the keyed object. Since these voltages are two waveforms such that the period of each is determined by the number of frames sampled times the clock frequency and the shape and amplitude by the path of the object, the signals are meaningful as control or audio sources.

Hardware Description of the Key Positioner

The heart of the hardware for implementing the key positioner consists of two eight bit binary counters, one for horizontal position, the other for vertical. These counters require latches to store the data until the computer is ready to retrieve the counts, some means of strobing the data from the latches onto the computer data bus and some control logic to synchronize all of the functions. The current version of the positioner, built for a PDP8, fits on one wire wrap card and uses about 22 old fashioned TTL integrated circuits.

Aside from the special positioner board, a certain amount of fairly common video equipment and a small computer of one sort or another is necessary. A video image to key off of, either a camera, VTR or some pattern maker (SEG, wipe generator), is used as input to a keyer with TTL output. If a keyer is not available, a good clamped video signal with a comparator after it can be used -to fake the key signal. The video equipment must also be able to supply H drive and V drive and a 3.58Mz. chroma signal. In a black and white studio, an oscillator running at about chroma frequency will work almost as well. Minimally, the computer should have two DACs and of course, some means of getting programmed (such as a programmer!).

Principles of Operation (see attached block diagram)

The HD signal is tied to increment an eight bit counter (V counter) that is cleared by VD at the end of each field. Similarly, chroma is used to clock the H counter which is cleared by HD at the end of each video line. The chroma signal in an NTSC format signal will produce a resolution of about 190 points of horizontal resolution during an unblanked line without jitter. A free running clock cauld be used with resolution versus frequency tradeoffs and a +/- one point uncertainty. Vertical drive is also available to the processor for program timing. The sequence of events for normal operation upon key signal is:

    1)the counters are strobed into their respective latches,
    2)the data ready flag is set preventing resetting the latches until the data ready flag is cleared by the processor,
    3)the interrupt line to the computer is asserted.

The processor then collects the data from the latches and can clear the ready flag and wait until the next field or frame to clear the flag.

Generally, the current mini-computer is fast enough to grab on key per horizontal line. The position of the key signal can be transferred to the DAC's or stored in memory for subsequent outputting.

Some Quick Applications and Summary

The voltage outputs from the DAC can be used for all sorts of handy tasks if that is how you would like to relate to machines. Left/right panning of a sound following a performer is easily implemented with one output, an inverter and two Voltage controlled amplifiers. Adding, subtracting and synchronizing movement for dance requires a much more complicated set-up but is possible and will prove to allow new levels of collaboration between dance and electronics. Programs can be written for extraction of rhythm and dynamic changes. There is a whole class of operations using techniques making the input image based on the input image (feedback) that can be used for tracking multiple objects. By placing program control over key levels and masking it is possible to digitize an image given enough time.

We are seeing how related applications have already been realized by interdisciplinary collaboration between artists inside of electronics and artists in other disciplines as well as designs by individuals in many mediums. Key positioner techniques should allow interactions of another nature between artists and electronic media. I hope that with the realization of hardware comes an awareness, sensitivity and human use of our technological environment. Pantomation, the use of visuals to control electronic processes, is one possible meeting place for this to happen.




Thanks to George Kindler of Electronic Body Arts and the rest of the staff at EBA as well as the Electronic Music Studio at SUNYA.

Key Positioner Installation at the Electronic Music Studio of the University at Albany. Basic Fred's chin position is read into a PDP 8 L computer via a chroma key positioner reading. 3/6/77


Key Positioner Interface Diagram. Realized in hardware using a wirewrapped "Flipchip" module on the PDP 8L Omnibus.

Tool Name: 
DeWitt Pantograph
Other Tool Designers: