T h i r t y - F o u r t h

The Making of a
Virtual Human Being

By Jon Kaufthal

See Jack. See Jack run. And slam dunk, do the limbo, and even test drive an Apache helicopter. But Jack's more than just your average computer simulation; he's the latest generation of an incredibly ambitious project: building a virtual human being -- not a robot, but rather a computer model of man. Jack and the research behind him are making considerable progress in blurring the lines between reality and artificially-constructed virtual worlds -- and in the process, changing the ways in which we interact with computers.

So what is Jack, anyway? The answer can be hard to pin down at times: essentially, Jack is a piece of software that puts a virtual human on a computer screen. Jack's physical specifications closely match those of humans. His joints all move in the same ways that ours do, so his body can be contorted into any shape that our own bodies could take on. He walks, bends, and reaches like we do, and he's slowly learning to talk and "think" on an elementary level.

Because he makes such a good guinea pig, Jack has been used for a wide variety of purposes. The US Armed Forces have used Jack to design helicopters and tanks, as well as to train soldiers for battle situations and medical emergencies. John Deere has used Jack to design tractors, and Ford has used him to test their Fiesta. Jack's customer list reads like the Fortune 500: Boeing, Lockheed Martin, GE, GM, NASA, McDonnell-Douglas -- these companies and dozens of others have put Jack to work for them.

Jack takes his bulldozer for a spin
But Jack is as much a concept as a commercial software package; a set of long-terms goals as well as an ever-evolving collection of thousands of lines of code. Jack is the tangible product of years of research at Penn's Center for Human Modeling and Simulation; the result of the endless work done by a small group of computer scientists.

Located in the Engineering School's Moore Building, the Center for Human Modeling and Simulation is a collection of faculty and students whose major goal is the imitation and animation of human movement. Jack showcases the Center's work, tying together nearly all of the research done there. Dr. Norman Badler, director of the Center, explains Jack's purpose: "We wanted to have the same kind of joints, the same kind of movements, if possible, that would actually look sort of like a human, and to get the computer to actually manipulate one of these virtual humans. We can then put it into different environments where we couldn't put real people -- for example, we can't cram real people into a computer-generated world, so we have to populate it with computer-generated people." And these computer-generated worlds are where Jack lives, while those in the real world watch Jack and control his movements.

Birth of a Virtual Human

Jack wasn't born yesterday. Badler notes, "his ancestry goes back to about 1973 -- the first time I tried to put up a...human-like model on a computer screen. The Jack program itself was the fourth version of such software we did here at Penn, and it really got its inauguration somewhere around 1986/ he's barely ten years old." But Jack has learned a lot in a short time: the early Jack was a crude, clunky set of boxes resembling a human body. As time went on, Jack became more complicated and visually convincing. A full spine was developed in 1989, making Jack's entire body act as a cohesive unit. And Jack gets increasingly more lifelike each year. Last year, for example, a smooth body was developed that goes a long way towards making Jack look just like one of us.

Today's Jack

Today, Jack has full internal organs and nearly every relevant body joint. His smooth skin is drawn from approximately 6,000 polygons, and it is rendered in real time, meaning that Jack moves when you ask him to, not after a computer delay. Jack is run on Silicon Graphics workstations, high-end machines with specialized hardware for graphics. He can be controlled by mouse, simply by choosing menu items such as "move hand" and then dragging the limb with the mouse. In addition, Jack can be viewed from any angle and rotated like any 3-D object. He has some rudimentary "intelligence" as well: put into a virtual environment, he avoids other people and objects, and can even obey traffic lights, much like any of us. When one of Jack's limbs is moved, his entire body recalculates its position in order to support itself and avoid falling.

One of Jack's major tasks today is "human factors" or ergonomics evaluations. Badler explains, "This is basically how people fit in and work in various environments like vehicles or factories or assembly what we provide is a whole body that can be used as a substitute for the real thing in order to determine that these environments have been designed correctly: is there enough room for people to work in, can they see what they're supposed to, are there clearance problems, can they lift the weight they're supposed to? So it just gives designers a tool for placing virtual people into their virtual designs before they ever build real full-scale mock-ups." The advantages, Badler explains, are saved time and money: "If you're building a very large device like a new airplane or a bus or a space really can't afford to change that design very often, so it's best to leave the design in the computer and iterate the design, constantly checking against the virtual human." And just as human sizes vary, Jack's body can also be adjusted. Using data from a 1988 Army survey, an entire family of Jacks was formed, representing different variations in height, weight, and build so that designs can be tested for the entire range of potential users.

Looking Ahead: Jack Moves to the Desktop

Jack in the box: old Jack meets new
As PCs continue to get more powerful, Jack's appeal may shift from limited, technical uses to everyday ones. Badler has a vision for Jack, asserting, "Undoubtedly, these virtual humans are going to inhabit everybody's PC pretty soon. As the computing power gets vastly increased, the kinds of software that we build now on expensive workstations can get pushed down into the everyday high-performance workstations that people have in their homes. One of the goals within the next one or two years is to push some of the Jack technology down into Internet as available virtual humans -- they may not have exactly the same accuracy and the same joint structure, but they'll have a lot of the same properties and appearances that we now have."

Badler may not be far off. The same Silicon Graphics workstations that sold for $25,000 several years ago now go for about $6,000, not much more expensive than an upscale PC. And today's high-end PC offers about the same processing power as a SG workstation, minus the specialized graphics hardware. Clearly, cramming Jack onto a standard PC won't be much of a technical hurdle. And in keeping with this trend, more emphasis has been placed on marketing: last fall, Jack was acquired by Transom Technologies, a new corporation formed around the Jack software. Transom has taken over all responsibilities for licensing and distribution, allowing the Center to focus on research. Transom and the University maintain close ties, working together on Jack's development.

Though it may be relatively simple to make PC versions of Jack technically feasible, finding a lasting role for Jack in the mass market could prove to be more challenging. Badler sees several possibilities, musing "An area that's wide open but...awaiting the the whole issue of educational resources on the Internet -- if one could have a virtual teacher or a virtual mentor, would that help -- or not -- in helping people acquire knowledge?" Or perhaps, Badler suggests, "what we really need is an Internet with a guide, with a teacher who can personalize it, especially for people who...need assistance in navigating such vast information resources. Basically, that's what teachers do -- they show you a path and then hopefully you follow that path until you need some guidance; the teacher or mentor intervenes, directs you another way -- we think there's a role for virtual humans to play."

Building Virtual Common Sense

While Jack's movement improves, more effort will be focused on interface. Badler explains, "Our challenge is going to be providing control -- how do you get this guy to do what you want him to does he understand what we expect other people to understand? So the real clincher for us is how do we build on this intelligence level to the point where he can accept commands and communicate with us through the medium that we're used to communicating with each other in?" While we may take for granted that "get my slippers" actually means "go upstairs, open the door, find my slippers, pick them up, and bring them to me," getting a machine to understand this type of implicit meaning is a great challenge. Even the simpler steps like "open the door" consist of sub-steps like reaching towards the doorknob, turning it, and pushing or pulling the door. The trick, Badler says, is "understanding what you're told to do. Anyone who has spoken to little kids understands that there's a disparity between what you're expressing and what you actually see accomplished. So you could think of Jack as a real little kid in this sense, because...he knows very little about how to execute the commands that we give him -- so we have to educate him to understand a much larger vocabulary of actions."

Jack Gets Personal: Avatars

Jack be nimble
Interaction will become a key feature as Jack moves to the desktop. One possible use of Jack technology is to build avatars, or virtual people controlled by real people. These avatars are 3-D representations of people, interacting in virtual space on the Internet. Avatars can carry on "face-to-face" conversations with each other, though the people they represent may be thousands of miles apart (and may, for that matter, look nothing like their avatars). Already, crude versions of such avatars are popular on the Internet. Badler explains, "There's got to be a motivator. There aren't millions of people in chat groups for nothing -- they must like to talk to each other and they must find that electronic communication of some value in order to spend their time and money doing it." And avatars should prove even more enticing: why talk via text when you can talk almost face-to-face? Demand for this sort of software should spark growth, providing incentives for development of avatar technology.

Other interface possibilities are already in development. Strapping on a virtual reality headset and sensors, it's possible to become Jack, seeing the world through his eyes and having his movements match your own. Pei-Hwa Ho, a systems programmer at the Center, describes a simulation done for the Army where "you have half purely virtual soldiers, the other half may be avatars, meaning there's a real person behind them. But just by looking at it you won't know which one is real and which one is not, and you can simulate training." As the virtual soldiers follow pre-programmed behaviors, the avatars (controlled by real-life soldiers) have to respond. Simulations like this one show the power of Jack to combine real and virtual worlds: on this virtual battlefield, it's impossible to tell whether the soldier you're up against is actually your friend, feet away from you in his own VR headset, or the creation of some programmer's fancy, existing only on some silent, faceless hard drive. Ironically, the virtual soldiers may be so "smart" that it will ultimately make no difference to the user.

Jack may not replace your professor anytime soon, but he does illustrate what can be accomplished by combining ever-more-powerful computers with talented and dedicated minds driving new research. Today's Jack is already a powerful tool, tomorrow's Jack promises to have far-ranging impact. While Dr. Badler and his colleagues at the Center for Human Modeling and Simulation and around the globe work on modeling the physical, and eventually, the mental characteristics of man, they will struggle with perhaps the greatest challenge software engineers will encounter, and what may just become their holy grail: translating the simple, yet dazzlingly complex genius of the human body and mind into a binary trail of zeroes and ones.

Jack and TV & Multimedia Editor Jon Kaufthal have much in common. Both live on computers, and both can "think" on an elementary level. Frankly, though, Jon does a better limbo.

Graphics courtesy of S. Sheridan, D. Chi, R. Bindiganavale, and C. Phillips, Center for HMS, U. Pennsylvania

Please E-mail with any questions or comments.

© Copyright 1996 The Daily Pennsylvanian, Inc. All rights reserved.

back to my writings

Back to

Internet Link Exchange
Member of the Internet Link Exchange