Superimposing virtual information over real objects is still an effective means of communication that certainly will be explored further in the future. This principle has proven to be one that can be implemented using a variety of techniques and interfaces.
After Home of the Bain (1992), VR Daimler (1992) and VR Urban Planning (1989) had been complete, virtual environments with interaction via data-glasses and data-gloves, our research quickly focussed on the combination of real and virtual space, because users were totally isolated in highly immersive VR environments. VR interfaces sealed off real space so completely that any reference to it was no longer possible and gloves, glasses and their associated cabling severely restricted freedom of movement. Augmentation, however, opens up new possibilities for the communication of location and object related information. Content that does not exist in the real world can be made available in the form of a scaled, virtual layer superimposed over real space. This principle has proven to be one that can be implemented using a variety of techniques.
AR with Mobile Interfaces
By the mid-90s ART+COM had already taken on what is today the most common AR interface: the mobile screen. For the project Window into Virtuality (1997) we developed a showroom for DaimlerChrysler where a car could be viewed simultaneously with all available options and extras. There were two possible ways to present the car at one-to-one scale: we could either augment a camera image of the real showroom space with a virtual vehicle, or model the virtual vehicle along with the showroom surroundings. For the DaimlerChrysler showroom we decided on the second solution for a number of reasons. At the time it was less onerous to bring an original-sized vehicle into the space, and it allowed a sufficiently convincing experience of the car. In addition, it neatly avoided the problem of a virtual vehicle image being overlaid by the live showroom visitors. A mobile screen was mounted on a telescopic arm, which could be moved around by the user: over, around and inside the virtual vehicle. The system was built to scale. When a user holding a screen moved one metre in the showroom, there was a corresponding metre of movement in the virtual space. A simple and natural orientation in space was thus added to the already intuitive interface.
The first AR prototype interface that we developed in 1993 was made using the earliest commercially available LCD monitor, a bar camera, and a Polhemus 3D position sensor.
Twenty years later, home-developed hardware is no longer a necessity. For the interactive exhibits for the Reifenhäuser Group we used commercially available tablets to augment the physical objects. Detailed, 3D-printed models of the production facilities could be walked-around using tablets with interactive, layered information superimposed over a live image. The system was designed using an existing development environment that enabled the detection of three-dimensional objects without needing visual markers. To eliminate potentially disruptive factors, the exhibits were designed with a clear, neutral environment. The models were set on deliberately low bases, prompting viewing from above so that the models filled the camera image.
AR with fixed interfaces
Berlin is a city of rapid change; the present is constantly overwriting history, making it invisible. The cityscape has undergone dramatic upheavals, particularly in the short period after reunification when the wall and border fortifications were demolished; the formerly invincible seeming bulwark disappeared from view. From this situation and somewhat parallel to Window into Virtuality came the idea to make the Berlin Wall, now almost completely demolished, visible again through augmentation via a fixed interface, so that people who’d never experienced the city divided could see what the wall was like. Originally intended for a site at the Brandenburg Gate, Timescope (1996) worked just as well without a live image. A freeze-frame of the current empty border strip taken from the position of Timescope could be blended with historical edited recordings.
Timescope was the ‘slim’ first version of Virtuscopes, which in time were further developed into several variants. Jurascopes (2007) at Berlin’s Museum für Naturkunde, for example, allow visitors looking at the dinosaur skeletons on display to see how dinosaurs looked, hunted and ate in their natural habitat.
Airportscopes (2011) at Zurich airport has integrated live cameras. Used to observe planes taking off and landing, information about the aircraft type, place of origin and destination is displayed over the live image. Real-time data is channelled from air-traffic control as well as ground radar to identify planes based on their position.
In contrast to AR applications on the market today, our projects have been developed from a design perspective and for very specific contexts. Whether the interface is fixed or mobile, the augmentation is driven by the content and the design is, accordingly, heavily focussed on the spaces and subjects. But regardless of the technology, the design principles remain the same, even as the use of mobile devices as interfaces increases in the future.
AR with Pepper’s Ghost
Long before the advent of computers and data projectors, the principle of augmentation was realised with other techniques. Pepper’s Ghost, an analogue projection technique that first came into use on stage in the mid-nineteenth century, is one example. In comparison with other augmentation techniques, it has the advantage of being technically relatively simple, and therefore robust, so the Pepper’s Ghost effect offers an elegant solution for superimposing still or moving images over real objects and spaces. We have gone back to this old augmentation technique in the context of various projects, at times combining it with new media. For the exhibition of the project The Invisible Shape of Things Past (Museum of Contemporary Art, Porto, 2001) an architectural model was augmented with a dynamic, virtual model. To increase the realism, in addition to a Pepper’s Ghost projection on mirrored glass, the shadow of the object was projected directly onto the architectural model.
For Boulevard of the Stars (2010) the effect was used for the stargazers. Film stars appear as still images on the boulevard where they can be watched and photographed through the stargazers.
In the BrandSpace (2011) at the Deutschen Bank in Frankfurt there is another exhibit based on the Pepper’s Ghost effect: animated graphics illustrating the ecological effects of the renovation of the Deutsche Bank Twin Towers are superimposed onto their plastic model. In front of the model lies a thematically structured book providing information about the various renovation measures. As visitors turn the pages, animated graphics matching the book’s themes appear on the towers and around them.
Superimposing virtual information over real objects is still an effective means of communication that certainly will be explored further in the future. Visual effects in the cinema are increasingly looking realistic, but they stay in the cinema, while AR in the physical world uses real spatial references. The augmented objects retain their aura while onsite visualisations convey additional meaning.