An Artist's View on Pokémon Go and Augmented Reality
Augmented Reality, or AR, uses mobile devices as viewfinders to add computer generated graphics to the world around us. This differs from Virtual Reality, or VR, which is experienced through goggles and where the real world is blocked out completely, replaced with an entirely computer generated world.
Pokémon Go, the wildly popular AR-based smartphone app, tracks players’ locations via GPS and uses AR to bring the lively animated characters into the real world through our smartphones. The overnight success of the game has brought the term augmented reality to the forefront of the public consciousness, but the technology is nothing new to Will Pappenheimer, artist, professor and co-chair of Pace's art department. Pappenheimer has been experimenting with AR as a medium for more than six years, and his work has been exhibited in galleries and museums around the world, and even in the sky.
The Dyson Digital Digest sat with Pappenheimer to talk about the incredible applications of augmented reality.
1. What was your first experience working with AR? What piqued your interest in the technology?
I first became interested in AR in 2010, shortly after several tech companies began to offer the technology in smartphones. I was fascinated by the idea of merging social media and network space into the real-world physical space. That year, with a group of other artists interested in the medium, we staged an uninvited “flash-mob” style exhibition at the Museum of Modern Art in New York, and invited audiences to view our show through their smartphones. The intervention got a lot of publicity both through the museum, the New York Times and Wired magazine, and we eventually went on to form the international artists group, Manifest.AR.
2. What is your biggest, or most favorite AR project to date?
It’s very hard to choose a favorite, but one super engaging project that got a lot of publicity was a virtual designer drug experience I created called Proxy, 5-WM2A; it was as a commission for the last party at the Whitney Museum of American Art’s Breuer Building. Visitors entered a huge room on the fourth floor and scanned a QR code “pill” which sent them into a 360-degree world of patterns, floating objects and pieces of the famous building via their mobile devices. The idea was not only to recreate the altered experience of psychotropic drugs, but to create both a community experience and the idea of merging psychologically and perceptually with our myriad of devices. The event was featured in multiple art and fashion publications; I was even interviewed by Vogue magazine
3. How do your AR pieces involve collaboration with the public?
Programmable and Internet-based media allows for what we know as "interactivity" and crowd-sourced participation. As soon as I started working with AR, I wanted to find ways to allow the audience to participate in creating the work. In a continuing series that uses a mobile app called Skywrite, which I collaborated on with designer Zach Brady, the audience is invited to write messages and create drawings, which can be uploaded for placement in the virtual sky above them. This gives the public a voice, which is especially important in cities like Washington D.C. where the app was very popular. In another work, Biomer Skelters, which uses software developed by Pace students, participants wear an exercise heart rate monitor, and as they walk, reaching a state between their resting and active heart rate, they begin to generate a beautiful and rich tapestry of AR plants and vegetation in their path.
4. What effect does augmented reality have on the public?
You see people walking and looking around themselves through their phones as if they are searching for something; friends used to laugh at me when they saw me doing this while I was creating and installing artworks. Pokémon Go has created an alternative city with its own separate narrative and we can jump from one version to another, or live in both simultaneously. It’s also important to note that the basic notion of augmented reality is not in fact new. Literature, history, and culture augment the world such that we always see a combination of what is physically there and the cultural context we have learned throughout our lives. In this sense, every object and place has its ghosts.
5. Have you played Pokémon Go? Why was it so effective at bringing AR to the masses?
I have played Pokémon Go and have lots of fun with it. It's actually a very primitive form of GPS-based AR, and my guess is it's going to get a lot more realistic and interactive. The simplest answer for its success is the wide reach and popularity of Nintendo. For Pokémon fans, there’s an automatic motivation to want to find the characters. Because the characters are graphically simple, no realistic attributes have to be mimicked, which keeps the necessary computing power to a minimum and allows the game to be played on as many devices as possible.
6. What does the success of Pokémon Go mean for the future of AR? Do you have plans for future work with AR or another technology?
I will continue to work with AR because I believe in it as a fascinating medium with important implications. My assumption is that with the success of Pokémon Go, and big tech companies like Apple, Google, and Microsoft investing billions in this technology, it is bound to move a long way forward. As the technology advances, it will open up new avenues for creating artwork. I am also beginning to learn how to produce for VR, which is a medium with an entirely different set of experiences and challenges. Right now I am working on creating surround AR experiences for interior spaces such that, through video projection, the audience sees themselves in the same space as the virtual world. It presents the possibility of developing large scale mixed reality experiences that occur within a space that is both defined and limitless. The technological development is challenging and of course I never seem to have enough time!