Medium: GPT2, Tacotron2, face classification, RPi, NVIDIA 2080Ti, metallic recliner, metallic headcover, inkjet printed fabrics, camera, LEDs, speakers, WiFi router, Ethernet/Power Cables
The “I am {Emotion}” installation is an exploration of the present and future of machine learning. A multisensory work that invites the audience to lie on a sun lounger and receive an endless stream of machine-generated psychological advice and stories.
The advice is generated in real-time and it is seeded with the emotion extracted from the participant’s face. The sun lounger is decorated with Memento Mori prints and crowned by a black metallic structure, containing part of the artificial brain. With speakers on each side along with a hypnotizing light, the technology is working in unison to put the subject in an altered state of consciousness. When someone lies in the recliner, an emotion classification algorithm analyses their face and generates a sentence describing the person’s psychological state. This sentence is subsequently used as the seed that generates the content. The generated advice is then fed to the recliner and vocalized through the speakers. Every couple of minutes new text is generated, resulting in endless continuous advice that is unique and never repeated to anyone else, other than the person lying on the lounger.
In a world where superintelligent machines will be a reality, we are trying to answer the ultimate question, will the optimisation agents of the future be able to read the human psyche?
The Prints
Three distinct designs on the upholstery celebrate life by depicting it in a kitschy illustrative way.
The prints bare similarities to Vanitas and Memento Mori style paintings, and as expected, things are deeper than they seem. Plants, fruits, and animals are not randomly chosen but they have a meaning and are symbols in the greater narrative.
One print is full of beautiful flowers full of nectar and succulent fruit resting on pristine tropical leaves surrounded by flies, saturating the space between the fruit compositions.
Whereas another print depicts a monkey eating from a beautiful display of fruits opposite to a hominid skull.
Rotten fruit, flies, or sculls next to monkeys symbolize our origins, the fragility of life and ask us to question life itself in the general context of the artwork.
None of this is visible at first sight, we just see vivid colors and beautiful lush displays, but just look a bit closer you will see the decay.
The Technology
The relaxing welcoming chairs are crowned by a dominating black heavy metallic structure that symbolizes and literally is the machine. With speakers on each side and a hypnotizing light on top and a camera that is too close for comfort.
There are three machine learning algorithms running in the machine
- GPT2 a state of the art text generator.
- Tacotron2 which is an algorithm that vocalizes text.
- A real-time face detection and emotion/gender classification algorithm
The way the whole system works is as follows:
Whenever a member of the audience sits on the recliner the distance sensor located on the side of the headpiece activates the camera and takes a picture of their face. The image is sent to the server and the extracted emotion is added the seed that generates the text. For example, if a “happy” emotion is extracted the seed will be “I am {happy}”.
The text is generated, vocalized and send to the raspberry pi which plays it back through the speakers.
The Human
Although the installation works in an aesthetic level it’s not complete without the 3rd element, the human. Sandwiched between nature at the bottom and the machine on top.
When someone lies in the recliner a proximity sensor sends a signal to the program to take a picture of the subject’s face. The face is analyzed and the extracted emotional state generates a sentence by populating the {emotion} variable resulting in a sentence like the following: “I am happy”, “I am sad”, “I am bored”. This sentence is used as the seed that generates the text. The generated text is then fed to the chair and vocalized through the speakers. Every couple of minutes new text is generated, resulting in endless continuous advice that is unique and never repeated to anyone else other to the person lying on the lounger.