This is an interactive art installation using a piano playing computer algorithm I wrote that generates music reminiscent of early twentieth century expressionist composers like Arnold Schönberg. Since the software is capable of generating its own œuvre, I gave it a human-like name, Ferin Martino. In this installation, motion of the viewers, seen by the computer’s video camera, influences the intensity of the music created. This creates a situation that lets us reflect on the ontological nature of music: this music cannot be heard without the audience causing changes in the composition. By extension, it offers a chance to reflect on the way that any composer’s music only has its existence in the minds of its audiences, and that the modes of it existence may be as diverse as its listeners. This is an idea suggested by literary theorist Umberto Eco in The Open Work (1962).
This work consists entirely of software and can generate new material practically indefinitely. The fact that the code fits on one screen indicates the elegance of the approach to generating endless music with pleasing results.
In an era when interactive technologies are increasingly able to “know” more about their surroundings, this work reminds us to fully explore the possibilities of past and existing technologies and especially to seek the “native voice” of each technology: What does it do most naturally? What does it “want” to say when we’re not forcing it to think like we do? This is the key to maximizing the creative potential of our new advances, and it also importantly lets us reflect on the impact of technology on the human experience. By setting aside detailed computer vision and generative music algorithms and simply exploring the shortest, most natural digital paths between camera and piano keyboard, this work has come into surprisingly rich, subtle, and playful forms of interactivity, with a musical output that is infinitely varied but still maintains a consistent “character,” a distinct style. This work also brings advanced questions from aesthetics to the minds of newcomers. When a composer writes sheet music, a performer reads it aloud with a musical instrument, and an audience comes to listen, everyone’s role is clear, and we know what “the work” of art is. In contrast, this installation has a creator that wrote no pitches or rhythms, and the performance is shaped by the listeners. To what extent is the listener a composer? When is a machine creative enough that it earns a human-like name? What does it mean to present a body of “collected works” that is infinite and always distorted by the listener, so that the real “whole” of the work can never be known?
Every exhibition is tailored to the venue. The responsiveness of the algorithm is adjusted to its surroundings, and it can use a synthesized piano or (preferably) a Disklavier digital player piano when a local sponsor can be found. It has functioned as lively lobby music, low-key music for a chill-out area, and has even been used on stage and for composing sheet music for humans to play.
This project has appeared in the International Computer Music Conference (ICMC) at the Onassis Cultural Center main lobby (Athens, Greece), Generative Art international conference at the Triennale museum (Milan, Italy), Society for Electroacoustic Music in the United States (SEAMUS) at Virginia Tech, I-Park environmental sculpture garden (Connecticut), and Sculpture Magazine.
I’ve presented talks and papers on this project in the Sound and Music Computing international conference and the International Workshop on Musical Metacreation during the Conference on Artificial Intelligence and Interactive Digital Entertainment.
Installation—recording:
How it works:
Shown at SEAMUS 2014 with Disklavier piano
Canada Fantasia for piano duo—score (PDF), recording:
The Garden of Forking Paths for string quartet—score (PDF), recording by the Apollo Chamber Players (Houston, TX):
Used as part of Where Ferin Was, installation for the I-Park 2015 Environmental Art Biennale
More about this installation here.
Live improvisation with Jayson Beaster-Jones, saxophone
Also used in experiments with synesthetic painter April Zanne Johnson (herself featured by Saatchi Art and ART MAZE Mag).
Most recent exploratory session in 2016:
https://www.youtube.com/watch?v=yLeUEdpvq4k
Initial exploratory session at the Atlantic Center for the Arts, summer 2015:
https://www.youtube.com/watch?v=ZZT1uFzCbcs
Technical details:
Low or bright light is fine (as long as visitors’ faces are visible). Could be presented with headphones if necessary, but it really flourishes in a lobby/pathway area with fluctuating traffic and a little space to stop off and look closer. The computer displays the software’s view of the visitors, to help them see how their interaction influences it. This can be shown on your own large display or projection if desired. 1–3 electrical outlets (computer and (preferred option:) digital player piano or (alternative:) 1–2 powered speakers).
Set up: 1.5 hours to connect hardware and dress the set; 2 hours to adjust the algorithm to the lighting and traffic of the space. May need an additional adjustment if traffic changes significantly when the event opens, but I can usually do this via remote connection with minimal downtime to restart the software.
Need: Digital player piano (I can seek a local sponsor to loan one) or 1–2 high quality powered speakers (nearfield/bookshelf size or larger), cables to connect my stereo 1/8″ or dual mono 1/4″ phone jacks to your speakers, and an immobile object to secure computer with a security cable (the piano would do).
I’ll bring: Computer, camera, audio/MIDI interface, stands, and cables for these items.