Self Portrait with 7 Fingers
LCO & Jonny Greenwood

We were privileged to collaborate with London Contemporary Orchestra and Jonny Greenwood on a mobile sound tool.

A few weeks ago, the London Contemporary Orchestra got in touch with us about building a tool for audience participation at an upcoming performance held at Wapping Hydraulic Power Station. Jonny Greenwood, Radiohead multi-instrumentalist and composer of scores for There Will Be Blood and The Master, was to unveil some new orchestral compositions alongside a number of his critically-acclaimed soundtrack pieces.

Among this new material was the tentatively-titled ‘Seven’, a haunting five-minute piece featuring a layer of electronically generated tones chiming over the surface at random. Greenwood and the LCO saw an opportunity to place control of these tones in the audience’s hands and asked if With Associates would be interested in bringing the idea to life. Jamie and Calum, as music enthusiasts (to put it lightly), quickly replied with “yes please of course thank you”.

As we mulled over the possibilities, it was decided that using the Web Audio API was the way forward. It is fairly well supported on popular mobile browsers, and we both wanted to get better acquainted with the technology. It would also allow us to generate sound right in the browser using only Javascript, and keep things fast and small without the need to complicate matters with apps for a range of phones and operating systems.

We dived into several ideas, including a multi-touch theremin-esque version which changed the pitch and volume of multiple oscillators according to where on the screen you touched, and a cluster of buttons that gave the user seven different tones to emit at will.

After playing with the Web Audio API for a while and getting more of a feel for what the performance would entail, we settled on the concept of a single button that would enable the audience members to play a random tone from a range of pitches specified by Greenwood.

When the button is pressed, a triangle wave is generated, given a short fade in and fade out to eliminate any harsh clicking sounds and fed through a delay module we adapted from Tuna. The delay module’s time and feedback properties are randomised every time it is triggered to create a wider scope of timbral possibilities. Happy with what we had achieved at this point, we sent it to the LCO for feedback.

// The Web Audio API is not consistent
// between browsers yet, so we have to do
// a bit of a dance to find AudioContext.
var AudioContext =
  window.AudioContext ||
  window.webkitAudioContext;

// Likewise the createGain function.
AudioContext.prototype.createGain =
  AudioContext.prototype.createGain ||
  AudioContext.prototype.createGainNode;

// You can have up to 4 audio contexts,
// so we store our instance on window.
window.audioContext =
  window.audioContext ||
  new AudioContext();

// We create and setup our gain node,
// which is in charge of volume.
var gain = audioContext.createGain();
gain.gain.value = 0.2;
gain.connect(audioContext.destination);

// We create and setup our oscillator,
// which is going to produce the tone.
var osc = audioContext.createOscillator();
osc.type = 'triangle';
osc.frequency.value = 440;
osc.connect(gain);
osc.start(0);

// Stop that annoying sound after 250ms!
setTimeout(function() {
  osc.stop(0);
}, 250);

In the initial version, several notes could be played at once, making for a bit of an ungainly racket and sometimes overloading the device’s audio engine. The idea of limiting the audience members to one tone at a time came up as both a fix and a more aesthetically pleasing answer. We tweaked the code to disable the button once triggered, re-enabling it again when the note and its delay have tailed off to an inaudible point.

Greenwood and the LCO then suggested that limiting the amount of tones an audience member could emit would gently enforce a more considered approach to performing the piece. The number 49 was settled upon — seven times seven. We added the countdown, disabling the button fully once all chances had been spent.

To make absolutely sure that the whole thing wouldn’t be for nothing, Jamie and Calum visited the venue to check there was sufficient 3G signal in the building. Surely, there could be nothing worse than asking 350 attendees to visit a website on a cripplingly slow mobile connection! Luckily, there was a couple of bars of 3G, plenty to load a small website.

One caveat of using the Web Audio API was its mobile browser support. We knew going into the project that we wouldn’t be able to support or add graceful degradation for older devices — even Windows Phones — they just don’t have the capability. Still, we believed the prevalence of modern devices meant that the majority of the audience would be able to take part, and perhaps share their devices with any disappointed neighbours.

LCOrchestra
Thank you to the brilliant and very smart @WithAssociates for creating Sunday’s @JnnyG #LCOSoloists app. #WebAudioAPI pic.twitter.com/dZraX6KMTh

DanielPioro
@LCOrchestra @withassociates @JnnyG That was masterful. Brilliantly executed.

On the night of the performance, we bundled down to Wapping and nervously bit our nails in anticipation. The piece, since titled ‘Self Portrait with 7 Fingers’, closed out an evening of incredible music. LCO conductor Hugh Brunt wordlessly directed the audience’s attention to the site’s URL projected onto the wall. Slowly but surely, the audience began sounding out a shimmering blanket of tones that hung over the Orchestra’s sustained chords. It was quite the experience, and a wonderful way to end a fantastic set.

It was a pleasure and privilege to work with Jonny and the LCO. We can’t wait for our next opportunity to work with them and the Web Audio API. Watch this space.

Self Portrait with 7 FingersLCO & Jonny Greenwood

Further reading

Self Portrait with 7 Fingers
LCO & Jonny Greenwood