Jamie Bullock

The Listening Machine

The web is literally buzzing about The Listening Machine, a new musical work by Daniel Jones and Peter Gregson. The piece takes the content of 500 Twitter accounts, and through some clever computer programming and musical mapping turns them into an indefinite bricolage of sound, which somehow represents their content.

The result is one of the most successful pieces of data sonification I’ve heard. What’s great for me is the clear relationship between the twitter stream content and the end result. If you asked me “what would Twitter sound like as music?”, The Listening Machine isn’t far off. It’s also aesthetically interesting, visually and musically — I wan’t to keep coming back to hear “what’s happening now?”, “what does Twitter sound like at night time?”.

The Listening Machine

The piece also presents some interesting ideas about music and technology. The question “can machines listen?”, often has the subtext “…like humans”. In The Listening Machine, we have a machine that listens to humans like a machine, and generates music from fragments played by humans. Technology and art serve each other. Beautiful.

The piece is running from May until October 2012 on The Space, the new on-demand digital arts channel from the BBC and Arts Council England.

Published: May, 2012.

How does design work?

I’ve been talking recently about researchers collaborating with designers to make better systems for musicians. But what do I actually mean by that? This article gives a lovely, concise overview on the different types of design, who designers are, and why they are important for making software usable:

http://startupsthisishowdesignworks.com/

Published: April, 2012.

Musician-centred Design

Software as medium

In live electronic music, software mediates the experience of composition and performance. Software is the medium through which ideas are developed and structured, it is the medium through which sound is generated and processed, and it is the medium through which sound is controlled. The nature and design of the software is therefore critical to an understanding of the ways in which it mediates user experience. Well-designed software can lead to a pleasant and productive experience, poorly-designed software can lead to frustration and rejection.

Max and the myth of neutral software

It’s not just a question of learning and software literacy. It’s true that the notion of ‘intuitive’ software is a myth, but software that draws upon users’ existing experiences has been shown to be more learnable and therefore more usable. These references to existing experience are called affordances. Affordances can be used to introduce novelty by calling the user to a given action. Software can also be made more usable by introducing assumptions about how the software will be used. These assumptions can be embodied in structured workflows, which mediate the users’ experiences in such a way that it reinforces their intentions.

The primary means for people to work with live electronic music is currently Max by Cycling 74. Max is a tour de force of software engineering, and incredibly powerful on a number of levels. It was originally written in the mid-1980’s as means to control hardware-based audio processing systems. It has since evolved to become very broad in scope, the current website says:

Max gives you the parts to create unique sounds, stunning visuals, and engaging interactive media. These parts are called ‘objects’ – visual boxes that contain tiny programs to do something specific. Each object does something different. Some make noises, some make video effects, others just do simple calculations or make decisions. In Max you add objects to a visual canvas and connect them together with patchcords. You can use as many as you like. By combining objects, you create interactive and unique software without ever writing any code (you can do that too if you really want to). Just connect.

Max makes few assumptions about what it will be used for beyond providing a low-level dataflow and DSP patching language for the creation of anything related to sonic and visual media. Max therefore presents itself as a ‘neutral’ tool for creative work, a tool where ‘the only limit is your imagination’. However, in reality Max has deeply embedded in it, the non-neutral assumptions of a software development workflow. Max is actually an advanced integrated development environment (IDE) for programming multimedia interaction and processing. Far from being a neutral tool, Max mediates the experience of the musician by imposing the workflow, prerequisites and aesthetics of software development on their user experience. With Max, the musician is programmer, instrument maker, designer and composer all in one. Many would argue this to be a positive trait, and in some respects it can be.

Musician-centred design

I believe that musicians need and deserve software that is better tailored to their specific requirements. They need software that has embedded in it, the assumptions of the musician, not the assumptions of the programmer. I call this musician-centred design. The way in which the software mediates the experience of the musician should be centred around the existing knowledge, practices and thought processes of composers and performers. Workflows should be structured around achieving the most common tasks quickly and easily. Interactive music technology has been through many experimental phases, we now know the creative possibilities are ‘endless’. It’s time to consolidate these possibilities and make them more accessible and more musically meaningful.

Some aims of musician-centred software:

Software should encode the assumptions of the musician in how it operates
Workflows should be structured around musical tasks
Software should draw upon musicians’ existing experience
Software should employ innovative UI design to achieve its ends

Conclusions

Max is a wonderful tool for the technically literate who want to employ creative coding to achieve their goals. However, there are many musicians for whom Max is simply too distant from their existing experiences to even approach. I believe the central challenge ahead for music technology research is to investigate and develop new user interfaces that are musician-centred. I see many possibilities here, and feel we haven’t started to scratch the surface in solving these problems. I don’t have the solutions yet, but I believe that collaboration between music technology researchers and designers should be the starting point. Comments welcome…

Published: March, 2012.

Bret Victor talks about his principle that “creators need an immediate connection with what they create”.

This is brilliant stuff. Skip to 12 minutes if you don’t have much time.

Source: vimeo.com

Published: February, 2012.

Jonathan Harvey: total immersion

Seven years ago, my colleague Lamberto Coccioli and I wrote a research paper for the International Computer Music Conference outlining our intentions to develop a new software version of the electronics in Jonathan Harvey’s Madonna of Winter and Spring. Last night the system was used for the first time in public performance at the BBC Symphony Orchestra’s ‘total immersion’ event at the Barbican Centre in London.

Madonna of Winter and Spring is an incredible piece, featuring Symphony Orchestra, amplification, ring modulation, reverb, dual panning, Yamaha TX816, Yamaha DX1 and Emulator II sampler. However, these synthesis components have made the piece cumbersome to perform, with hardware versions being rare and unwieldy to setup and software versions requiring complex combinations of commercial software, some of which is already obsolete.

The new version we have developed as part of the EU-funded Integra Project, attempts to make the electronics for the piece both more sustainable and more performable. It achieves this through the use of a fully open source, entirely software based system, which simple to setup and use. This system runs on two computers and consists of:

the Hexter DX7 emulation plugin by Sean Bolton
a new external for the Pure Data environment, for hosting DSSI plugins
a new software sampler implemented entirely in Pure Data
ring modulation and panning provided by Integra Live

In the Barbican performance last night, the panning was controlled by an iPad running TouchOSC and communicating with Integra Live via wireless MIDI.

The performance was played in its entirety on BBC Radio 3’s Hear and Now programme. Moments where the synths are prominent include 1:01 - 1:07 and 1:17 - 1:20, although synthesis and electronics appear throughout.

Working with Sound Intermedia and the the BBC Symphony Orchestra this week has been a delight. It has been both exciting and rewarding to see the results of seven years’ work come to fruition. To my knowledge, this marks the first ever use of open source DX7 emulation software for a large scale orchestral work with electronics. The materials we have developed now reside with Harvey’s publishers, Faber Music, and will be used in all future performances.

This is a great achievement for the future performability of works involving DX7-based synthesis and I’d like to thank all who’ve made this possible, specifically Sean Bolton, Miller Puckette, Lamberto Coccioli, Dag Henning Kalvøy and James Dooley. At this moment, I am very proud.

Published: January, 2012.

Thoughts on #codingforkids

This week, government minister Michael Gove announced government plans proposing that current school ICT is replaced by a new Computer Science GCSE. I have no actual numbers for this, but my guess is that most people think this is a great idea. Why wouldn’t we want kids to learn to code?

The tricky part comes when we start to dig into the detail. What will the new curriculum contain? How will it be taught? Who will teach it?

One thing I am clear about is that the choice of programming language should be left to individual teachers and pupils.

What I’d like to see the new curriculum achieve:

sharpen creative and analytical skills through projects and play
understand abstraction and think in an abstracted way
self-teaching through experimentation and research
learn to modularise and classify problems and solutions
understand algorithms, data structures, interface, and state

This is what I personally understand as ‘computational thinking’.

The end results, be it an mobile app, a game, or a robot controller and the languages used, should result from the skills and interests of teachers and students.

Published: January, 2012.

Vision: the future of music technology…

I read with great interest Bret Victor’s rant on the future of interaction design. The starting point for his post is Microsoft’s ‘Productivity Future Vision’, which is intended to be utopian, but actually gives a dystopian and de-humanising portrayal of future interaction.

Bret’s response to the video is inspiring, particularly his definition of tools:

A tool addresses human needs by amplifying human capabilities.

Furthermore:

A tool converts what we can do into what we want to do.

Reading this has left me thinking about how I can apply this concept in my own field of music technology research. Traditional musical instruments do the job beautifully: they amplify human gestures, ideas and emotions. Projects I’ve been working on recently have the aim of bringing together music and digital technology, but I feel we are still a long way from creating truly musically-meaningful tools.

I’m not really sure how we’ll get there, but my vision is that we should create digital tools that enable musicians to harness the transformative power of technology through already-learned expressive semantics. Current tools work the other way, and force musicians to understand the semantics of specific technology.

What’s your vision for the future?

*Hammer Images used by kind permission of Bret Victor

Published: November, 2011.

What makes something a musical instrument?

I’ve been thinking about this question since my final year as an undergraduate, and explored it in some detail in my M.Phil. thesis. I’ve also blogged about the subject, and discussed it many times. I’m now approaching a kind of opaque definition:

“If we take two performances of a piece, played on the same instrument, with identical articulation, and subtract one performance from the other, we’re left with the instrument”

In other words, to be called a musical instrument something must be capable of nuance.

Tone arm

Mechanical and analogue electronic instruments, are broadly predictable in their behaviour, but at a detailed level they constantly produce small, unpredictable variations in tone and response even with the same playing techniques.

With digital instruments, this capability for nuance and unpredictability isn’t inherent and therefore needs to be added artificially. For me, providing these nuances of interaction remains one of the key challenges of digital instrument design.

*Image by Soophoo from Wikimedia commons

Published: October, 2011.

Thoughts on sustainability

Over the last 7 years at least some part of my professional work has involved dealing with the issue of ‘sustainability’ in music technology. This has largely centred on the question of making musical works with live electronics more sustainable, for instance, by storing data in an application-neutral format or developing software versions of legacy hardware systems.

Stockhausen Electronics

However, now I’m coming to the end of a six year research project, which has sustainability at its core, I’m starting to question whether we have achieved our sustainability goals…

What is sustainability?

The first thing we need to acknowledge is that this is a sliding scale. It’s not a question of whether something is sustainable or not, but rather how sustainable. Whilst I previously viewed this as a purely technical matter (e.g. binary file formats are less sustainable than text-based formats), I’m starting to look at the problem along a number of separate dimensions, namely: openness, funding and acceptance. A truly sustainable project needs all three.

Sustainability Dimensions

Openness can be achieved through open licensing and the use of open standards. Funding can potentially be gained through academic or public research funding as well as commercialisation.

Possibly the most important, but also difficult aspect of sustainability to fulfil is ‘acceptance’. In order to be sustainable, software, protocols and standards need to be accepted by communities that use them. Without a community of users, contributors, supporters and developers who ‘accept’ it, a project ultimately becomes redundant.

Published: September, 2011.

How Does ‘Incremental’ Define Apple?

joewilcox:

I’d like to discuss how Apple innovates, which I understand very well. I posted about Apple’s incremental product strategy last September at Apple Watch: “Apple Demands a High Price to Be Cool.”

The pattern is consistent: Apple launches a “one more thing” product with modest hardware features but something else nevertheless killer—something people want. During the launch, Apple CEO Steve Jobs performs his marketing magic, demonstrating how this “one more thing” product will make peoples’ lives better.

Read More

Source: joewilcox

Published: June, 2011.