Bret Victor talks about his principle that “creators need an immediate connection with what they create”.
This is brilliant stuff. Skip to 12 minutes if you don’t have much time.
Jamie Bullock
“I'm Jamie Bullock, welcome to my website. Here you'll find posts on music tech, software and ideas as well as links to my work... ”
Twitter: @jamiebullock-
Source: vimeo.com
-
Jonathan Harvey: total immersion
Seven years ago, my colleague Lamberto Coccioli and I wrote a research paper for the International Computer Music Conference outlining our intentions to develop a new software version of the electronics in Jonathan Harvey’s Madonna of Winter and Spring. Last night the system was used for the first time in public performance at the BBC Symphony Orchestra’s ‘total immersion’ event at the Barbican Centre in London.
Madonna of Winter and Spring is an incredible piece, featuring Symphony Orchestra, amplification, ring modulation, reverb, dual panning, Yamaha TX816, Yamaha DX1 and Emulator II sampler. However, these synthesis components have made the piece cumbersome to perform, with hardware versions being rare and unwieldy to setup and software versions requiring complex combinations of commercial software, some of which is already obsolete.
The new version we have developed as part of the EU-funded Integra Project, attempts to make the electronics for the piece both more sustainable and more performable. It achieves this through the use of a fully open source, entirely software based system, which simple to setup and use. This system runs on two computers and consists of:
- the Hexter DX7 emulation plugin by Sean Bolton
- a new external for the Pure Data environment, for hosting DSSI plugins
- a new software sampler implemented entirely in Pure Data
- ring modulation and panning provided by Integra Live
In the Barbican performance last night, the panning was controlled by an iPad running TouchOSC and communicating with Integra Live via wireless MIDI.
The performance was played in its entirety on BBC Radio 3’s Hear and Now programme. Moments where the synths are prominent include 1:01 - 1:07 and 1:17 - 1:20, although synthesis and electronics appear throughout.
Working with Sound Intermedia and the the BBC Symphony Orchestra this week has been a delight. It has been both exciting and rewarding to see the results of seven years’ work come to fruition. To my knowledge, this marks the first ever use of open source DX7 emulation software for a large scale orchestral work with electronics. The materials we have developed now reside with Harvey’s publishers, Faber Music, and will be used in all future performances.
This is a great achievement for the future performability of works involving DX7-based synthesis and I’d like to thank all who’ve made this possible, specifically Sean Bolton, Miller Puckette, Lamberto Coccioli, Dag Henning Kalvøy and James Dooley. At this moment, I am very proud.
-
Thoughts on #codingforkids
This week, government minister Michael Gove announced government plans proposing that current school ICT is replaced by a new Computer Science GCSE. I have no actual numbers for this, but my guess is that most people think this is a great idea. Why wouldn’t we want kids to learn to code?
The tricky part comes when we start to dig into the detail. What will the new curriculum contain? How will it be taught? Who will teach it?
One thing I am clear about is that the choice of programming language should be left to individual teachers and pupils.
What I’d like to see the new curriculum achieve:
- sharpen creative and analytical skills through projects and play
- understand abstraction and think in an abstracted way
- self-teaching through experimentation and research
- learn to modularise and classify problems and solutions
- understand algorithms, data structures, interface, and state
This is what I personally understand as ‘computational thinking’.
The end results, be it an mobile app, a game, or a robot controller and the languages used, should result from the skills and interests of teachers and students.
-
Vision: the future of music technology…
I read with great interest Bret Victor’s rant on the future of interaction design. The starting point for his post is Microsoft’s ‘Productivity Future Vision’, which is intended to be utopian, but actually gives a dystopian and de-humanising portrayal of future interaction.
Bret’s response to the video is inspiring, particularly his definition of tools:
A tool addresses human needs by amplifying human capabilities.
Furthermore:
A tool converts what we can do into what we want to do.
Reading this has left me thinking about how I can apply this concept in my own field of music technology research. Traditional musical instruments do the job beautifully: they amplify human gestures, ideas and emotions. Projects I’ve been working on recently have the aim of bringing together music and digital technology, but I feel we are still a long way from creating truly musically-meaningful tools.
I’m not really sure how we’ll get there, but my vision is that we should create digital tools that enable musicians to harness the transformative power of technology through already-learned expressive semantics. Current tools work the other way, and force musicians to understand the semantics of specific technology.
What’s your vision for the future?
*Hammer Images used by kind permission of Bret Victor
-
What makes something a musical instrument?
I’ve been thinking about this question since my final year as an undergraduate, and explored it in some detail in my M.Phil. thesis. I’ve also blogged about the subject, and discussed it many times. I’m now approaching a kind of opaque definition:
“If we take two performances of a piece, played on the same instrument, with identical articulation, and subtract one performance from the other, we’re left with the instrument”
In other words, to be called a musical instrument something must be capable of nuance.
Mechanical and analogue electronic instruments, are broadly predictable in their behaviour, but at a detailed level they constantly produce small, unpredictable variations in tone and response even with the same playing techniques.
With digital instruments, this capability for nuance and unpredictability isn’t inherent and therefore needs to be added artificially. For me, providing these nuances of interaction remains one of the key challenges of digital instrument design.
*Image by Soophoo from Wikimedia commons
-
Thoughts on sustainability
Over the last 7 years at least some part of my professional work has involved dealing with the issue of ‘sustainability’ in music technology. This has largely centred on the question of making musical works with live electronics more sustainable, for instance, by storing data in an application-neutral format or developing software versions of legacy hardware systems.
However, now I’m coming to the end of a six year research project, which has sustainability at its core, I’m starting to question whether we have achieved our sustainability goals…
What is sustainability?
The first thing we need to acknowledge is that this is a sliding scale. It’s not a question of whether something is sustainable or not, but rather how sustainable. Whilst I previously viewed this as a purely technical matter (e.g. binary file formats are less sustainable than text-based formats), I’m starting to look at the problem along a number of separate dimensions, namely: openness, funding and acceptance. A truly sustainable project needs all three.
Openness can be achieved through open licensing and the use of open standards. Funding can potentially be gained through academic or public research funding as well as commercialisation.
Possibly the most important, but also difficult aspect of sustainability to fulfil is ‘acceptance’. In order to be sustainable, software, protocols and standards need to be accepted by communities that use them. Without a community of users, contributors, supporters and developers who ‘accept’ it, a project ultimately becomes redundant.
-
How Does ‘Incremental’ Define Apple?
I’d like to discuss how Apple innovates, which I understand very well. I posted about Apple’s incremental product strategy last September at Apple Watch: “Apple Demands a High Price to Be Cool.”The pattern is consistent: Apple launches a “one more thing” product with modest hardware features but something else nevertheless killer—something people want. During the launch, Apple CEO Steve Jobs performs his marketing magic, demonstrating how this “one more thing” product will make peoples’ lives better.
Source: joewilcox
-
A book about the practice of live electronic music
…basically there isn’t one.
There seem to three classes of book referring to the subject, the most common are technical books about the theory and techniques of digital audio processing.
Examples include:
- Computer Music (Dodge)
- Elements of Computer Music (Moore)
- Introduction to Computer Music (Collins)
- The Computer Music Tutorial (Roads)
- Theory and Techniques of Electronic Music (Puckette)
These books are remarkably similar in terms of structure and content. Fundamental digital audio concepts are introduced (bits and bytes), amplitude, periodicity, digital filter theory and delays, frequency domain processing (FFT), synthesis. Additionally psychoacoustics, audio programming, and control processing are usually covered to some extent. Sometimes theory is contextualised with reference to specific musical examples.
Additionally, there are discursive books about history and aesthetics, e.g.
- Living Electronic Music (Emmerson)
- Electronic and Computer Music (Manning)
- Electronic and Experimental Music (Holmes)
These books also share common structure and content. Historical overview starting from Cage’s Imaginary Landscape #1, presentation of the main components of an electronics setup — the microphone to the loudspeaker — the evolution of digital processing, all presented from what might be considered a ‘musicological’ perspective.
Finally, we have books that deal with a specific system or software, e.g.
- The Csound Book (Boulanger)
- The SuperCollider Book (Wilson, Cottle, Collins)
- Electronic Music and Sound Design - Theory and Practice with Max/MSP (Cipriani, Giri)
- Loadbang: Programming Electronic Music with Pure Data (Kreidler)
These books take the presentation of techniques in the given software as their starting point, providing theoretical background and practical examples as needed. Techniques are sometimes presented in the context of a given musical work.
What I’m missing is a book that focuses on the music and not the technology or the historical context. So, rather than have a section on score following, with Boulez’ Anthèmes given as an example, let’s have a case study on Anthèmes with score following discussed as one of the many techniques used in the piece.
I’d also like to see discussion about the day-to-day practicalities of running concerts with live electronics. How does the process work? How is the electronics rehearsed? What should a technical rider look like? How do/should composers describe electronics in their scores? How is it notated (if at all)? What materials do publishers hold and how do we use them? How do works transfer between venues? Who maintains the electronics and how?
Also…
- examples of ‘typical’ live electronics configurations
- common combinations of processing — common musical idioms
- approaches to the problems of live electronics and musical time
- how to create effective electronics configurations — foreground/background
- how to create effective sense of interaction for the performer and audience
- electronics and musical style — composers and ‘schools’
- scale — differences in approach between small-scale and large-scale works
I guess I’m looking for something between Introducing Music (Karolyi) and Designing Sound (Farnell), at least in spirit.
Maybe I’ll sketch out a chapter outline in a future post…
-
Max, Integra and the Turing Tar Pit
With Integra Live, we’re trying to make live sound processing simple for musicians. We achieve this by hiding the complexity of signal processing from our users and exposing only musically-useful controls. This technique is known as abstraction because the user-facing representation is more abstract than that used in the underlying software.
Integra Live modules — simple-to-use, but the implementation details are hidden:
The problem with abstraction is that it limits possibilities. Audio programming environments like Pure Data and Max, remove this limitation by providing low-level processing objects that allow the user to make custom abstractions. This means that users coming from a Max or Pd background often find Integra Live constraining. Conversely, Max represents a Turing Tar Pit, requiring significant technical skill for even basic musical tasks.
Bottom-up approach in Max — the TapDelay~ abstraction is created by connecting low-level objects:
This highlights an interesting design problem: how do we build sound-processing software that makes simple things simple and complex things possible?
My first instinct is that we should invert the Max paradigm and create a top-down approach where high-level modules can be edited through an integrated UI.
Any other thoughts…?
-
The art of illusion
This week I had the pleasure of working with the Palindrome dance company for a performance at Birmingham Conservatoire. The company is led by Robert Wechsler, who has spent 10 years researching and developing the EyeCon video motion tracking platform in collaboration with developer Friedrich Weiss.
As part of the performance, the group gave an hour-long workshop in which they talked about their techniques and how they were used. I was particularly interested to hear about their use of motion tracking since Palindrome have developed some of the most effective work I have seen, apparently achieving extremely accurate synchronisation between movement (gesture) and sound (music). The sync seems accurate in terms of both spatial and temporal localisation, for example dancers can trigger sonic events with tiny movements of their fingers or eyes even at some distance from the camera.
I use words like ‘seems accurate’ and ‘apparently’, because it became evident from the workshop that unlike many motion capture systems, EyeCon doesn’t work by giving accurate data about the dancer’s location in 3D space, neither does it capture data about the position of individual body parts. Instead it takes a 2D window of the space and tells us where the dancer is in that window, whether they are intersecting a marked area, the amount of movement within that area and degree of left-right symmetry. It can’t tell us the dancer’s absolute height, but it can tell us if they appear bigger or smaller within the window.
The word ‘appearance’ is critical here because in many regards EyeCon and the Palindrome approach is more about how things appear than how they are. In the following video clip, for example, it appears as though the motion capture system is accurately tracking the movements of eyes and mouth. In reality, it has nothing like that level of detail. Instead it knows roughly where movement is occurring, and the intensity of that movement. The appearance of eye movement controlling sound is an illusion, effectively created through the correlation of movement, motion sensing and audio triggering. We see the eyes move and we hear a sound, so our brains assume that there is some connection between eye movement and sound production. This may be the case, but often the system will just wait for any movement within a given window region and then react.
A Human Conversation from Jamie Bullock on Vimeo.
This approaches something I have used in my own work for a long time, and which I believe very strongly, which is that the audience’s perception of what happens is far more important than what actually happens. The obvious analogy in the world of live electronic music is score following. State of the art score following typically uses a stored model of the musical work (a representation of the score), and tries to match incoming to this model thereby giving a precise temporal location within the score as output. This approach is OK for music where a high degree of reproducibility is required, but personally I’m more interested in creating a more organic sense of interaction for the performer and audience. That is, I’m not so interested in where the performer is within a score, but rather where they are in the space of musical possibilities.
So as live electronics musicians, what can we learn from Palindrome and their approach? I think the key for me is that we need to ‘think smarter’ about how we use our technical resources in order to create synergies between art and technology where the whole is greater than the sum of constituent parts. We need to always be aware of perception and how the human mind always tries to connect cause and effect. For me this is an interesting space to play with both technically and artistically.
