Sony CSL Paris develops technology for AI-assisted music production. The goal is not to replace musicians, but to provide them with better tools to be more efficient in realizing their creative ideas. DrumNet is based on an artificial neural network which learns rhythmic relationships between different instruments and encodes these relationships in a 16-dimensional style space. A similar example is the Logic Pro X Drummer, allowing the user to specify the playing style by navigating a two-dimensional space. The difference of DrumNet to the Logic Pro X Drummer, however, is that it dynamically adapts to the existing music. In its current form, DrumNet can either autonomously generate kick drum tracks (following the statistics of the training data), be controlled by manually navigating the style space, or be used to extract a style from an existing piece.

As opposed to many other generative music technologies, we aim to directly use existing audio tracks as input to which we generate the kick drum track as audio output. Using audio input directly makes DrumNet more useful for music production than models based on MIDI input. We show the generality of the model, by providing many examples of full songs with different generated kick drum tracks on this website. For a proof-of-concept, we trained the model only on kick drum rhythms, but we are currently extending the model to generate a whole drum set.

Please use headphones or good speakers to listen to the examples, as kick drums are not audible otherwise.

The video below shows a demo of the DrumNet prototype, which uses models trained on different percussion types (while in the paper only the kick drum model is presented).

Style Transfer

In this section, a style vector is inferred from source songs and applied to target songs in order to transfer the "style" to play the kick drum to other tracks. The similarities are not always obvious, as the output is not only determined by the style vector, but also by the way bass and snare is played (the style vector defines the kick drum in relation to them).

Orgs Waltz has a very regular electronic kick drum hitting at every beat. In most target songs, this characteristic is reproduced.

In the style extracted from Drehscheibe the kick drum tends to play on beat 1 and 3 of a bar, and sometimes on beat 2 and 4 when a strong bass onset is present.

The kick drum of Gipsy Love has a tendency to play shortly before and at the downbeat, and concurrently with the bass. This tendency can be observed when the style is transferred to the other pieces. Note that Gipsy Love has a time signature of 3/4 but its style can still be reasonably transferred to the other songs with 4/4 measure.

Miss you provides a style vector which causes the kick drum to form a pickup before the actual beat (before 1 or 3).

Tempo Invariance

Here, we extracted the style from each song in original tempo, and applied it to the same song in time-stretched variants.

This results in kick drum tracks which are almost identical, but adjusted to the correct time.

This shows that style vectors are indeed tempo-invariant, and the model automatically adjusts the output tempo to the input tempo.

Please use headphones or good speakers to listen to the examples, as kick drums are not audible otherwise.