A few things

twitter: @JosephErnest
email: here

Articles about:
#music
#photo
#programming

Don't read #tech articles except you really want to.

Some of my projects:
BigPicture
Jeux d'orgues
SamplerBox
Ojourdui

Low latency audio on a Windows PC with the built-in soundcard (bonus: it's multi-client!)

So you want to use your music production software, with low latency on your PC/Windows laptop?

You have basically two options:

ASIO4ALL is incredibly useful for the PC music community since more than 10 years, because it turns your cheap computer's built-in soundcard into a low-latency one! With ASIO4ALL, you can plug a MIDI keyboard and play piano or synth with no "delay". Without it, the delay of more than 50 ms between the keypress and the sound makes it nearly impossible to play.

But ASIO4ALL has one major drawback: it's not multi-client. This means that if your DAW is open with ASIO4ALL as sound driver, then, if you open:

... then it won't work: the audio is not available for them: your DAW and ASIO4ALL have locked your soundcard.

This is really annoying and I can't count how many hours of my life I wasted since 10 years to find a solution for this (every few months/years I retried and retried and benchmarked every new solution). (Ok switching to Mac would have been a faster solution...)

The real difficulty is that we would like to use

(*) A music software in ASIO + a standard application like Firefox using the so-called Windows WDM driver

Here is a list of things I tried, unsuccessfully:

Now, promising solutions:

Here is AsioLinkPro's clever idea: you still use ASIO4ALL as output, but this way (**):

Ableton Live (or any other DAW)    --> ASIO: AsioLinkPro                  \
                                                                            --- AsioLinkPro mixer --> ASIO4ALL
Firefox or Chrome or SoundForge    --> WDM: ASIOVADPRO virtual device     /
     or MP3 player                          (AsioLinkPro)

Clever, because even if there are 2 programs producing sound, AsioLinkPro is the only one which speaks directly with ASIO4ALL (which would not support 2 programs).

It must have been tricky to code it because it requires to code a "WDM virtual speaker device" Windows driver + an ASIO driver, phew!

Even if it's discontinued, at least it gives an idea about how to do it. Let's write such a minimalist open-source tool?

Note: not something very big and complex like Jack, but just a small WDM virtual speaker driver and an ASIO driver that both mix their content and send it to the ASIO4ALL output. (No GUI is even required).

How to find seamless loops in audio files (with a little bit of math and programming)

When making instrument sample sets (e.g. church organ sample sets used with Hauptwerk or GrandOrgue, see my project Jeux d'orgues), we need to set looping points in WAV audio files:

such that when playing the part [a, b] in loop, we don't hear any click or pop when the sample reaches the end of the loop.

Example 1: bad loop with audible clicks

Example 2: seamless loop with no click, that's what we are looking for! The loop has a ~ 2.670 second period, can you hear where are the looping points?

 

Finding looping points can be done manually but this is a very long and tedious task. A few programs exist to do this process automatically such as Extreme Sample Converter (it has an excellent auto-looping algorithm), LoopAuditioneer (open source), Zero-X Seamless Looper, SampleLooper, etc.

Here we'll look at a home-cooked algorithm that works well to detect looping points.

First of all, let's load the audio file (downloadable here) with Python:

from scipy.io import wavfile
import numpy as np
import itertools

sr, x = wavfile.read('060.wav')
x0 = x if x.ndim == 1 else x[:, 0]     # let's keep only 1 channel for simplicity, but we could easily generalize this for 2 channels
x0 = np.asarray(x0, dtype=np.float32)

Let's say the audio file's sustain part (this is precisely where we're looking for a loop!) begins at t=2 sec and finishes at t=9 sec. We will now subdivide the time-interval [2 sec, 9 sec] into a 250 milliseconds grid: 2, 2.25, 2.5, 2.75, 3, 3.25, ..., 8.75, 9.

From this sequence, we now create "loop candidates" (a, b) of length at least 1 second, example: (2.5, 7.5), (3.25, 5.75), (6.0, 8.75), etc. Then, for each loop candidate, we'll improve the loop (this is the core of the algorithm, it will be discussed in the next paragraph) and compute a distance d. We finally keep the loop that has the minimal distance (among all loop candidates). Finished!

A = [int((2 + 0.25 * k) * sr) for k in range(29)]  # the grid 2, 2.25, 2.5, ... 8.75, 9
dist = np.inf
for a, b in itertools.product(A, A):  # cartesian product: pairs (a, b) of points on the grid
    if b - a < 1 * sr:
        continue
    a, B, d = improveloop(x0, a, b, sr=sr)
    print 'Loop (%.3fs, %.3fs) improved to (%.3fs, %.3fs), distance: %i' % (a * 1.0 / sr, b * 1.0 / sr, a * 1.0 / sr, B * 1.0 / sr, d)

    if d < dist:
        aa = a
        BB = B
        dist = d 

print "The final loop is (%.3fs, %.3fs), i.e. (%i, %i)." % (aa * 1.0 / sr, BB * 1.0 / sr, aa, BB)

Finished? Not yet! We need to explain what we mean by improving a loop, as that's the crucial part of the algorithm. More precisely, we'll now explain how to transform a loop (3.25, 5.75) with points taken on the grid (this random loop probably "clicks" like in Example 1 before!) into a "good loop" (3.25, 5.831). Let's zoom on the junction point to understand what's going on:

How to measure if a loop is good or not? Ideally, if the loop (a, b) is perfect/seamless, x[a:a+10 ms] should be very close to x[b:b+10 ms]. Measuring how close two arrays x and y are can be done by computing sum((x[n]-y[n])^2), and if the sum is small, x and y are close.

Finding k such that np.sum(np.abs(x0[a:a+W1]-x0[k+b:k+b+W1])**2) is minimal can be obtained by noting that

(x[n] - y[n+k])**2  = x[n]**2 - 2*x[n]*y[n+k] + y[n+k]**2

and by using numpy.correlate. We can now define this function:

def improveloop(x0, a, b, sr=44100, w1=0.010, w2=0.100):
    """
    Input:  (a, b) is a loop
    Output: (a, B) is a better loop 
            distance (the less the distance the better the loop)
    This function moves the loop's endpoint b to B (up to 100 ms further) such that (a, B) is a "better" loop, i.e. sum((x0[a:a+10ms] - x0[B:B+10ms])^2) is minimal
    """

    W1 = int(w1*sr)
    W2 = int(w2*sr)
    x = x0[a:a+W1]
    y = x0[b:b+W2]
    delta = np.sum(x**2) - 2*np.correlate(y, x) + np.correlate(y**2, np.ones_like(x))
    K = np.argmin(delta)
    B = K + b
    distance = delta[K]

    return a, B, distance

That's it, in less than 50 lines of Python code!

This audio file

(looped 4 times here but we could loop it forever) has been obtained with the algorithm described here. Not too bad, n'est-ce pas?


Example of output:

Loop (2.000s, 3.000s) improved to (2.000s, 3.009s), distance: 1003724800
Loop (2.000s, 3.250s) improved to (2.000s, 3.340s), distance: 839278592
Loop (2.000s, 3.500s) improved to (2.000s, 3.559s), distance: 1281863680
[...]
Loop (2.000s, 8.500s) improved to (2.000s, 8.544s), distance: 1092337664
Loop (2.000s, 8.750s) improved to (2.000s, 8.789s), distance: 964747264
Loop (2.000s, 9.000s) improved to (2.000s, 9.004s), distance: 2488913920
[...]
Loop (7.750s, 9.000s) improved to (7.750s, 9.004s), distance: 1167093760
Loop (8.000s, 9.000s) improved to (8.000s, 9.001s), distance: 1710333952

The final loop is (6.750s, 8.322s), i.e. (297675, 366989).

Note: Wouldn't it be possible to save these loop markers inside the WAV file's metadata instead of just printing them on screen? Sure it is, but as Python's standard library doesn't support WAV markers editing, you'll have to use these techniques to do this.

Working with audio files in Python, advanced use cases (24-bit, 32-bit, cue and loop markers, etc.)

Python comes with the built-in wave module and for most use cases, it's enough to read and write .wav audio files.

But in some cases, you need to be able to work with 24 or 32-bit audio files, to read cue markers, loop markers or other metadata (required for example when designing a sampler software). As I needed this for various projects such as SamplerBox, here are some contributions I made:

  1. The Python standard library's wave module doesn't read cue markers and doesn't support 24-bit files. Here is an updated module:

    wave.py (enhanced)

    that adds some little useful things. (See Revision #1 to see diff with the original stdlib code).

    Usage example:

    from wave import open
    
    f = open('Take1.wav')
    print(f.getmarkers())

    If you're familiar with main Python repositery contributions (I'm not), feel free to include these additions there.

  2. The module scipy.io.wavfile is very useful too. So here is an enhanced version:

    wavfile.py (enhanced)

    Among other things, it adds 24-bit and 32-bit IEEE support, cue marker & cue marker labels support, pitch metadata, etc.

    Usage example:

    from wavfile import read, write
    
    (sr, samples, br, cue, cuelabels, cuelist, loops, f0) = read('Take1.wav', readmarkers=True, readmarkerlabels=True, readmarkerslist=True, readpitch=True, readloops=True)
    print read('Take1.wav', readmarkers=True, readmarkerlabels=True, readmarkerslist=True, readpitch=True, readloops=True)
    
    write('Take2.wav', sr, samples, bitrate=br, markers=cue, loops=loops, pitch=130.82)
    print read('Take2.wav', readmarkers=True, readmarkerlabels=True, readmarkerslist=True, readpitch=True, readloops=True)
    
    write('Take3.wav', sr, samples, bitrate=br, markers=cuelist, loops=loops, pitch=130.82)

    Here is a Github post and pull-request about a hypothetical merge to Scipy.

Here is how loop markers look like in the good old (non open-source but soooo useful) SoundForge:


Lastly, this is how to convert a WAV to MP3 with pydub, for future reference. As usual, do pip install pydub and make sure ffmpeg is in the system path. Then:

from pydub import AudioSegment
song = AudioSegment.from_wav("test.wav")
song.export("test.mp3", format="mp3", bitrate="256k")

will convert a WAV file to MP3.

Get the reverb impulse response of a church

I recently recorded an impulse response of the reverb of a 14th-century church (more or less the footprint of the sound ambiance of the building). Here is how I did it.

Quite a lot of reverb, that's exactly what we want to catch with an IR!

Then, of course, we can do some cleaning, fade out, etc.


But what is this useful for? You can use this Impulse Response in any music production software (the VST SIR1 is quite good and freeware) , and make any of your recordings (voice, instrument, etc.) sound like if they were recorded in this church. This is the magic of convolution reverb!


Useful trick when you record your own IR: play sweep0.wav in the building instead of sweep.wav. The initial "beep" is helpful to see exactly where things begin. If you don't do that, as the sweep begins with very low frequencies (starting from 20 Hz), you won't know exactly where is the beginning of your microphone-recording. Once your recording is done, you can trim the soundfile by making it begin exactly 10 seconds after the short beep.

Some related reading in this topic, and this blog post.

Older articles