After many very long days, My project Domifare is working. For me. It won’t work for you because there is a bug in TuningLib. I have raised an issue, which the package maintainer will get to shortly. The package maintainer, who is me will fix it shortly. When I get back from Austria. I need to test my fix properly.
Only a subset of specified commands have been implemented, but I can record a loop and re-order the playback of a loop based on detected onsets. Hypothetically, I can also start and stop loops. In practice, pitch detection is terrible and the language is barely usable. Annoyingly, the utility of it depends on how good my tuba playing sounds.
If I want to use this as an actual tool, the way forward is playing the key phrases in as training data to an AI thing.
While writing this project, I raised three issues with the SuperCollider project over documentation and one issue with the LinuxExternals Quark over Pipewire. That will turn into a merge request. I might update the documentation for it.
If you want to hear this thing in progress, I’ll be using it on Friday. You can turn up in person to Linz, Austria or tune into the live stream. This is part of AMRO, who have a helpful schedule.
I feel like a zombie and will say something more coherent later.
I put in a bid to play Domifare at AMRO, knowing it was in no state to perform, but also knowing that nothing motivates like a deadline. I thought it was likely to be accepted, so I planned to start working on it during the break between spring and summer terms.
But then I got covid and felt terrible for weeks, but also got brain fog which, to be honest, has not completely dissipated. I mean, it’s hard to tell. How could I possibly have a bassline on my mental state? I do know that my sense of taste is still messed up and if I exercise a lot I feel ill the next day, so let’s say I’m not at 100% mentally. It could be all in my head, but what difference would that make?
I wanted to finish my marking before dedicating all my time to this. I have not finished my marking, but now both are an emergency. Indeed, the list of things I have not done is kilometres long. My tuba needs a service. I haven’t played it for months and lips are completely unfit.
This is an overly-honest research update. The subject line is the Solresol word for “fear.”
This is the state of the language:
The “language” has always been conceived of as a way of defining loops. So I have some syntax for recording loops as an audio recording or as a series of onsets, the ability to “shake” an onset loop, the ability to schedule shakes, and the ability to start and stop loops. These are all a series of short musical licks I should ideally memorise but at least be able to play without hesitation or split notes.
Meanwhile, the language currently has the ability to read and receive notes, which is necessarily flaky and a scaffold to hand the rest of the operations …. and a GUI to adjust thresholds because that’s necessary while playing… and that’s kind of it.
This coming weekend is a four day one which is actually a disaster because it means I can’t work during it.
Writing out what I actually have to do makes it sound fully achievable, but it will take longer than I think it will. The GUI took all of yesterday. If I spend part of every day programming and part of every day practising, I should get there. Hopefully.
I haven’t bought my train tickets yet, but I really don’t want to drop out.
Jitsi Meet works best in Chromium, so these instructions are for that browser. It also assumes that your live coding language uses Jack Audio.
Jack and Pulse
The first thing you need to do is to get Pulse Audio and Jack Audio running at the same time and talking to each other. For me, I was able to solve this in Qjackctl. In settings, under the option tab, go to Execute Script after startup. Paste in “pactl load-module module-jack-sink channels=2; pactl load-module module-jack-source channels=2; pacmd set-default-sink jack_out” without the quotes.
You can test if both are running at once by starting jack and then playing audio from a normal system program like your web browser. If you hear sounds, it worked.
More information about getting both running is here.
You get two sinks. One of them is going to be used to send audio into Jitsi and the other will be used to get audio out.
Start your live coding language audio server. (Ie Boot SuperCollider). Go back to Qjackctl. Click on connections. Go to the Audio tab. Make a connection from your live coding language output to Pulse Audio Jack Source-01 input. Do this by clicking on the language in the left column and the Source-01 in the right column so both are highlighted. Then click the “Connect” button on the lower left.
Disconnect the the system output from that Source’s input if you want to turn off your microphone. Do this by clicking on the system in the left column and Source-01 in the right column and clicking the “Disconnect” button.
(These will also work for Chrome.)
First open your Jitsi Meet connection. If you are using the server at Meet.jit.si, you can skip this step.
As we’re live coding, you’ll want to share your screens. Mouse over the screen so you can see the icons at the bottom appear. The one in the lower left corner looks like a screen. Click on it.
It lets you pick a video source. If you’re on Wayland, like most linux users have been for years now, you can share your entire screen, but you should be able to share a single window. If you don’t see your live coding language listed as a window, make sure Chromium and it are on the same virtual desktop.
Click the screen icon again to switch back to your webcam.
Fancy New Options
If you’re on a shiny new version of jitsi, such as the one at Meet.jit.si, You’ll see little carrots by the mic and video icons in the centre bottom of your browser window.
These allow you to pick your audio source without having to go into Chrom/ium settings. If you have more than one webcam, you can also pick which one you want to use there, without having to go into Chrom/ium settings for that either.
Be Careful of Levels!
Jitsi was made assuming that more or less one person would be talking at a time, so multiple streams at full volume can distort. Make sure to leave your output low enough to leave room for your collaborators. Your system volume controls will not work, so be sure to set the levels in your live coding language IDE.
Also be aware that while the compression sounds good for free improvisation on acoustic instruments, the transformations on digital audio will seem more pronounced. Go with it. This is your aesthetic now.
As threatened, I have once again made some Christmas music.
If you enjoy (or hate it, or are indifferent), please consider donating to the UKLGIG. cafdonate.cafonline.org/111#/DonationDetails They support LGBTQI+ people through the asylum and immigration process. Their vision is a world where there is equality, dignity, respect and safety for all people in the expression of their sexual or gender identity.
The notes in the 5 movements all come from the same pop song, but in 4 of the movements, they pass through a class I (accidentally) wrote called MidiMangler. It’s undocumented, but the constructor expects the kind of midi events that come from SimpleMIDIFile in wslib and the .p method spits out a pbind.
The instruments are some of the sample I used a couple of years ago, but the organ is new. It’s based on one from http://sccode.org/1-5as but modified to be played with a PmonoArtic.
The other instruments are the default synthdef *cough*, a Risset bell and Karplus Strong – taken directly from a help file with no changes. These are presented at the bottom for the sake of completion. The other sound is a bomb sample I found on freesound.
The video is taken from an atom bomb test video, but slowed down and stretched. I used ffmpeg to do this. The original film was 24 frames per second. I used a ffmpeg filter to create a lot of extra in-between frames and then, separately, changed the frame rate to be much slower. The original film was a bit over 20 seconds and got stretched out to 15 minutes. The really low frame rate is a bit choppy, but I think more tweening would have just increased distortion. The commands for that were:
The other day, I read someone putting for the idea that apocalyptic thinking is so profoundly unhelpful as to be self-indulgent. Climate change is not going out with a bang, but a very prolonged whimper, whilst, for the duration, failing to make any significant changes. We can address it and avoid many of the worst impacts, but we need to get very serious about it immediately. If we can build thousands of expensive, terrifying bombs just in case there might be a war nobody wants, surely, we can afford to spend some of that resource averting a disaster that we know is actually coming.
This is the fourth part of a series of posts about how I created my album Christmaswave. Previously, part 1 posted a list of questions and answered the first two of them: ‘What sample am I going to play?’ and ‘How many times am I going to divide it in half?’. Part 2 answered the question, ‘Once it’s chopped into little (or not-so-little) pieces, which one of them am I going to play?’. Part 3 talked endlessly in the meandering way of hangovers about playback rates.
One thing I have not addressed is how I picked which parts of source material to use. Unless the words were compelling in some way, I tended to go for phrases that were instrumental. In some songs, this left me only the intro and the outro. This is why most of the pieces have source material taken from multiple versions of the same song.
How much should an event overlap whatever comes after?
Normally, you would do this by setting the \legato part of the event, which is ‘The ratio of the synth’s duration to the event’s duration.’ I set this to 1.1 in most of the pieces, which helped make up a bit in case the slicing was in slightly the wrong place.
Most I varied:
\legato, Pwhite(0.8, 1.5)
Some depended on the duration of the it of sample I’m playing. This is what I did with Sledge Trudge
How long should I wait before going to the next thing (which might be a repetition of what I just did)?
This is a question as to what to put for the \dur part of the event, which depends on the number of frames of sample that we're playing, the sample rate of it, and the playback rate.
I pulled a lot of my source material from youtube - by using a browser plugin to download and convert to mp4 and the using audacity to convert to wav files. These files were often 48k, which is the standard for film, but audio is generally at 44.1k, so I did have to take the source file's sample rate into account.
It makes sense to calculate the start frame and duration together. This example is from Out in the Cold:
The glitchy repeating of the same event multiple times is a repetition of the entire event, not just some parts of it. I did this with Pclutch, which has a slightly weird syntax.
The pattern is just your Pbind or similar, but the connected part is slightly odd. If it's true, the Pbind is evaluated to produce a new value. It's it's false, the previous event is repeated. It's possible to use a pattern to control this.
Starting with the Pseq means that the Pclutch will repeat 4 times, then make a new event. After that, it picks randomly from 3 Pseqs. The first repeats twice, the second 3 times and the third 4 times. It will randomly pick between these repetitions for 100 times and then stop.
This is part 3 of a series about how I created my album Christmaswave. Previously, part 1 posted a list of questions and answered the first two of them: ‘What sample am I going to play?’ and ‘How many times am I going to divide it in half?’. Part 2 answered the question, ‘Once it’s chopped into little (or not-so-little) pieces, which one of them am I going to play?’
One thing I’ve talked about in both posts is balancing a desire for predictability in running the code and in allowing for random variation. This may seem strange for a studio album, but with my previous Christmas album, 12 Days of Crimbo, I did end up playing one of the pieces, Little Dubstep Boy live at an algorave in Birmingham, so it is possible some of these pieces may have a live afterlife.
Perhaps more importantly, allowing for variation allows the discovery of good juxtapositions, which can be made more likely in subsequent revisions of the code. It also makes making the pieces themselves more interesting. Listening to the same thing over and over again in a DAW can get tedious, but subtle changes give something to listen carefully for, keeping one’s ears fresh. I have way more skill in coding pieces than trying to piece them together manually and allowing randomness is an important part of my process.
What speed am I going to play the next bit at?
Having decided which sample to play, how small to chop it up and which subsection of the sample to play, the next question is what speed to play it back at. This is a question the rate at which I playback the buffer, such that both speed and pitch are effected.
A lot of the Vaporwave I’ve listened to seems to play back their samples at slower than the original speed. It also sometimes switches speeds, so that both pitch and timing jump. I like this effect and chose to copy it.
I thought of the timing changes in terms of scales. In Just Intonation, scale steps are given as fractions between the ratios 1/1 and 2/1, where 2/1 is one octave higher than 1/1. Because I wanted slower speeds, I adjust downwards, so that the ratios I used were either between 1/2 and 1/1 or between 1/4 and 1/2. These ratios can then be used directly as the rate parameter in a PlayBuf.ar ugen.
Obviously, almost all of the source material is in 12TET – that is to say: 12 tone equal temperament, the tuning one finds across the white and black keys of a standardly-tuned piano. Any tuning I apply on top of that is more or less in conflict with the original. This can be minimised by only using one rate at a time and repeating it across several subsequent bits. By sticking to one rate for a while, a change later on becomes roughly analogous to a key change. It would have been possible to have a set of sensible key changes that work with the standardised harmony conventions of the original song, but this is not what I did.
Long story short: I picked some scales and tunings
The first piece I worked on, Have Yourself a Scary Christmas is based on Have Yourself a Merry Little Christmas, sung by Judy Garland. That piece and especially that performance of the piece has quite a bit of emotional complexity. The ambiguity of the piece is apparent even in the title, which is not ‘Have a Merry Christmas’, but a more emotionally strangled ‘Have Yourself a Merry Little Christmas’, like someone trying unsuccessfully to be friendly to an ex they’re desperately not over. The music itself is ambiguous about whether it’s in a major key or in it’s relative minor. Garland’s performance adds to this further, soundling slightly drunken and sliding the tuning around in a way that sounds like she’s papering over crushing unhappiness. This effect is especially apparent if you listen to the song slowed down by a factor of 4 with Paulstretch.
In order to draw out the sadness a bit, I used a Romainian minor scale. This is part of the included scale library that comes with the SuperCollider Scale object. I then randomly picked ratios from the scale.
var scale, degree, rate;
Support for scales is built into Patterns, but it’s most commonly used to compute frequencies, not playback rates.
Almost all the pieces I made used minor scales. I used trial and error to pick which scale worked best with which samples. For all the pieces, I picked degrees randomly and never used anything like a Finite State Machine or a random walk, even those are usually well-suited to picking scale degrees.
In most pieces I repeated the chosen rate, using Pstutter. This is code from Santa Loves You:
The Pstutter on the scale degree repeats the degree 8 times before picking a new one with the Pfunc. The rate is calculated from the degree. Because this piece uses voices, I’ve decided to stay closer to a normal playback rate. Using 2/3 instead of 0.5 or smaller means that the samples will sometimes play out faster than their original pitch.
Previously, I talked about the method I used for creating the Christmaswave pieces and the issues this raises. My last post answered the questions ‘What sample am I going to play?’ and ‘How many times am I going to divide it in half?’ and, alas, the answer for the latter of those was hopelessly over complicated. Doing everything in terms of powers of two was not always a good approach, even for this project. Some songs are in 3/4 time and won’t do well with these divisions. Also, a lot of Christmas classics are performed in a jazz style and thus have a strong triplet feel. Play any Bing Crosby Christmas song at 1/2 speed and the swing becomes extremely apparent. It becomes ludicrously strongly swung at quarter speed. Well, get back to these issues a bit later on.
Once it’s chopped into little (or not-so-little) pieces, which one of them am I going to play?
Over the course of the album, I tried a few approaches. For several of the songs, I just picked randomly. This did work, but this means that some runs of the code come out with much better results than other runs of the code. Random number generators in The Santopticon (He Sees You) meant I had to try recording that piece 14 times before I found the version I wanted to use.
Another approach I used was to start at the beginning of the sample and move towards the end of it. This copies the arc of the original and makes the playback/recording more reliable, although it does remove some opportunities for serendipity.
\env, Pseg(Pseq([0, 1]), 200),
var frac, start, div;
div = evt[\div];
frac = evt[\env];
start = frac * evt[\div];
start = start.asInt;
"div is %, start is %, frac is %".format(div, start, frac.round(0.001)).postln;
}) + Pwhite(-1, 1),
\env has a value in it that goes from 0 to 1 over 200 seconds, which was the length of this section.
\start holds the number of the division to start on – for example, if there are 4 divisions, it might hold 0, 1, 2, or 3. It computes this by first getting the \div, computer previously in the event. Then it looks at \env, which, as it goes from 0 to 1, is a fractional representation of how far we are through the section. A quarter of the way through, it should be 0.25. At the half way point, it should be 0.5, etc. It puts this in the variable frac.
I multiplied frac by the number of divisions. Then, because this number will always be a fraction, I convert it to an integer, so as to get a good index number. This rounds it.
Finally, in order to make this slightly less predictable, I take the result of that calculation and subtract 1, do nothing or add 1.
In Lettuce, No, I picked a section adjacent to the one just played. This tends to meander around the middle of the sample, but the content of the neighbouring divisions are always related to each other, so the results tend to sound good without being too predictable.
var div, start, pos;
pos = 0; // starting position
div = 2.pow(evt[\pow]).asInt;
start = (pos*div).asInt;
// figure out pos for next time
pos = start + [-1, 1, 1].choose; // tend to increment
pos = (pos/div).abs.min(1);
evt = start.yield;
The Prout returns the start index, but also, while doing so, computes the position. This position is a fraction, like \env above. However, instead of moving from 0 to 1, it meanders around, tending to move towards 1.
It does this calculation based partially on the number of divisions. So if the sample is cut in half, it will tend to jump from 0 to 0.5 in a single step!
The pos must be between 0 and 1, so it takes the absolute value and then compares it to 1 and takes whichever of the two values is smaller.
It would have been possible to do this in a way less dependant on the divisions, for example, by deciding the position should be an integer between 0-127 and then computing a faction based on that. This would meander quite differently as it would be freed from the divisions actually used in the piece. I have no examples of this approach from this album, but will give it a try the next time I’m cutting samples!
Almost all of the pieces are constructed using variations on one algorithm. I found hoary, old baby boomer Christmas favourites and then took the instrumental sections, which was sometimes just the intro or the outro. All of the songs were in 4/4 and most of the instrumental parts where cut into either in 2 or 4 bar phrases. This means every sample is divisible by many powers of 2 and can be cut in half several times before it loses musical/rhythmic meaning.
I made these cuts, played the section of the sample with some stuttering and then went on to another section of the sample. This method requires some decision making:
Which sample am I going to play?
How many times am I going to divide it in half?
Once it’s chopped into little (or not-so-little) pieces, which one of them am I going to play?
What speed am I going to play that bit at?
How much should it overlap whatever comes after?
How long should I wait before going to the next thing (which might be a repetition of what I just did)?
How many times should I repeat this thing?
All of the pieces answered these questions in slightly different ways. (Or very different ways, in the case of question 1!) Some of the structuring of how I thought about these questions and how I solved them have to do with how the Pattern library works in SuperCollider.
What sample am I going to play?
In almost every case, I switched samples based on how much time had passed since the start of the piece. I used Ptpars to start different sections.
How many times am I going to divide it in half?
Another way of asking the question is, ‘what power of 2 am I going to use?’ I did this a few different ways. In most cases, I stuffed this into part of the event I called pow. Here are some ways I figured out what power of 2 to use:
\pow, Prand([0, 0, 0, 0, 0, 1, 1], inf)
Then later, on I could go from that to powers of two:
(Usually, I would compute the \div in a larger Pfunc that figures out more things.) The advantages of figuring out the power of 2 instead of just having a Prand full of 1, 2, 4, etc are that this is harder to screw up. I don’t need to worry about a stray 3 sneaking in, and, if a sample is longer, I can add some number to the \pow to make the \div bigger.
Another way I computed the \pow was using a Finite State Machine. This was completely overkill, but I’ll walk you through how it worked.
What I wanted was to have a possibility of a \pow being as small as 0 or as big as 8, but not to jump from one of those numbers to the other. Instead, I wanted a route going through intermediate numbers, in which it could potentially get to an 8 and have a path back to 0. I wanted a way for it to wander from one extreme to the other.
Pfsm is the pattern library that does state machines. It takes an array. The first item in the array is an array of what states it can start with. Next comes pairs. Each pair is a state. The first pair is state 0. The second pair is state 1. The third pair is state 2, etc. The first item in a pair is the output. In the example above, the output of state 0 is 2 and the output of state 1 is Prand([0, 0, 1], 1).
The second item in the pair is an array of one or more integers. The numbers in the array are the states you can go to next. So with state 0, the array is ‘#’, so it goes on to state 3. When it gets to state 3, it produces the output, which is Prand([1, 2]) and then looks where it can go next, which is ‘#[3, 4]’. That is, it can go state 3 again, or it can go on to state 4.
I could draw a map of this (which would reveal that there is no path to states 1 and 2 – oops). In the graph, you can see a lot of arrows pointing up and few pointing down, only along a path that goes through all the states. Thus, it’s relatively unlikely to reach state 7.
Because Pfsm is just another pattern, I could add 1 or 2 to the output of it in the case of a particularly long sample and it would gracefully handle the maths.
Answers to the other questions will be forthcoming in following posts!
Testing showed that for human voice, the frequency domain onsets and pitch tracking were more accurate and faster than the time domain, which is good to know.
Once the frequency is detected, it needs to be mapped to a scale degree. I’ve added this functionality to the Tuning Lib quark. While doing this, I could the help file was confusing and badly laid out and some of the names of flags on the quantisations were not helpful, so I fixed the helpfile, documented the new method, renamed some of the flags (the old ones still work). And then I found it wasn’t handling octaves correctly – it assumed the octave ratio is always 2, which is not true for Bohlen Pierce scales, or some scales derived by Dissonance Curve. So this was good because that bug is not fixed after a mere 8 years of lurking there. HOWEVER, the more I think about it, the less I think this belongs in Key….
Pitch detecting is flaky as hell, but onsets are solid, which is going to make the creation of melodic loops difficult, unless they actually just record the tuba and do stuff with it.
Entering code requires the ability to determine pitch and entering data requires both pitch and onset. Ergo, we need a synthdef to listen for both things. There is also two ways to determine pitch, one in the time domain and the other in the frequency domain.
The frequency domain, of course, refers to FFT and is probably the best method for instruments like flute. It has a pure tone, where the loudest one is the fundamental. However, brass instruments and the human voice both have formants (loud overtones). In the case of tuba, in low notes, the overtones can be louder than the main pitch. I’ve described time-domain frequency tracking for brass and voice in an old post.
The following is completely untested sample code…. It’s my wife’s birthday and I had to go out before I could try it. It does both time and frequency domain tracking, using the fft code to trigger sending the pitch in both cases. For time domain tracking, it could -and possibly should- use the amplitude follower as a gate/trigger in combination with a frequency change of greater than some threshold. The onset cannot be used as the trigger, as the pitch doesn’t stabilise for some time after the note begins. A good player will get it within two periods, which is still rather a long time in such a low instrument. A less good player will take longer to stabilise on a pitch.
Everything in the code is default values, aside from the RMS window, so some tweaking is probably required. Presumably, every performer of this language would need to make some changes to reflect their instrument and playing technique.