Tracking Session is an installation that took place between May 10th - 14th at CalArts in the WaveCave.
Through out the week, four large language abliterated models represented different musicians to create an album, with a listening party happening at the end.
I extended the LLM conversation engine, seen here, that I created to allow the four models to easily communicate locally on a PC. The extension included using ACE-Step v1.5 , to create the tracks that were laid out by the LLMs.
In the installation space, I connected my PC to a Mac Studio to run a TouchDesigner patch to show the conversation, with the words via OSC. This allowed people who were walking in and out see what the musicians were talking about without having to sit in the space to hear the text-to-speech to be processed.
At night, when there would be minimal traffic going through the space, we would take the document created with all the parameters the musicians determined through out the day and run that through ACE-Step to get the song. We got 3 versions of each track, a total of 6 tracks each night, in case there was a failure during one of the generations. During the generation at night, we would play random music tracks that I have written or enjoyed in general.
You can see the final generations, conversation logs, song states(which show the parameters needed to generate the song), and EP-State here.
The Github repo has a README that goes deeper into the technical details of the installation, which can be found here
Quick Note: This all ran locally on a PC and Mac Studio, a primary goal of mine to have a more ethical installation and goes towards my current research.
This is a quick video of what the space looked like,
For the generated audio, it needed a description of the genre it should represent. For me, I enjoy metal/metalcore as I was growing up so this was the prompt given ACE-Step during generation:
"90's grunge, 2010's deathcore. in the vein of Nirvana, Alice and Chains, early TOOL (Undertow era), chelsea grin, and Born of Osiris "
"Heavily distorted electric guitar: high-gain tube saturation, buzzsaw tone, palm-muted power chord chugs, abrasive. "
"Overdriven gritty bass, punchy low-end. "
"Live d-beat drum kit, along with double kicks, loud and driving, crushing half-time breakdown. "
"Male screamed vocals, raw and confrontational. "
"Analog recording, live room sound, natural noise floor. "
It also allows for negative prompts on what to avoid which can be found here:
"clean guitar, synth, synthesizer,
keyboard, piano, strings, pads, atmospheric, silence"
Sometimes it followed it, sometimes not so much. Mostly with guitar tone.
A small collection of the music generated can be found below, with a comparison on how the same prompt can sound different as well