That | SuperCollider 3.13.0 Help

Description

This system is called "that", as in "you know that".

This is a condensed version of the real time sound analysis we did in the Musikinformatik seminar May/June 2021 @ RSH Düsseldorf which was rewritten into a Quark.

That allows to use values of UGens (which are running on the server) in the sclang domain which allows for interaction with live material by using analysis UGens on the signal. This allows the creation of autonomous systems such as Voyager by George Lewis, but is not limited to that.

Technical

SuperCollider is designed in such a way that sclang sends OSC messages to the server. This way of communication is well established, e.g. when adding a SynthDef to the server it gets transferred from sclang to the server via OSC as the server is only capable of generating sound. For more information on this please refer to OSC Communication.

But communicating values from the server (so a value of a UGen) to sclang has always been a bit tricky but possible via SendReply. That is a framework which helps reducing the cognitive and maintenance overhead of the server->language communication.

That relies on an input signal on which an analyzer can be applied upon. The result of the analyzer (which is an Event) gets calculated on the server and send back to sclang via OSC. The analyzer runs as a Ndef on the server (with a separated namespace by prepending that_ on its name) and the receiving OSCdef function receives those values and stores its value to the That instance and also allows for a callback everytime a new value is received.

Hello world examlpe

Before we start discussing each fragment of That we will demonstrate it.// create a simple sound via an Ndef which we want to analyze ( Ndef(\thatSound, { SinOscFB.ar( freq: LinExp.kr( in: LFNoise2.kr([1.5, 0.7]), srclo: -1.0, dstlo: 200.0, dsthi: 1000.0, ), feedback: LFNoise0.kr([1.5, 2.0]) )*EnvGen.kr( envelope: Env.perc, gate: Impulse.kr(LFNoise1.kr([1.2, 1.0], add: 0.7)) )*0.5; }).play; ) // we can use contsructors of That which analyse of specific // parameters, e.g. the amplitude ( That.amp( name: \ampTest, input: Ndef(\thatSound), callback: {|result| result.postln;}, triggerFunction: {|in, defaultTrigger| Impulse.kr(5)}, ); ) // the callback gets called 5 times a secound and prints the amp of both channels now // change the callback to calculate the difference between both channels That(\ampTest).callback = {|r| (r[0] - r[1]).postln} // clear it like a Ndef That(\ampTest).clear; // now analyze the played frequencies // note that we have not defined a callback ( That.freq( name: \freqTest, input: Ndef(\thatSound), ) ) // but we can still access the values via That(\freqTest).latestValue // stopping the analysis is as easy as That(\freqTest).clear; // We are also not bound to using Ndefs as input // and can easily make our trigger depened on the input signal ( That.amp(\ampTest, input: {SinOsc.ar(0.1)}, callback: {|r| r.postln}, // only trigger if SinOsc is above 0.6 and rate limit it by 10 Hz triggerFunction: {|in| (in.abs > 0.6)*Impulse.kr(10.0) }, ) ) That(\ampTest).clear; // and we are not limited to using the predefined constructors // as we can use the .new constructor to create an analyzer from scratch. // Notice that in this constructor we do not have a triggerFunction // as the analyzerFunction takes care of triggering the callback ( That(\thatAnalyzer, input: Ndef(\thatSound), analyzerFunction: {|in| ( \amp: Amplitude.kr(in), \impulseClock: Pulse.kr(1.1), // the trigger defines when // the callback is called \trigger: Impulse.kr(5.0), ) }, callback: {|r| r.postln;}, ) ) That(\thatAnalyzer).clear; // thats enough for a hello world Ndef(\thatSound).clear;

Class Methods

.all

an Event in which all running that instances are registered

.new

Creates a new That analyzer from scratch. Take a look at the constructors which use an existing analyzerFunction or at the examples.

Arguments:

name	Unique Symbol which is also used to internally set up the Ndef for the analyzer and the OSCdef for communication.
input	Either a SynthDef, a Function or a Ndef which is the signal one wants to perform analysis on.
analyzerFunction	Function which analyzes the signal and determines the times when the callback is called. The first parameter of the function is the providid input signal. The function must return an Event with at least the key trigger with a UGen as value which defines the times when the callback function gets called .For specifics check out the examples.
callback	A function which gets called when the analyzerFunction returns a new value. The first parameter of the d function is the result of the analyzer. Iff the analyzerFunction returns one value it is an Array of values with the respecting number of input channels, otherwhise the argument will be an Event with key-value pairs of name of the parameter and its channel array.

Returns:

The created That instance

.amp

A constructor which uses Amplitude to analyse the amplitude of each channel of the signal. See new for more information on the parameters.

Arguments:

name
input
callback
triggerFunction	A function which allows to specify the triggering of messages. The first parameter is the input signal and the secound parameter is the default trigger of the analyzerFunction. Take a look at the examples or the source code for more information.

.freq

A constructor which uses Tartini to analyse the pitch of each channel of the signal. See new and amp for more information on the parameters.

Arguments:

name
input
callback
triggerFunction

.freqTime

A constructor which uses Tartini to analyse the pitch and Onsets to analyse the onsets on each channel of the signal. See new and amp for more information on the parameters.

Arguments:

name
input
callback
triggerFunction

.identity

A constructor which allows to access the raw values of the input signal. See new and amp for more information on the parameters.( That.identity( name: \thatIdentity, input: {SinOsc.ar(0.2)}, callback: {|r| r.postln;} ) ) That(\thatIdentity).clear;

Arguments:

name
input
callback
triggerFunction

.mfcc

A constructor which uses MFCC to analyse the mel spectogram of the mono signal. See new and amp for more information on the parameters.

Arguments:

name
input
callback
triggerFunction
fftSize

Instance Methods

.analyzerFunction

Allows to replace the analyzerFunction while running. Check source code for further information on how such an analyzerFunction needs to look like.

Arguments:

newAnalyzerFunction

.analyzer

Allows access to the Ndef which runs the analyzerFunction which can be used to adjust the parameters of the analyzer, e.g.Ndef(\thatSound, {SinOsc.ar*(SinOsc.ar(0.5).pow(8))*0.2}).play; That.amp(\ampTest, Ndef(\thatSound), {|a| a.postln}); That(\ampTest).analyzer.gui; // clear up That(\ampTest).clear; Ndef(\thatSound).clear;

.clear

Clears the internally created OSCdef and Ndef and therefore also stops evaluating the input signal.

.latestValue

Allows to access the latest value send from the analyzer independent of a callback.

.name

Name of That.

.input

Input used to analyse the material, possible to change while running.( Ndef(\thatSound, {SinOsc.ar*0.2}); Ndef(\thatOtherSound, {Saw.ar*0.1}); That.amp(\thatTest, Ndef(\thatSound), {|r| r.postln}, {Impulse.kr(1.0)}); ) // now change the input That(\thatTest).input = Ndef(\thatOtherSound); // clear up ( That(\thatTest).clear; Ndef(\thatSound).clear; Ndef(\thatOtherSound).clear; )

Arguments:

newInput

.callback

Allows to access and replace the callback function.

Examples

Amplitude analysis

// start by creating a sound

(
Ndef(\thatSound, {
    SinOscFB.ar(
        freq: LinExp.kr(
            in: LFNoise2.kr([1.5, 0.7]),
            srclo: -1.0,
            dstlo: 200.0,
            dsthi: 1000.0,
        ),
        feedback: LFNoise0.kr(1.5)
    )*EnvGen.kr(Env.perc, gate: Impulse.kr([1.2, 1.0]))*0.2;
}).play;
)

// we will use amplitude analysis as a simple example to explain everything
// further down are examples for more complicated analysis forms.
(
That.amp(
    name: \ampTest,
    input: Ndef(\thatSound),
    callback: {|result| result.postln},
    triggerFunction: {Impulse.kr(10)}
);
)

// the analyzerFunction runs in a Ndef which is
// accessible by .analyzer
That(\ampTest).analyzer
// we can pause the execution of the analyzer
// by pausing this ndef
That(\ampTest).analyzer.pause
// and can also wake it up again
That(\ampTest).analyzer.resume;

// we can change the triggerFunction while it is running
// because the triggerFunction is attached to the analyzer
// we must use the constructor ".amp"
// which will build us a new analyzerFunction which respects
// the new trigger

// we will explicetely state the input as this allows for non-linear
// working through the examples
(
That.amp(
    name: \ampTest,
    input: Ndef(\thatSound), // optional
    triggerFunction: {Impulse.kr(2)}
);
)

// We can also use the default trigger
// by explicetely setting the default trigger
(
That.amp(
    name: \ampTest,
    input: Ndef(\thatSound), // optional
    triggerFunction: {|in, defaultTrigger| defaultTrigger},
);
)
// now we can tweak the parameters of this analyzer
// via the Ndef gui
That(\ampTest).analyzer.gui;

// We can also create a trigger which takes
// account of the original trigger and custom
// behaviour
(
That.amp(
    name: \ampTest,
    input: Ndef(\thatSound), // optional
    triggerFunction: {|in, defaultTrigger| defaultTrigger * Impulse.kr(5)},
);
)
// This now only updates if our defaultTrigger is active
// and is rate to 5 Hz limited by multiplying
// an impulse to it which acts as a boolean AND

// We can also create a logical or
// but for this we want another signal

Ndef(\thatSound, {SinOscFB.ar(LinExp.kr(LFNoise0.kr([1.0, 3.4]), -1.0, 1.0, 100, 400), feedback: LFNoise0.kr([4, 7]).abs)*EnvGen.kr(Env.perc, gate: Impulse.kr([0.5, 0.15]))*0.2}).play;
// note that our analyzer immediately picks up the
// changed signal

// now lets use an OR, at least an update at 5Hz
(
That.amp(
    name: \ampTest,
    input: Ndef(\thatSound), // optional
    triggerFunction: {|in, defaultTrigger| (defaultTrigger + Impulse.kr(5))}
);
)

// after we have finished our analysis we can
// clear our that instance just like a Ndef
That(\ampTest).clear;
Ndef(\thatSound).clear;

Custom analyzerFunction

Although there are a variety of analyzerFunctions provided via the constructors it is possible to write such a custom function.( f = {|in| // in is the input of the attached That instance ( \amp: Amplitude.kr(in), \freq: Tartini.kr(in), // it is important to provide a trigger key which triggers // the sending of a OSC message from \trigger: Impulse.kr(1.0), ) }; That( name: \thatCustom, input: Ndef(\thatSound, {SinOsc.ar(LinExp.kr(LFNoise0.kr(1.0), -1.0, 1.0, 100, 200))}).play, analyzerFunction: f, callback: {|r| r.postln;} ); ) // clear That(\thatCustom).clear; Ndef(\thatSound).clear;

Feedback

Markov Chain

(
// create an event in which we store our markov functions
q = q?();
// some parameters to tweak
q[\timeStep] = 0.05;
q[\timeMax] = 2.0;
q[\noteRange] = (0..88);

// init some private varibales
q[\lastMidiNote] = 0;
q[\currentTime] = 0;
q[\lastTime] = 0;
q[\lowInitValue] = 1e-5;

// helper function
q[\maxTimeStep] = (q[\timeMax]/q[\timeStep]).asInteger;
q[\timeMapper] = {|t| ((t/q[\timeStep]).roundUp).min(q[\maxTimeStep]-1)};
q[\timeMapperReverse] = {|s| s*q[\timeStep]};

// create 88x88 transition matrix for midi notes
// which is diagonal so we will create a chromatic glissando
// as an init distribution
q[\markovNote] = (q[\noteRange].size-1).collect({|i|
    (q[\noteRange].size-1).collect({|j|
        if((i+1)==j, {q[\lowInitValue]}, {0});
    })
});
// add 87->0
q[\markovNote][q[\noteRange].last-1][q[\noteRange].first] = q[\lowInitValue];

// create maxTimeStep x maxTimeStep transition matrix for
//time events which init is random
q[\markovTime] = (q[\maxTimeStep]).collect({
    (q[\maxTimeStep]).collect({
        // same init as notes
        q[\lowInitValue];
    })
});
)

(
// create some input signal - play is optional
Ndef(\guitar, {
    SoundIn.ar(0)
}).play;
)

(
// we use the results of our analyzers to modify the markov
// transition matrix to reflect our playing
That.freqTime(\thatGuitar,
    input: Ndef(\guitar),
    callback: {|r|
        var midiNote = r[\freq][0].cpsmidi.round;
        q[\currentTime] = q[\currentTime] + r[\timePassed][0];

if(q[\lastMidiNote] != midiNote, {
            var time = q[\timeMapper].(q[\currentTime]);
            // increment markov matrix counter
            q[\markovNote][q[\lastMidiNote]][midiNote] = q[\markovNote][q[\lastMidiNote]][midiNote] + 1;
            q[\markovTime][q[\lastTime]][time] = q[\markovTime][q[\lastTime]][time] + 1;

// re-adjust state
            q[\lastMidiNote] = midiNote;
            q[\lastTime] = time;
            q[\currentTime] = 0;

["humanPlayer", time, midiNote].postln; // debug
        });
    },
);
)

(
// some synth to be played
SynthDef(\sinePerc, {|out|
    var sig = SinOscFB.ar(\freq.kr(200), \feedback.kr(1.0))*\amp.kr(0.5);
    var env = EnvGen.kr(Env.perc, doneAction: Done.freeSelf);
    sig = Pan2.ar(sig*env, 1.0); // map the output to the right channel
    Out.ar(out, sig);
}).add;

// play the synth according to the markov transition matrix
Tdef(\markovPlayer, {
    var curNote = q[\noteRange].choose;
    var curTime = 0;
    inf.do({
        // choose next event based on distribution of the markov transition matrix
        curNote = q[\noteRange].wchoose(q[\markovNote][curNote].normalizeSum);
        curTime = (0..q[\maxTimeStep]).wchoose(q[\markovTime][curTime].normalizeSum);
        (
            \instrument: \sinePerc,
            \freq: curNote.midicps,
        ).play;
        ["markovPlayer", curTime, curNote].postln; // debug
        q[\timeMapperReverse].(curTime).wait;
    });
}).play;
)

// stop and clean up
(
Tdef(\markovPlayer).clear;
Tdef(\guitar).clear;
That(\thatGuitar).clear;
q = nil;
)

helpfile source: HelpSource/Classes/That.schelp
link::that/Classes/That::

ThatExtension

.all

.new

Arguments:

Returns:

.amp

Arguments:

.freq

Arguments:

.freqTime

Arguments:

.identity

Arguments:

.mfcc

Arguments:

.analyzerFunction

Arguments:

.analyzer

.clear

.latestValue

.name

.input

Arguments:

.callback

That
Extension