FluCoMa/Guides (extension)
| Libraries > FluidCorpusManipulation
Fluid Corpus Manipulation Toolkit
Extension
ExtensionThe FluCoMa toolkit to analyse, transform and learn from sounds
Description
The Fluid Corpus Manipulation toolkit provides an open-ended, loosely coupled set of objects for exploring and music making with collections of sound. It includes tools for slicing, analysing, decomposition and transforming audio, as well as a suite of objects for exploring and organisng using machine learning.
Almost all objects for audio analysis or transformation have audio-rate and buffer-based versions, and there are custom server-side containers for data analysis.1
Many useful examples can be found in the help files as well as in the example folder, which is here:
Contents
Slice Audio
| on signals | on buffers | digest |
| FluidAmpGate | FluidBufAmpGate | Events from amplitude enevelope |
| FluidAmpSlice | FluidBufAmpSlice | Onsets from amplitude envelope |
| FluidOnsetSlice | FluidBufOnsetSlice | Spectral onset detector |
| FluidTransientSlice | FluidBufTransientSlice | Transient model onset detector |
| FluidNoveltySlice | FluidBufNoveltySlice | Novelty based onset detection on a choice of descriptors |
Analyse Audio
| on signals | on buffers | digest |
| FluidPitch | FluidBufPitch | Choice of pitch descriptors |
| FluidLoudness | FluidBufLoudness | Loudness Descriptor |
| FluidMelBands | FluidBufMelBands | Energy in Mel Bands |
| FluidMFCC | FluidBufMFCC | Timbral Descriptor with Mel Frequency Cepstral Coefficients |
| FluidSpectralShape | FluidBufSpectralShape | Seven Spectral Shape Descriptors |
| FluidChroma | FluidBufChroma | Pitch Classes Descriptor |
| FluidNMFMatch | Real-time activation of NMF bases | |
| FluidBufNMFSeed | Quick starting estimates for NMF components using Singular Value Decomposition | |
| FluidBufSTFT | Perform STFT / ISTFT on Buffers | |
| FluidAmpFeature | FluidBufAmpFeature | Detrending Amplitude Envelope Descriptor |
| FluidNoveltyFeature | FluidBufNoveltyFeature | Novelty descriptor based on a choice of analysis descriptors |
| FluidOnsetFeature | FluidBufOnsetFeature | Descriptor comparing spectral frames using a choice of comparisons |
| FluidSineFeature | FluidBufSineFeature | Sinusoidal peak extraction |
Decompose Audio
| on signals | on buffers | digest |
| FluidSines | FluidBufSines | Decompose into sines + residual |
| FluidTransients | FluidBufTransients | Decompose into transients + residual |
| FluidHPSS | FluidBufHPSS | Decompose into 'harmonic' and 'percussive' layers |
| FluidBufNMF | Use Nonnegative Matrix Factorisation to explore and decompose sounds |
Transform Audio
| on signals | on buffers | digest |
| FluidAudioTransport | FluidBufAudioTransport | Interpolate between sounds using Optimal Transport |
| FluidNMFFilter | Filter sound using NMF bases | |
| FluidNMFMorph | Morph between sounds using NMF components | |
| FluidBufNMFCross | Cross synthesise buffers using NMF components |
Analyse Data
Containers
| FluidDataSet | Container that associates data points with identifiers |
| FluidLabelSet | Container of labels associated with IDs |
Analyse Data
| FluidStats | FluidBufStats | Compute statistics |
Preprocessing
| FluidNormalize | Normalize FluidDataSets and Buffers |
| FluidStandardize | Standardize FluidDataSets and Buffers |
| FluidRobustScale | Scale FluidDataSets and Buffers using order statistics |
Searching and Querying
| FluidKDTree | Nearest Neighbour queries on FluidDataSet |
| FluidDataSetQuery | Construct custom queries on FluidDataSet |
Supervised Machine Learning
| FluidKNNRegressor | Regression by Nearest Neighbour modelling |
| FluidKNNClassifier | Classification by Nearest Neighbour modelling |
| FluidMLPRegressor | Regression using Multilayer Perceptron model |
| FluidMLPClassifier | Classification using Multilayer Perceptron model |
Unsupervised Machine Learning
| FluidPCA | Principal Component Analysis for preprocessing and dimension reduction |
| FluidMDS | Multidimensional Scaling for dimension reduction |
| FluidKMeans | K-Means clustering |
| FluidSKMeans | Spherical K-Means clustering |
| FluidUMAP | Dimension reduction with UMAP algorithm |
| FluidGrid | Transform a data set of two dimensional points into a two dimensional grid using the Munkres Algorithm. |
Helpers
Buffer Utilities
| FluidBufCompose | Copy, slice, stack, mix concatenate. All the things you've wanted to do with buffers... |
| FluidBufScale | Remap range of values (like linlin etc.) |
| FluidBufThresh | Zero elements below threshold |
| FluidBufSelect | Select range (frame-wise or channel wise) |
| FluidBufSelectEvery | Select every N elements (frame-wise or channel wise) |
| FluidBufFlatten | Flatten multichannel data into single channel 'point' |
| FluidBufToKr | Read data from a buffer into a Kr stream |
| FluidKrToBuf | Write data into a buffer from a Kr Stream |
Viewers
| FluidPlotter | View a FluidDataSet in a plotter window |
| FluidWaveform | View an audio buffer with overlays, such as slices from a FluCoMa slicer |
Corpus Building
| FluidLoadFolder | Load a folder of sounds into a Buffer |
| FluidSliceCorpus | Batch-slice a corpus |
| FluidProcessSlices | Batch-analyse slices |
[1] - This toolbox was made possible thanks to the FluCoMa project funded by the European Research Council ( https://erc.europa.eu/ ) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 725899).
helpfile source: release-packaging/HelpSource/Guides/FluidCorpusManipulation.schelp
link::FluCoMa/Guides/FluidCorpusManipulation::
link::FluCoMa/Guides/FluidCorpusManipulation::