Practically Universal Music Pre-Processor¶
Pumpp is designed to make it easy to convert pairs of (audio, jams) into data that can be easily consumed by statistical algorithms. Some desired features:
- Converting tags to sparse encoding vectors
- Sampling (start, end, label) to frame-level annotations at a specific sampling rate
- Extracting first-level features (eg, Mel spectra or CQT) from audio
- Aligning and storing the results in a simple data structure (npz, hdf5)
- Converting between annotation spaces for a given task
- Helper variables for semi-supervised learning
API¶
Core functionality¶
Pump (*ops) |
Top-level pump object. |
transform (audio_f, jam, *ops) |
Apply a set of operations to a track |
-
class
pumpp.core.
Pump
(*ops)[source] Top-level pump object.
This class is used to collect feature and task transformers
See also
Examples
Create a CQT and chord transformer
>>> p_cqt = pumpp.feature.CQT('cqt', sr=44100, hop_length=1024) >>> p_chord = pumpp.task.ChordTagTransformer(sr=44100, hop_length=1024) >>> pump = pumpp.Pump(p_cqt, p_chord) >>> data = pump.transform('/my/audio/file.mp3', '/my/jams/annotation.jams')
Access all the fields produced by this pump:
>>> pump.fields {'chord/chord': Tensor(shape=(None, 170), dtype=<class 'bool'>), 'cqt/mag': Tensor(shape=(None, 288), dtype=<class 'numpy.float32'>), 'cqt/phase': Tensor(shape=(None, 288), dtype=<class 'numpy.float32'>)}
Access a constituent operator by name:
>>> pump['chord'].fields {'chord/chord': Tensor(shape=(None, 170), dtype=<class 'bool'>)}
Attributes
ops (list of (BaseTaskTransformer, FeatureExtractor)) The operations to apply -
add
(op)[source] Add an operation to this pump.
Parameters: op : BaseTaskTransformer, FeatureExtractor
The operation to add
Raises: ParameterError
if op is not of a correct type
-
layers
()[source] Construct Keras input layers for all feature transformers in the pump.
Returns: layers : {field: keras.layers.Input}
A dictionary of keras input layers, keyed by the corresponding fields.
-
sampler
(n_samples, duration)[source] Construct a sampler object for this pump’s operators.
Parameters: n_samples : None or int > 0
The number of samples to generate
duration : int > 0
The duration (in frames) of each sample patch
Returns: sampler : pumpp.Sampler
The sampler object
See also
-
transform
(audio_f, jam=None)[source] Apply the transformations to an audio file, and optionally JAMS object.
Parameters: audio_f : str
Path to audio file
jam : optional, jams.JAMS, str or file-like
Optional JAMS object/path to JAMS file/open file descriptor.
If provided, this will provide data for task transformers.
Returns: data : dict
Data dictionary containing the transformed audio (and annotations)
-
-
pumpp.core.
transform
(audio_f, jam, *ops)[source] Apply a set of operations to a track
Parameters: audio_f : str
The path to the audio file
jam : str, jams.JAMS, or file-like
A JAMS object, or path to a JAMS file.
If not provided, an empty jams object will be created.
ops : list of task.BaseTaskTransform or feature.FeatureExtractor
The operators to apply to the input data
Returns: data : dict
Extracted features and annotation encodings
Feature extractors¶
FeatureExtractor (name, sr, hop_length[, conv]) |
The base feature extractor class. |
CQT (name, sr, hop_length[, n_octaves, ...]) |
Constant-Q transform |
CQTMag (*args, **kwargs) |
Magnitude CQT |
CQTPhaseDiff (*args, **kwargs) |
CQT with unwrapped phase differentials |
STFT (name, sr, hop_length, n_fft[, log, conv]) |
Short-time Fourier Transform (STFT) with both magnitude and phase. |
STFTMag (*args, **kwargs) |
STFT with only magnitude. |
STFTPhaseDiff (*args, **kwargs) |
STFT with phase differentials |
Mel (name, sr, hop_length, n_fft, n_mels[, ...]) |
Mel spectra feature extraction |
Tempogram (name, sr, hop_length, win_length) |
Tempogram: the short-time autocorrelation of the accent signal |
TempoScale (name, sr, hop_length, win_length) |
Tempogram scale transform. |
Task transformations¶
BaseTaskTransformer (name, namespace, sr, ...) |
Base class for task transformer objects |
BeatTransformer ([name, sr, hop_length]) |
Task transformation for beat tracking |
ChordTransformer ([name, sr, hop_length]) |
Chord annotation transformers. |
SimpleChordTransformer ([name, sr, hop_length]) |
Simplified chord transformations. |
ChordTagTransformer ([name, vocab, sr, ...]) |
Chord transformer that uses a tag-space encoding for chord labels. |
VectorTransformer (name, namespace, dimension) |
Vector regression transformer. |
DynamicLabelTransformer (name, namespace[, ...]) |
Time-series label transformer. |
StaticLabelTransformer (name, namespace[, labels]) |
Static label transformer. |
Data subsampling¶
Sampler (n_samples, duration, *ops) |
Generate samples from a pumpp data dict. |
-
class
pumpp.sampler.
Sampler
(n_samples, duration, *ops)[source] Generate samples from a pumpp data dict.
Examples
>>> # Set up the parameters >>> sr, n_fft, hop_length = 22050, 512, 2048 >>> # Instantiate some transformers >>> p_stft = pumpp.feature.STFTMag('stft', sr=sr, n_fft=n_fft, ... hop_length=hop_length) >>> p_beat = pumpp.task.BeatTransformer('beat', sr=sr, ... hop_length=hop_length) >>> # Apply the transformers to the data >>> data = pumpp.transform('test.ogg', 'test.jams', p_stft, p_beat) >>> # We'll sample 10 patches of duration = 32 frames >>> stream = pumpp.Sampler(10, 32, p_stft, p_beat) >>> # Apply the streamer to the data dict >>> for example in stream(data): ... process(data)
Attributes
n_samples (int or None) the number of samples to generate. If None, generate indefinitely. duration (int > 0) the duration (in frames) of each sample ops (one or more pumpp.feature.FeatureExtractor or pumpp.task.BaseTaskTransformer) The operators to include when sampling data. -
__call__
(data)[source] Generate samples from a data dict.
Parameters: data : dict
As produced by pumpp.transform
Yields: data_sample : dict
A sequence of patch samples from data, as parameterized by the sampler object.
-
__weakref__
list of weak references to the object (if defined)
-
data_duration
(data)[source] Compute the valid data duration of a dict
Parameters: data : dict
As produced by pumpp.transform
Returns: length : int
The minimum temporal extent of a dynamic observation in data
-
sample
(data, interval)[source] Sample a patch from the data object
Parameters: data : dict
A data dict as produced by pumpp.transform
interval : slice
The time interval to sample
Returns: data_slice : dict
data restricted to interval.
-