resona is a small, hackable toolkit for self-supervised audio representation learning. It gives you a pure-PyTorch log-mel frontend, a spectrogram transformer encoder, and three pretraining objectives that share the same backbone:
- Masked-spectrogram modelling (
MaskedSpectrogramModel) — MAE-style patch masking and reconstruction. - Contrastive (
ContrastiveModel) — SimCLR/NT-Xent over two augmented views. - BYOL (
BYOLModel) — non-contrastive learning with an EMA target network.
Everything runs on CPU out of the box with a built-in synthetic dataset, so you can smoke-test the full pipeline before pointing it at real audio.
Most reference implementations are tied to a specific paper, a specific dataset layout and a heavyweight config system. resona pulls the reusable pieces — the frontend, the patch/mask machinery, the encoder and the loss functions — into a clean, typed, well-tested library you can drop into your own training code.
The mel frontend is built directly on torch.stft plus a hand-rolled Slaney/HTK
filterbank, so there is no dependency on torchaudio or librosa for the
core path and the whole frontend stays differentiable.
pip install resona # core (torch + numpy)
pip install "resona[audio]" # + soundfile/librosa for reading audio files
pip install "resona[yaml]" # + pyyaml for config filesFrom source:
git clone https://github.com/jamesa94/resona
cd resona
pip install -e ".[dev]"Pretrain a masked-spectrogram model on synthetic audio:
resona pretrain --mode mae --epochs 3 --out mae.ptOr in Python:
import torch
from resona import LogMelSpectrogram, MaskedSpectrogramModel
frontend = LogMelSpectrogram(n_mels=64)
model = MaskedSpectrogramModel(spec_shape=(64, 96), mask_ratio=0.75)
wave = torch.randn(8, 15_200) # (batch, samples)
spec = frontend(wave).unsqueeze(1) # (batch, 1, mels, frames)
out = model(spec)
out.loss.backward()Extract embeddings after pretraining:
from resona import AudioTransformer, Embedder
encoder = AudioTransformer(spec_shape=(64, 96))
embedder = Embedder(encoder, frontend)
emb = embedder(torch.randn(4, 15_200)) # (4, 384)See examples/ for runnable scripts covering all three objectives
and a linear-probe evaluation.
pip install -e ".[dev]"
ruff check .
mypy
pytest --cov=resonaMIT — see LICENSE.