Current Projects
← Probing the World for Groove

✳ Experiments

34 experiments, 11 rounds, 4 questions

The work progressed from a narrow proof of concept to the full 74 class benchmark. Read the curated progression below, then explore, filter, and compare every run.

01 · The progression

How the study unfolded

  1. 1

    Initial configuration

    Configuration

    Do a baseline CNN and PaSST work on a narrow rock style subset?

    CNN 1.1 0.9204 PaSST 1.2 0.8544
  2. 2

    PaSST configuration

    Configuration

    How should PaSST be configured before scaling up?

    PaSST 2.1 0.8660 PaSST 2.2 0.8699
  3. 3

    Low data comparison

    Dataset

    Which model wins when training data is scarce (GMD-mini)?

    CNN 3.1 0.3267 PaSST 3.2 0.3911
  4. 4

    Full dataset comparison

    Dataset

    How do the models compare on the full 74 class task?

    CNN 4.1 0.7531 PaSST 4.2 0.7367
  5. 5

    Repeatability

    Dataset

    Are the full data results stable across repeated runs?

    PaSST 5.1 0.7269 PaSST 5.2 0.7313 CNN 5.3 0.7646 CNN 5.4 0.7827
  6. 6

    t-SNE & time patchout

    Model depth & patchout

    Does time patchout help, and how are embeddings structured?

    PaSST 6.1 0.7247 PaSST 6.2 0.3861
  7. 7

    PaSST head depth I

    Model depth & patchout

    Does a deeper MLP head help PaSST?

    PaSST 7.1 0.8582
  8. 8

    PaSST head depth & bottlenecks

    Model depth & patchout

    What head depth and bottleneck shape is best for PaSST?

    PaSST 8.1 0.8604 PaSST 8.2 0.7964 PaSST 8.3 0.8659 PaSST 8.4 0.8352 PaSST 8.5 0.8424 PaSST 8.6 0.8577 PaSST 8.7 0.8604
  9. 9

    CNN convolutional depth

    Model depth & patchout

    How many convolutional layers does the CNN need?

    CNN 9.1 0.8779 CNN 9.2 0.8944 CNN 9.3 0.8900
  10. 10

    Augmentation

    Augmentation & padding

    Do audio augmentations improve either model?

    CNN 10.1 0.9080 PaSST 10.2 0.7953 PaSST 10.3 0.8446 CNN 10.4 0.8632
  11. 11

    Padding

    Augmentation & padding

    Does the padding mode affect performance and representation?

    PaSST 11.1 0.8429 PaSST 11.2 0.8752 CNN 11.3 0.8747 CNN 11.4 0.8736 CNN 11.5 0.8747

02 · Explore

Every run, filterable

Model
Category
Dataset scope
Padding
Augmentation
GPU

0 0.25 0.5 0.75 1 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 exp 1.1 · CNN · F1 0.9204: Defaults with rock styles exp 1.2 · PaSST · F1 0.8544: Defaults with rock styles exp 2.1 · PaSST · F1 0.8660: PaSST config exp 2.2 · PaSST · F1 0.8699: PaSST config exp 3.1 · CNN · F1 0.3267: GMD-mini exp 3.2 · PaSST · F1 0.3911: GMD-mini exp 4.1 · CNN · F1 0.7531: GMD-full exp 4.2 · PaSST · F1 0.7367: GMD-full exp 5.1 · PaSST · F1 0.7269: GMD-full repeat exp 5.2 · PaSST · F1 0.7313: GMD-full repeat exp 5.3 · CNN · F1 0.7646: GMD-full repeat exp 5.4 · CNN · F1 0.7827: GMD-full repeat exp 6.1 · PaSST · F1 0.7247: t-SNE and Time patchout exp 6.2 · PaSST · F1 0.3861: t-SNE and Time patchout exp 7.1 · PaSST · F1 0.8582: PaSST MLP 3 Layers exp 8.1 · PaSST · F1 0.8604: PaSST MLP 5 Layers exp 8.2 · PaSST · F1 0.7964: PaSST MLP 7 Layers exp 8.3 · PaSST · F1 0.8659: PaSST MLP 4 Layers exp 8.4 · PaSST · F1 0.8352: PaSST MLP 2 Layers exp 8.5 · PaSST · F1 0.8424: PaSST MLP 6 Layers exp 8.6 · PaSST · F1 0.8577: PaSST MLP 4 Layers, Symmetric Bottleneck exp 8.7 · PaSST · F1 0.8604: PaSST MLP 4 Layers, Progressive Bottleneck exp 9.1 · CNN · F1 0.8779: CNN 5 Conv Layers exp 9.2 · CNN · F1 0.8944: CNN 7 Conv Layers exp 9.3 · CNN · F1 0.8900: CNN 9 Conv Layers exp 10.1 · CNN · F1 0.9080: CNN 7 Conv Layers , GaussianNoise, RoomSimulator exp 10.2 · PaSST · F1 0.7953: PaSST MLP 4 Layers, GaussianNoise, RoomSimulator exp 10.3 · PaSST · F1 0.8446: PaSST MLP 4 Layers, TimeStretch exp 10.4 · CNN · F1 0.8632: CNN 7 Conv Layers , TimeStretch exp 11.1 · PaSST · F1 0.8429: PaSST MLP 4 Layers, Circular padding exp 11.2 · PaSST · F1 0.8752: PaSST MLP 4 Layers, Reflection padding exp 11.3 · CNN · F1 0.8747: CNN 7 Conv Layers , GaussianNoise, RoomSimulator , Reflection padding exp 11.4 · CNN · F1 0.8736: CNN 7 Conv Layers , Reflection padding exp 11.5 · CNN · F1 0.8747: CNN 7 Conv Layers , Circular padding
Macro-F1 by round; ● CNN, ● PaSST; ringed dots are the headline results. Source: workbook · Overall.
Model Dataset Configuration Notebook
1 1.1 CNN Rock only (narrow) Defaults with rock styles 0.9204 n/a GMD_CNN_prototype3
1 1.2 PaSST Rock only (narrow) Defaults with rock styles 0.8544 L4 PaSST_setup2
2 2.1 PaSST GMD-full (74 class) PaSST config 0.8660 L4 PaSST_setup3
2 2.2 PaSST GMD-full (74 class) PaSST config 0.8699 L4 PaSST_setup4
3 3.1 CNN GMD-mini (low data) GMD-mini 0.3267 n/a GMD_CNN_prototype4
3 3.2 PaSST GMD-mini (low data) GMD-mini 0.3911 L4 PaSST_setup5
4 4.1 CNN GMD-full (74 class) GMD-full 0.7531 n/a GMD_CNN_prototype5
4 4.2 PaSST GMD-full (74 class) GMD-full 0.7367 L4 PaSST_setup6
5 5.1 PaSST GMD-full (74 class) GMD-full repeat 0.7269 A100 PaSST_setup6_2
5 5.2 PaSST GMD-full (74 class) GMD-full repeat 0.7313 L4 PaSST_setup6_3
5 5.3 CNN GMD-full (74 class) GMD-full repeat 0.7646 n/a GMD_CNN_prototype5_2
5 5.4 CNN GMD-full (74 class) GMD-full repeat 0.7827 A100 GMD_CNN_prototype5_3
6 6.1 PaSST GMD-full (74 class) t-SNE and Time patchout 0.7247 L4 PaSST_setup6_4
6 6.2 PaSST GMD-full (74 class) t-SNE and Time patchout 0.3861 L4 PaSST_setup5_2
7 7.1 PaSST GMD-full (74 class) PaSST MLP 3 Layers 0.8582 L4 PaSST_setup6_5
8 8.1 PaSST GMD-full (74 class) PaSST MLP 5 Layers 0.8604 L4 PaSST_setup6_6
8 8.2 PaSST GMD-full (74 class) PaSST MLP 7 Layers 0.7964 A100 PaSST_setup6_7
8 8.3 PaSST GMD-full (74 class) PaSST MLP 4 Layers 0.8659 A100 PaSST_setup6_8
8 8.4 PaSST GMD-full (74 class) PaSST MLP 2 Layers 0.8352 A100 PaSST_setup6_9
8 8.5 PaSST GMD-full (74 class) PaSST MLP 6 Layers 0.8424 A100 PaSST_setup6_10
8 8.6 PaSST GMD-full (74 class) PaSST MLP 4 Layers, Symmetric Bottleneck 0.8577 A100 PaSST_setup6_11
8 8.7 PaSST GMD-full (74 class) PaSST MLP 4 Layers, Progressive Bottleneck 0.8604 A100 PaSST_setup6_12
9 9.1 CNN GMD-full (74 class) CNN 5 Conv Layers 0.8779 L4 GMD_CNN_prototype6
9 9.2 CNN GMD-full (74 class) CNN 7 Conv Layers 0.8944 A100 GMD_CNN_prototype6_2
9 9.3 CNN GMD-full (74 class) CNN 9 Conv Layers 0.8900 A100 GMD_CNN_prototype6_3
10 10.1 CNN GMD-full (74 class) CNN 7 Conv Layers , GaussianNoise, RoomSimulator 0.9080 A100 GMD_CNN_prototype6_4
10 10.2 PaSST GMD-full (74 class) PaSST MLP 4 Layers, GaussianNoise, RoomSimulator 0.7953 A100 PaSST_setup6_13
10 10.3 PaSST GMD-full (74 class) PaSST MLP 4 Layers, TimeStretch 0.8446 A100 PaSST_setup6_14
10 10.4 CNN GMD-full (74 class) CNN 7 Conv Layers , TimeStretch 0.8632 A100 GMD_CNN_prototype6_5
11 11.1 PaSST GMD-full (74 class) PaSST MLP 4 Layers, Circular padding 0.8429 A100 PaSST_setup6_15
11 11.2 PaSST GMD-full (74 class) PaSST MLP 4 Layers, Reflection padding 0.8752 L4 PaSST_setup6_16
11 11.3 CNN GMD-full (74 class) CNN 7 Conv Layers , GaussianNoise, RoomSimulator , Reflection padding 0.8747 L4 GMD_CNN_prototype6_6
11 11.4 CNN GMD-full (74 class) CNN 7 Conv Layers , Reflection padding 0.8736 L4 GMD_CNN_prototype6_7
11 11.5 CNN GMD-full (74 class) CNN 7 Conv Layers , Circular padding 0.8747 L4 GMD_CNN_prototype6_8

03 · Compare

What changed between two runs?