Skip to content

Adding recipe for Listenable Maps for Audio Classifiers#2538

Merged
mravanelli merged 322 commits into
speechbrain:developfrom
fpaissan:icml2024_lmac
Jul 3, 2024
Merged

Adding recipe for Listenable Maps for Audio Classifiers#2538
mravanelli merged 322 commits into
speechbrain:developfrom
fpaissan:icml2024_lmac

Conversation

@fpaissan

@fpaissan fpaissan commented May 2, 2024

Copy link
Copy Markdown
Collaborator

What does this PR do?

  • Implements the work of the ICML'24 paper L-MAC (https://arxiv.org/abs/2403.13086).
  • Refactors exiting L2I and PIQ recipes to unify the interpretability approaches under a single structure. This simplifies code shared among the interpretability pipelines. Also, it minimizes the room for error in the comparison of the techniques.
Before submitting
  • Did you read the contributor guideline?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified
  • Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
  • Review the self-review checklist to ensure the code is ready for review

ycemsubakan and others added 30 commits January 21, 2024 14:39
@ycemsubakan
Merge branch 'icml2024_cemstuff' of github.com:fpaissan/audio_interpr…
f687b60 
…etability into icml2024_cemstuff
@fpaissan
works on cnn14 -- but have a bad checkpoint
a659533
@fpaissan
fixed l2i as well
f632940
@fpaissan
fixed acc in l2i
596b56c
@fpaissan
fix not listenable
163374b
@fpaissan
updated logging for eval
5f30445
@fpaissan
a bit less verbose
1362d62
@fpaissan
printing at sample level
a6b8581
@fpaissan
fix logging - was missing avg
5f7c5cc
@fpaissan
was messing up in the forward
61a2c00
@fpaissan
now running train_piq.py
399fc9f
@fpaissan
minor corrections
c94e788
@fpaissan
fix l2i training with wham!
dd63174
@fpaissan
fixed l2i computation
59f7d5c
@fpaissan
linters
a962410
@fpaissan
add check for wham usage in eval
af5c0a9
@fpaissan
add sample saving during eval
6047a48
@fpaissan
bug fixes
ad8cf66
@fpaissan
added predictions info to the logging
b895803
@fpaissan
fixed id for overlap test
3ddc0b8
@fpaissan
cutting sample before saving
a3ccf71
@fpaissan
fixed l2i sampling rate
bd7aa57
@fpaissan
fixed random seed so eval will match
4bf6815
@fpaissan
running on full set
dbc5b96
@ycemsubakan
faithfulness fix
e77bfc7
@ycemsubakan
remove pdb
f36f3cd
@fpaissan
fix smoothgrad and IG
c891dd5
@fpaissan
fix nmf for pre-training
a5ebd89
@fpaissan
removed nmf reconstructions
2529b85
@fpaissan
truncated gaussian fix for smoothgrad
af88a98
ycemsubakan and others added 28 commits June 29, 2024 14:13
@ycemsubakan
fixed the header in eval.py
7c356ec
@ycemsubakan
added missing l2i command in train_l2i.py
ad1d08e
@ycemsubakan
fixes to train_lmac.py
68e5b85
@ycemsubakan
description for classifier_temp
b332ca9
@ycemsubakan
added comments for pretrained_interpreter and ljspeech_path
cbba482
@ycemsubakan
updated README to have more information on how to use LJSpeech
cce2f9a
@ycemsubakan
added information for piq_vit.yaml and piq_focalnet.yaml
dc74cd9
@ycemsubakan
added more explanation for LJSpeech downloading
e3767db
@ycemsubakan
added missing use_melspectra_log1p attribute to piq_vit.yaml and piq_…
ad7cb57 
…focalnet.yaml
@ycemsubakan
added an assert in eval.py for the pretrained path
5547ada
@ycemsubakan
updates to the readme, added table, updated l2i to print quantus metrics
aba93d5
@mravanelli
Update README.md
9809ecf
@ycemsubakan
added the description of pretained_interpreter in README.md.
d0b132e
@ycemsubakan
Merge branch 'icml2024_lmac' of github.com:fpaissan/speechbrain into …
4a9f128 
…icml2024_lmac
@ycemsubakan
fixed the problem in vit
1a4f25c
@mravanelli
Merge remote-tracking branch 'upstream/develop' into icml2024_lmac
3ed6bff
@ycemsubakan
fixing l2i tests
237817a
@ycemsubakan
Merge branch 'icml2024_lmac' of github.com:fpaissan/speechbrain into …
5a55a57 
…icml2024_lmac
@ycemsubakan
fixed ESC50.csv
2d5e6a1
@ycemsubakan
fixed the yaml tets
9ff7c66
@ycemsubakan
added links to files
25927a8
@ycemsubakan
fixed docstring tests
9e3ec34
@ycemsubakan
bug fix on psi model
99b19d5
@ycemsubakan
removing the classes from PIQ.py
f30e18a
@ycemsubakan
fixes in L2I psi classes
616cc7d
@ycemsubakan
handling sps comp exceptions
fb6480c
@ycemsubakan
added the dropbox links
5283876
@mravanelli
Update README.md
8b59cb2
@mravanelli mravanelli merged commit 2f9e2fd into speechbrain:develop Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mravanelli mravanelli mravanelli approved these changes

@ycemsubakan ycemsubakan Awaiting requested review from ycemsubakan

Assignees

@ycemsubakan ycemsubakan

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@fpaissan @mravanelli @ycemsubakan