Fastspeech2#1572

Merged

mravanelli merged 97 commits into

speechbrain:developfrom

bloodraven66:fastspeech2

Jul 17, 2023

bloodraven66 commented Sep 9, 2022

Contributor

@mravanelli , same as fastpitch, need to refactor a bit once fastspeech is merged.

bloodraven66 added 11 commits

June 7, 2022 13:21


          adding clean fs codes for pr

bc4bdae


          adding clean fs model codes

7a9792f


          addinf optional 1dcnn configuration instead of ffn for transformer en…

7840fee

…coder layer


          add hifigan vocoder inference


          metadata related

899efe3


          clean codes and add docs

7ddaeb7


          fix bug

a806db5


          changes based on pr comments

8bf823b


          update readme with fs info

ad1e4af


          added fastspeech2

f8040fb


          working fastspeech2

6ffce85

BenoitWang commented Sep 23, 2022

Collaborator

Hi @bloodraven66, I start with fastspeech2 while I am running fastpitch.
The quality seems very good! Some small points :

better with speechbrain CNN wrappers
please update the docstrings
seems better to use MSELoss for duration, energy and pitch
don't forget to add the recipe into test/recipes.csv

I will do a more detailed review when the tests are finished, thanks!


          reduce params to 4layer 1024, changed energy from librosa to torch, i…

f7ba7f5

…ncreased workers to 32

mravanelli requested a review from BenoitWang

November 2, 2022 16:07

bloodraven66 added 2 commits

November 10, 2022 22:28


          add sb wrappers, skip for energy and pitch embed due to padding (same…

7814cb9

…) issue for input dim of 1


          fix predit pitch shape

6a02b86

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/README.md Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech/hparams/train.yaml Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech2/hparams/train.yaml Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech2/hparams/train.yaml Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech2/hparams/train.yaml Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/README.md Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech2/hparams/train.yaml Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech2/train.py Outdated

BenoitWang reviewed

View reviewed changes

recipes/LJSpeech/TTS/fastspeech2/train.py

BenoitWang reviewed

View reviewed changes

speechbrain/lobes/models/FastSpeech.py Outdated

BenoitWang reviewed

View reviewed changes

speechbrain/lobes/models/FastSpeech2.py Outdated

BenoitWang reviewed

View reviewed changes

speechbrain/lobes/models/FastSpeech2.py Outdated

BenoitWang reviewed

View reviewed changes

speechbrain/lobes/models/FastSpeech2.py Outdated

BenoitWang reviewed

View reviewed changes

speechbrain/lobes/models/FastSpeech2.py Outdated

BenoitWang and others added 19 commits

July 9, 2023 22:58


          dim=384, more epochs

66e8b26


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

2323cc3

…rain into fastspeech2


          recover learning rate

71fedce


          spn loss equals zero after 8 epochs, threshold 0.8 for spn predictor

ba02b23


          add silence for punctuations, modify functions

62e9c1a


          fixes

2d0f4a6


          freeze spn predictor after 8 epoch and suppress warning

5b4704e


          Update README.md


          Update README.md

2fa5988


          Merge branch 'develop' into fastspeech2

f6f41bf


          Update LJSpeech.csv

783f674


          Update LJSpeech.csv

3a53940


          final updates

f848b69


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

cd71bb1

…rain into fastspeech2


          update requirements

b9d3dc0


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

daa8aaf

…rain into fastspeech2


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

63cfd22

…rain into fastspeech2


          Update requirements.txt

fbcd142


          Update requirements.txt

7b05521

mravanelli self-requested a review

July 17, 2023 16:03

BenoitWang and others added 7 commits

July 17, 2023 16:30


          fix interface

c6958b5


          Update requirements.txt

10e6d6b


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

40de2d4

…rain into fastspeech2


          fix failing test in HuggingFaceWav2Vec2Pretrain

092375e


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

da8ad01

…rain into fastspeech2


          fix examples

be1a43a


          Merge branch 'fastspeech2' of https://github.com/bloodraven66/speechb…

8bdbda6

…rain into fastspeech2

mravanelli approved these changes

View reviewed changes

mravanelli commented Jul 17, 2023

Collaborator

Everything LGTM! It was quite a challenge obtaining satisfactory results with the fastspeech2 model. However, after remarkable efforts, we now have an exceptional fastspeech2 TTS in SpeechBrain. Thank you @BenoitWang @bloodraven66 @pradnya-git-dev and others for your commitment and hard work.

mravanelli merged commit 7510c5b into speechbrain:develop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet