mirror of
https://github.com/malarinv/tacotron2
synced 2026-03-08 09:42:34 +00:00
Compare commits
17 Commits
latest_mod
...
clean_slat
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ba8cf36198 | ||
|
|
b5e0a93946 | ||
|
|
58b0ec61bd | ||
|
|
1ad939df1a | ||
|
|
32b9a135d0 | ||
|
|
ce29e13959 | ||
|
|
1ea6ed5861 | ||
|
|
cdfde985e5 | ||
|
|
e314bb4cd0 | ||
|
|
4af4ccb135 | ||
|
|
1ec0e5e8cd | ||
|
|
249afd8043 | ||
|
|
1b243d5d5a | ||
|
|
d0aa9e7d32 | ||
|
|
1683a57ae5 | ||
|
|
fc0cf6a89a | ||
|
|
8de38495be |
3
.gitmodules
vendored
3
.gitmodules
vendored
@@ -1,3 +0,0 @@
|
||||
[submodule "waveglow"]
|
||||
path = waveglow
|
||||
url = https://github.com/NVIDIA/waveglow
|
||||
33
README.md
Normal file → Executable file
33
README.md
Normal file → Executable file
@@ -1,6 +1,6 @@
|
||||
# Tacotron 2 (without wavenet)
|
||||
|
||||
Tacotron 2 PyTorch implementation of [Natural TTS Synthesis By Conditioning
|
||||
PyTorch implementation of [Natural TTS Synthesis By Conditioning
|
||||
Wavenet On Mel Spectrogram Predictions](https://arxiv.org/pdf/1712.05884.pdf).
|
||||
|
||||
This implementation includes **distributed** and **fp16** support
|
||||
@@ -11,9 +11,7 @@ Distributed and FP16 support relies on work by Christian Sarofeen and NVIDIA's
|
||||
|
||||

|
||||
|
||||
[Download demo audio](https://github.com/NVIDIA/tacotron2/blob/master/demo.wav) trained on LJS and using Ryuchi Yamamoto's [pre-trained Mixture of Logistics
|
||||
wavenet](https://github.com/r9y9/wavenet_vocoder/)
|
||||
"Scientists at the CERN laboratory say they have discovered a new particle."
|
||||
Visit our [website] for audio samples.
|
||||
|
||||
## Pre-requisites
|
||||
1. NVIDIA GPU + CUDA cuDNN
|
||||
@@ -24,11 +22,9 @@ wavenet](https://github.com/r9y9/wavenet_vocoder/)
|
||||
3. CD into this repo: `cd tacotron2`
|
||||
4. Update .wav paths: `sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt`
|
||||
- Alternatively, set `load_mel_from_disk=True` in `hparams.py` and update mel-spectrogram paths
|
||||
5. Install [pytorch 0.4](https://github.com/pytorch/pytorch)
|
||||
5. Install [PyTorch 1.0]
|
||||
6. Install python requirements or build docker image
|
||||
- Install python requirements: `pip install -r requirements.txt`
|
||||
- **OR**
|
||||
- Build docker image: `docker build --tag tacotron2 .`
|
||||
|
||||
## Training
|
||||
1. `python train.py --output_directory=outdir --log_directory=logdir`
|
||||
@@ -37,17 +33,22 @@ wavenet](https://github.com/r9y9/wavenet_vocoder/)
|
||||
## Multi-GPU (distributed) and FP16 Training
|
||||
1. `python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True`
|
||||
|
||||
## Inference
|
||||
When performing Mel-Spectrogram to Audio synthesis with a WaveNet model, make sure Tacotron 2 and WaveNet were trained on the same mel-spectrogram representation. Follow these steps to use a a simple inference pipeline using griffin-lim:
|
||||
|
||||
1. `jupyter notebook --ip=127.0.0.1 --port=31337`
|
||||
2. load inference.ipynb
|
||||
## Inference demo
|
||||
1. Download our published [Tacotron 2] model
|
||||
2. Download our published [WaveGlow] model
|
||||
3. `jupyter notebook --ip=127.0.0.1 --port=31337`
|
||||
4. Load inference.ipynb
|
||||
|
||||
N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2
|
||||
and the Mel decoder were trained on the same mel-spectrogram representation.
|
||||
|
||||
|
||||
## Related repos
|
||||
[nv-wavenet](https://github.com/NVIDIA/nv-wavenet/): Faster than real-time
|
||||
wavenet inference
|
||||
[WaveGlow](https://github.com/NVIDIA/WaveGlow) Faster than real time Flow-based
|
||||
Generative Network for Speech Synthesis
|
||||
|
||||
[nv-wavenet](https://github.com/NVIDIA/nv-wavenet/) Faster than real time
|
||||
WaveNet.
|
||||
|
||||
## Acknowledgements
|
||||
This implementation uses code from the following repos: [Keith
|
||||
@@ -61,3 +62,7 @@ We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan
|
||||
Wang and Zongheng Yang.
|
||||
|
||||
|
||||
[WaveGlow]: https://drive.google.com/file/d/1cjKPHbtAMh_4HTHmuIGNkbOkPBD9qwhj/view?usp=sharing
|
||||
[Tacotron 2]: https://drive.google.com/file/d/1c5ZTuT7J08wLUoVZ2KkUs_VdZuJ86ZqA/view?usp=sharing
|
||||
[pytorch 1.0]: https://github.com/pytorch/pytorch#installation
|
||||
[website]: https://nv-adlr.github.io/WaveGlow
|
||||
|
||||
@@ -164,9 +164,9 @@ def apply_gradient_allreduce(module):
|
||||
param._execution_engine.queue_callback(allreduce_params)
|
||||
if param.requires_grad:
|
||||
param.register_hook(allreduce_hook)
|
||||
|
||||
|
||||
def set_needs_reduction(self, input, output):
|
||||
self.needs_reduction = True
|
||||
|
||||
|
||||
module.register_forward_hook(set_needs_reduction)
|
||||
return module
|
||||
|
||||
39
inference.ipynb
Executable file → Normal file
39
inference.ipynb
Executable file → Normal file
File diff suppressed because one or more lines are too long
@@ -1,4 +1,3 @@
|
||||
torch==0.4.0
|
||||
matplotlib==2.1.0
|
||||
tensorflow
|
||||
numpy==1.13.3
|
||||
|
||||
1
waveglow
1
waveglow
Submodule waveglow deleted from 4b1001fa33
Reference in New Issue
Block a user