1
0
mirror of https://github.com/malarinv/tacotron2 synced 2026-03-08 09:42:34 +00:00

17 Commits

Author SHA1 Message Date
rafaelvalle
ba8cf36198 requirements.txt: removing pytorch 0.4 from requirements. upgrading to 1.0 2018-11-27 08:04:21 -08:00
rafaelvalle
b5e0a93946 inference.ipynb: updating inference file with relative paths 2018-11-27 08:04:04 -08:00
rafaelvalle
58b0ec61bd README.md: updating requirements and inference demo 2018-11-27 08:03:34 -08:00
rafaelvalle
1ad939df1a inference.ipynb: setting relative model paths 2018-11-27 07:45:09 -08:00
rafaelvalle
32b9a135d0 utils.py: updating 2018-11-25 22:34:38 -08:00
rafaelvalle
ce29e13959 train.py: updating 2018-11-25 22:34:34 -08:00
rafaelvalle
1ea6ed5861 text/symbols.py: updating symbols 2018-11-25 22:34:26 -08:00
rafaelvalle
cdfde985e5 text/__init__.py: remove stop token 2018-11-25 22:34:11 -08:00
rafaelvalle
e314bb4cd0 stft.py: fix filter winlength error 2018-11-25 22:33:52 -08:00
rafaelvalle
4af4ccb135 model.py: rewrite 2018-11-25 22:33:38 -08:00
rafaelvalle
1ec0e5e8cd layers.py: rewrite 2018-11-25 22:33:32 -08:00
rafaelvalle
249afd8043 inference.ipynb: import taco2model to be public 2018-11-25 22:33:16 -08:00
rafaelvalle
1b243d5d5a hparams.py:rewrite 2018-11-25 22:33:05 -08:00
rafaelvalle
d0aa9e7d32 distributed.py: rewrite 2018-11-25 22:32:54 -08:00
rafaelvalle
1683a57ae5 data_utils.py: rewrite 2018-11-25 22:32:47 -08:00
Rafael Valle
fc0cf6a89a Merge pull request #53 from cobr123/patch-1
add pillow
2018-07-02 14:32:21 -07:00
cobr123
8de38495be add pillow 2018-07-02 19:35:51 +03:00
6 changed files with 45 additions and 36 deletions

3
.gitmodules vendored
View File

@@ -1,3 +0,0 @@
[submodule "waveglow"]
path = waveglow
url = https://github.com/NVIDIA/waveglow

33
README.md Normal file → Executable file
View File

@@ -1,6 +1,6 @@
# Tacotron 2 (without wavenet)
Tacotron 2 PyTorch implementation of [Natural TTS Synthesis By Conditioning
PyTorch implementation of [Natural TTS Synthesis By Conditioning
Wavenet On Mel Spectrogram Predictions](https://arxiv.org/pdf/1712.05884.pdf).
This implementation includes **distributed** and **fp16** support
@@ -11,9 +11,7 @@ Distributed and FP16 support relies on work by Christian Sarofeen and NVIDIA's
![Alignment, Predicted Mel Spectrogram, Target Mel Spectrogram](tensorboard.png)
[Download demo audio](https://github.com/NVIDIA/tacotron2/blob/master/demo.wav) trained on LJS and using Ryuchi Yamamoto's [pre-trained Mixture of Logistics
wavenet](https://github.com/r9y9/wavenet_vocoder/)
"Scientists at the CERN laboratory say they have discovered a new particle."
Visit our [website] for audio samples.
## Pre-requisites
1. NVIDIA GPU + CUDA cuDNN
@@ -24,11 +22,9 @@ wavenet](https://github.com/r9y9/wavenet_vocoder/)
3. CD into this repo: `cd tacotron2`
4. Update .wav paths: `sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt`
- Alternatively, set `load_mel_from_disk=True` in `hparams.py` and update mel-spectrogram paths
5. Install [pytorch 0.4](https://github.com/pytorch/pytorch)
5. Install [PyTorch 1.0]
6. Install python requirements or build docker image
- Install python requirements: `pip install -r requirements.txt`
- **OR**
- Build docker image: `docker build --tag tacotron2 .`
## Training
1. `python train.py --output_directory=outdir --log_directory=logdir`
@@ -37,17 +33,22 @@ wavenet](https://github.com/r9y9/wavenet_vocoder/)
## Multi-GPU (distributed) and FP16 Training
1. `python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True`
## Inference
When performing Mel-Spectrogram to Audio synthesis with a WaveNet model, make sure Tacotron 2 and WaveNet were trained on the same mel-spectrogram representation. Follow these steps to use a a simple inference pipeline using griffin-lim:
1. `jupyter notebook --ip=127.0.0.1 --port=31337`
2. load inference.ipynb
## Inference demo
1. Download our published [Tacotron 2] model
2. Download our published [WaveGlow] model
3. `jupyter notebook --ip=127.0.0.1 --port=31337`
4. Load inference.ipynb
N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2
and the Mel decoder were trained on the same mel-spectrogram representation.
## Related repos
[nv-wavenet](https://github.com/NVIDIA/nv-wavenet/): Faster than real-time
wavenet inference
[WaveGlow](https://github.com/NVIDIA/WaveGlow) Faster than real time Flow-based
Generative Network for Speech Synthesis
[nv-wavenet](https://github.com/NVIDIA/nv-wavenet/) Faster than real time
WaveNet.
## Acknowledgements
This implementation uses code from the following repos: [Keith
@@ -61,3 +62,7 @@ We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan
Wang and Zongheng Yang.
[WaveGlow]: https://drive.google.com/file/d/1cjKPHbtAMh_4HTHmuIGNkbOkPBD9qwhj/view?usp=sharing
[Tacotron 2]: https://drive.google.com/file/d/1c5ZTuT7J08wLUoVZ2KkUs_VdZuJ86ZqA/view?usp=sharing
[pytorch 1.0]: https://github.com/pytorch/pytorch#installation
[website]: https://nv-adlr.github.io/WaveGlow

View File

@@ -164,9 +164,9 @@ def apply_gradient_allreduce(module):
param._execution_engine.queue_callback(allreduce_params)
if param.requires_grad:
param.register_hook(allreduce_hook)
def set_needs_reduction(self, input, output):
self.needs_reduction = True
module.register_forward_hook(set_needs_reduction)
return module

39
inference.ipynb Executable file → Normal file

File diff suppressed because one or more lines are too long

View File

@@ -1,4 +1,3 @@
torch==0.4.0
matplotlib==2.1.0
tensorflow
numpy==1.13.3

Submodule waveglow deleted from 4b1001fa33