malarinv
/
tacotron2
mirror of https://github.com/malarinv/tacotron2
1
0
Fork 0
Code Issues Releases Wiki Activity
44 Commits
10 Branches
0 Tags
3.3 MiB
Python 96.9%
Makefile 3.1%
 
 
Go to file
Rafael Valle 1071023017 train.py: patching score_mask_value formerly inf, not concrete value, for compatibility with pytorch 2018-05-15 09:50:56 -07:00
filelists changing structure for better organization 2018-05-03 17:14:45 -07:00
text text/: adding Keith Itos text pre-processing 2018-05-03 15:17:26 -07:00
Dockerfile Dockerfile: adding dockerfile 2018-05-04 09:39:34 -07:00
LICENSE Update license such that it appears on repo fron tpage 2018-05-03 23:18:34 -07:00
README.md README.md: describing how to load mel from disk 2018-05-15 08:50:21 -07:00
audio_processing.py adding python files 2018-05-03 15:16:57 -07:00
data_utils.py data_utils.py: adding support for loading mel from disk 2018-05-15 08:42:06 -07:00
distributed.py adding python files 2018-05-03 15:16:57 -07:00
fp16_optimizer.py adding python files 2018-05-03 15:16:57 -07:00
hparams.py hparams.py: adding load_mel_from_disk params 2018-05-15 08:41:03 -07:00
inference.ipynb ipynb typo 2018-05-05 17:30:08 -07:00
layers.py adding python files 2018-05-03 15:16:57 -07:00
logger.py adding python files 2018-05-03 15:16:57 -07:00
loss_function.py adding python files 2018-05-03 15:16:57 -07:00
loss_scaler.py loss_scaler.py: patching loss scaler for compatibility with current pytorch 2018-05-15 09:50:08 -07:00
model.py model.py: mixed squeeze target. fixing 2018-05-06 08:58:01 -07:00
multiproc.py adding python files 2018-05-03 15:16:57 -07:00
plotting_utils.py adding python files 2018-05-03 15:16:57 -07:00
requirements.txt requirements.txt: updating tensorflow requirements 2018-05-04 09:44:14 -07:00
stft.py adding python files 2018-05-03 15:16:57 -07:00
tensorboard.png tensorboard.png: adding tensorboard image 2018-05-03 15:11:54 -07:00
train.py train.py: patching score_mask_value formerly inf, not concrete value, for compatibility with pytorch 2018-05-15 09:50:56 -07:00
utils.py mask utils update for 0.4 cuda 2018-05-04 10:14:30 -07:00

README.md

Tacotron 2 (without wavenet)

Tacotron 2 PyTorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions.

This implementation includes distributed and fp16 support and uses the LJSpeech dataset.

Distributed and FP16 support relies on work by Christian Sarofeen and NVIDIA's Apex Library.

Alignment, Predicted Mel Spectrogram, Target Mel Spectrogram

Pre-requisites

  1. NVIDIA GPU + CUDA cuDNN

Setup

  1. Download and extract the LJ Speech dataset
  2. Clone this repo: git clone https://github.com/NVIDIA/tacotron2.git
  3. CD into this repo: cd tacotron2
  4. Update .wav paths: sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt
    • Alternatively, set load_mel_from_disk=True in hparams.py and update mel-spectrogram paths
  5. Install pytorch 0.4
  6. Install python requirements or build docker image
    • Install python requirements: pip install -r requirements.txt
    • OR
    • Build docker image: docker build --tag tacotron2 .

Training

  1. python train.py --output_directory=outdir --log_directory=logdir
  2. (OPTIONAL) tensorboard --logdir=outdir/logdir

Multi-GPU (distributed) and FP16 Training

  1. python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True

Inference

  1. jupyter notebook --ip=127.0.0.1 --port=31337
  2. load inference.ipynb

Related repos

nv-wavenet: Faster than real-time wavenet inference

Acknowledgements

This implementation uses code from the following repos: Keith Ito, Prem Seetharaman as described in our code.

We are inspired by Ryuchi Yamamoto's Tacotron PyTorch implementation.

We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang.