Stay tuned for updates!

This repo includes a number of changes from NVIDIA's repo, Tacotron 2 (without wavenet). Our Grapheme-to-Phoneme solution relies on the eng_to_ipa module with an epitran fallback for non-dictionary words. Epitran requires the FLITE module to function, due to English's poor graphic-phonetic correspondance. Please note that this will likely cause installation difficulties on windows systems.

IPA Preprocessing: This repo contains an optional grapheme-to-phoneme conversion script for improved model stability and generalization. To use it, set ipa_preprocessing = True in hparams.py.

This model no longer uses english_cleaners during the training loop.

Installing epitran and FLITE: https://github.com/dmort27/epitran

Figure out the best way to integrate dataset WAV cleaner functions Standalone package? Command-line preprocessing tools?

Integrate Noice autotranscription tools

Preprocess WAVs to .npy for training

Add TQDM progress bar to preprocessing function

PyTorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions.

This implementation includes distributed and automatic mixed precision support and uses the LJSpeech dataset.

Distributed and Automatic Mixed Precision support relies on NVIDIA's Apex and AMP.

Visit our website for audio samples using our published Tacotron 2 and WaveGlow models.

1. NVIDIA GPU + CUDA cuDNN

1. Download and extract the LJ Speech dataset
2. Clone this repo: git clone https://github.com/NVIDIA/tacotron2.git
3. CD into this repo: cd tacotron2
4. Initialize submodule: git submodule init; git submodule update
5. Update .wav paths: sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt
   • Alternatively, set load_mel_from_disk=True in hparams.py and update mel-spectrogram paths
6. Install PyTorch 1.0
7. Install Apex
8. Install python requirements or build docker image
   • Install python requirements: pip install -r requirements.txt

1. python train.py --output_directory=outdir --log_directory=logdir
2. (OPTIONAL) tensorboard --logdir=outdir/logdir

Training using a pre-trained model can lead to faster convergence
By default, the dataset dependent text embedding layers are ignored

1. Download our published Tacotron 2 model
2. python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start

1. python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True

1. Download our published Tacotron 2 model
2. Download our published WaveGlow model
3. jupyter notebook --ip=127.0.0.1 --port=31337
4. Load inference.ipynb

N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2 and the Mel decoder were trained on the same mel-spectrogram representation.

WaveGlow Faster than real time Flow-based Generative Network for Speech Synthesis

nv-wavenet Faster than real time WaveNet.

This implementation uses code from the following repos: Keith Ito, Prem Seetharaman as described in our code.

We are inspired by Ryuchi Yamamoto's Tacotron PyTorch implementation.

We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang.