malarinv
/
tacotron2
mirror of https://github.com/malarinv/tacotron2
1
0
Fork 0
Code Issues Releases Wiki Activity

Merge pull request #136 from GrzegorzKarchNV/master 

Fixing concatenation error for fp16 distributed training
experiments
Rafael Valle 2019-02-01 12:10:42 -08:00 committed by GitHub
parent 825ffa47d1 df4a466af2
commit f2c94d94fd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
distributed.py
View File

@ -140,7 +140,7 @@ def apply_gradient_allreduce(module):
buckets = {} buckets = {}
for param in module.parameters(): for param in module.parameters():
if param.requires_grad and param.grad is not None: if param.requires_grad and param.grad is not None:
tp = type(param.data) tp = param.data.dtype
if tp not in buckets: if tp not in buckets:
buckets[tp] = [] buckets[tp] = []
buckets[tp].append(param) buckets[tp].append(param)