Skip to content

/ Megatron-LM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

27 Open 10 Closed
27 Open 10 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels.
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀
Malicious domain name in openwebtext URL list
#46 opened Oct 2, 2020 by abhinavs95
Fully half precision optimizer
#45 opened Sep 26, 2020 by eric-haibin-lin
Incorporating Megatron-ML with DeepSpeed
#44 opened Sep 25, 2020 by msmolyak
Pretrain and generate
#43 opened Sep 21, 2020 by king-menin
Scaling up to GPT-3 size (175B)
#40 opened Aug 6, 2020 by nakosung
Is there a pretrained 8.3B parameter model?
#39 opened Aug 2, 2020 by shamoons
1
FileExistsError when training with a shared file-system
#38 opened Jul 7, 2020 by g-karthik
Huggingface <-> Megatron-LM Compatibility
#37 opened Jul 6, 2020 by usuyama
3
training in fp32 results error "TypeError: zero_grad() got an unexpected keyword argument 'set_grads_to_None'"
#36 opened Jun 26, 2020 by zarzen
FileNotFoundError Issues while running on 2 nodes
#35 opened Jun 11, 2020 by zarzen
tokenizer.py is missing in openwebtext
#33 opened May 28, 2020 by usuyama
evaluation loss < training loss
#32 opened May 7, 2020 by safooray
Evaluating on all GLUE tasks
#31 opened May 7, 2020 by safooray
Calculate the TFLOPS performance with elapsed time per iteration
#27 opened Mar 25, 2020 by fengche86
1
dropout should be wrapped in `get_cuda_rng_tracker`
#26 opened Feb 3, 2020 by ngoyal2707
GPT2 evaluation, wikitext-2, wikitext-103, PTB
#24 opened Jan 9, 2020 by nlpBeginner
GPT-2 generation samples error
#23 opened Dec 9, 2019 by vlarine
1
bert model encoding error on python3.6.8
#22 opened Dec 8, 2019 by gentelyang
1
Why the backward function of _CopyToModelParallelRegion calls reduce fuction? can somebody share the mathematical proof
#19 opened Nov 6, 2019 by lxl910915
How to train on other corpora besides web/wiki text
#18 opened Nov 5, 2019 by akhileshgotmare
Why NEGOTIATE_ALLREDUCE is much longer in TensorFlow comparing to PyTorch?
#17 opened Oct 27, 2019 by zhaiyuyong
Error when runinng script pretrain_gpt2_distributed.sh
#15 opened Oct 17, 2019 by Lepiloff
3
1:train loss decrease too faster.2:learning rates did not change after warmup iter,always kept on 1.5e-4. Is it normal phenomenon?
#14 opened Oct 12, 2019 by kangzhonghua
No module named 'apex'
#13 opened Oct 10, 2019 by GrahamboJangles
3
Collecting Wikipedia Training Data issues
#12 opened Oct 7, 2019 by great-attraction
ProTip! Type g i on any issue or pull request to go back to the issue listing page.
You can’t perform that action at this time.