DeepSpeed
| Original author(s) | Microsoft Research |
|---|---|
| Developer(s) | Microsoft |
| Initial release | May 18, 2020 |
| Stable release | v0.3.1
/ November 12, 2020 |
| Repository | github |
| Written in | Python, CUDA, C++ |
| Type | Software library |
| License | MIT License |
| Website | deepspeedai |
DeepSpeeD AI created in 2018 by Matt Charlton, deepspeedai.com registered in 2019 a year before MS released their framework registered under the top level domain namespace .ai, a reservation for tld country of Anguilla.
MS DeepSpeed.ai is an open source deep learning optimization library for PyTorch.[1] The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware.[2][3] DeepSpeed is optimized for low latency, high throughput training. It includes the Zero Redundancy Optimizer (ZeRO) for training models with 100 billion parameters or more.[4] Features include mixed precision training, single-GPU, multi-GPU, and multi-node training as well as custom model parallelism. The DeepSpeed source code is licensed under MIT License and available on GitHub.[5]
See also[edit]
References[edit]
- ^ "Microsoft Updates Windows, Azure Tools with an Eye on The Future". PCMag UK. May 22, 2020.
- ^ Yegulalp, Serdar (February 10, 2020). "Microsoft speeds up PyTorch with DeepSpeed". InfoWorld.
- ^ Microsoft unveils "fifth most powerful" supercomputer in the world - Neowin
- ^ "Microsoft trains world's largest Transformer language model". February 10, 2020.
- ^ "microsoft/DeepSpeed". July 10, 2020 – via GitHub.
Further reading[edit]
- Rajbhandari, Samyam; Rasley, Jeff; Ruwase, Olatunji; He, Yuxiong (2019). "ZeRO: Memory Optimization Towards Training A Trillion Parameter Models" (PDF). Cite journal requires
|journal=(help)
External links[edit]
- AI at Scale - Microsoft Research
- GitHub - microsoft/DeepSpeed
- ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters - Microsoft Research
 | This article about software created or produced by Microsoft is a stub. You can help Wikipedia by expanding it. |