ZeRO & DeepSpeed: Enabling models with over 100 Billion Parametrs.

An open source library released by Microsoft; called DeepSpeed, greatly advances model training for very large models consisting of more than 100 Billion Parameters. Deep Speed improves four aspects: Scale: up to 10x bigger Speed: Up to 5x faster Cost: 5x reduction in cost Usability: Low code impact. Please follow the link at the Microsoft Research Blog.

High Performance Big Data Azure Virtual Machines H-Series now available.

Microsoft Announced the availability of H-Series machines in the West US 2 region. These machines are specific to very complex and high performing task, which typically include:       ✓ Computational Fluid Dynamics tasks run by engineers and scientists     ✓ Automotive crash testing     ✓ Genome and molecular research     ✓ Seismic exploration     ✓ Weather forecasting simulations These are certainly the fastest in … Read More