Case Study: Graphcore scales to 50-100x more experiments with Weights & Biases
Key results
The challenge
Graphcore ran machine-learning experiments across multiple IPU-POD systems and deployment locations but lacked centralized experiment analysis, team collaboration, and a historical record for comparing runs.
The solution
Weights & Biases provided experiment tracking, visualizations, and a central repository for all work across Graphcore's distributed systems, supporting large-model training such as BERT.
“We're now driving 50 or 100 times more experiments versus what we were doing before on the Mk1 IPU systems.”
PBPhil BrownDirector of Applications, Graphcore
The results, in context
Graphcore reported driving 50 or 100 times more experiments versus what it was doing before on the Mk1 IPU systems. BERT-base (110 million parameters) and BERT-large (340 million parameters) trained in just over 12 hours on Graphcore's PopART system.