Case Study: Vercel's v0 saw a 40x latency improvement with Fireworks AI
Key results
The challenge
Vercel's v0 needed faster, more reliable code generation and auto-fixing than a general-purpose model provided, where multiple passes were sometimes required to fix a file.
The solution
Vercel worked with Fireworks AI to build a fine-tuned reinforcement learning model for v0's auto-fixer and composite generation.
“Using a fine-tuned reinforcement learning model with Fireworks, we perform substantially better than SOTA. In our evaluation, Sonnet 3.5 compiled at 62%, and we got our error-free generation rate well into the 90s.”
MUMalte UblCTO, Vercel
The results, in context
Vercel reported a 40x improvement in end-to-end latency for v0's auto-fixer model versus gpt-4o-mini, with the v0-1.5-md model reaching a 93.87% error-free generation rate and the vercel-autofixer-01 model running at 8,130 chars/sec. The company said its error-free generation rate moved well into the 90s versus 62% for Sonnet 3.5 in its evaluation.