2x Throughput: Why AI Cloud Modal Picked Nvidia's Vera CPU

Last week, Nvidia hand-delivered a single server to Modal's San Francisco office. A few days later, the fast-growing AI cloud startup announced it would do something it had never done before: run its workloads on a chip that is not from Intel or AMD.

The reason was one number. In Modal's own tests, Nvidia's new Vera CPU delivered up to 2x the throughput for AI agent tasks.

Quick answer

• Modal, which rents computing power to AI developers, will start using Nvidia's Vera central processor to host AI agents. Until now it ran that work only on Intel and AMD chips.
• Modal's tests showed up to double the task throughput on Vera versus an equivalent x86 server, with per-task speed equal or better.
• Modal is not dropping Intel or AMD. It will run a mixed fleet and put each job on whichever hardware runs it best.
• For Nvidia, it is an early sign that its push into CPUs, a market it values at roughly $200 billion, is gaining traction.

What Modal announced

Modal said on Monday that it plans to use Nvidia's Vera CPU for hosting agents, the AI systems that carry out long-running tasks for people. The Economic Times, carrying the Reuters report, called it an early sign that Nvidia's CPU push has traction.

The move covers Modal Sandboxes. These are isolated computers where AI agents run the code they write, as TNW explains. The product makes $300M a year, according to what Modal said last week.

Modal has grown fast. Last week it raised $750 million in a round led by Accel at a $15.75 billion valuation. It says it is on track to record $1 billion in sales over the next year, that usage has tripled, and that revenue has grown more than 15 times year-over-year.

Modal did not say when Vera-powered sandboxes will reach customers.

Why a CPU matters in the age of GPUs

Most AI headlines are about GPUs, the chips that train and run large models. But AI agents do a lot of plain CPU work too.

When a coding agent runs npm install, compiles a project, searches files or runs git commands, that happens on a CPU, not a GPU. Modal says a sandbox can sit quiet while it waits for a model, then suddenly use 10x its baseline CPU during an install.

Hundreds of these sandboxes run side by side on each server. So one customer's spike can slow down someone else's work. Engineers call this the noisy neighbour problem.

Modal also points out that slow CPU work costs more than the CPU itself. In reinforcement learning, expensive GPUs often sit idle until sandboxed tasks finish. Any time saved on the CPU is time saved on the GPU.

What the tests showed

Modal built a benchmark it calls Agentless Agent Bench. It has 50 coding tasks from Terminal-Bench, stripped of network calls, covering compilation, file grepping, git operations and device emulation.

Run one at a time on a quiet machine, Vera was often faster than chips from Intel, AMD and Amazon, but not always, and not by much. Modal engineer Erik Dunteman admitted he was skeptical at first.

The picture changed under load. When Modal packed the server with more and more containers, rival chips hit a throughput ceiling, while Vera kept finishing each task at its original pace.

The result: up to double the task throughput compared with an equivalent x86 server, with individual task latency equal to, or sometimes lower than, the x86 machine. Modal credits Vera's high memory bandwidth, which lets more containers share each core without throttling.

One caution from TNW: these are Modal's own results and have not been independently checked.

Nvidia's bigger bet on CPUs

Vera is Nvidia's new Arm-compatible CPU. It was built for the Vera Rubin GPU platform but is also sold as a standalone CPU server. TNW notes Nvidia unveiled it in March as a CPU built for AI agents, with 88 cores.

Nvidia has told investors it expects at least $20 billion from Vera chip sales this fiscal year. That is its opening bid in a market it values at roughly $200 billion.

Nvidia expects at least $20 billion in Vera CPU sales this fiscal year, against a market it values at roughly $200 billion

Source: The Economic Times (Reuters), Yahoo Finance

Modal is not alone. Nvidia has said OpenAI, Anthropic and Oracle plan to use the chips, along with AI startups such as Perplexity. TNW reports that Perplexity saw 1.5 times faster coding tasks on Vera.

For Intel and AMD, this is the uncomfortable part. Their data centre CPU business has long been a two-company market, and a new rival now has a growing list of AI customers.

Intel and AMD are not out yet

Modal Chief Executive Erik Bernhardsson was careful to say Modal is not abandoning its existing suppliers. The company will run Nvidia chips alongside Intel and AMD in a mixed fleet.

He also noted that the tests compared Vera against Intel and AMD chips already on the market. Both rivals are developing newer models that could narrow the gap.

"We'll test them when they're available," Bernhardsson said, adding that a mixed setup means Modal "can always deploy whatever hardware performs best."

There is a real cost to the switch, too. Modal has a long history of running only x86 machines. Supporting Arm means deep changes to its infrastructure and its image build system.

What this means for Indian developers

If you are a student, fresher or working developer in India building with AI agents, three things stand out.

Arm is going mainstream in servers. Phones and many laptops already use Arm designs. If AI cloud servers follow, your Docker images and native packages will increasingly need Arm builds as well as x86 ones. Learning multi-architecture builds is a cheap skill to pick up now.

Agent infrastructure is a hot job area. Modal's revenue growth of more than 15 times year-over-year shows how quickly demand for running AI agents is rising. Skills like containers, sandboxing, scheduling and performance testing are in demand.

Benchmark like Modal did. The lesson from Modal's tests is that a chip that looks only slightly faster on single tasks can behave very differently under heavy load. When you pick cloud instances for your own projects, test with realistic, concurrent workloads, not one job at a time.

FAQs

What is Nvidia Vera? Vera is Nvidia's Arm-compatible central processor. It was built for the Vera Rubin GPU platform and is also available as a standalone CPU server, aimed at AI agent workloads.

What is Modal? Modal is a San Francisco-based cloud computing startup that rents out computing power to AI developers. Its Sandboxes product gives AI agents isolated computers to run code.

Is Vera really twice as fast? Not on single tasks. Modal found up to double the throughput when the server was heavily loaded with many containers. On one task at a time, the gains were small and not consistent.

Is Modal dropping Intel and AMD? No. It will run a mixed fleet and place each job on the hardware that runs it best.

Can Indian developers use Vera on Modal today? Modal has not said when Vera-powered sandboxes will reach customers.

Sources

• The Economic Times: Nvidia wins fast-growing AI startup Modal for new central processors
• Modal: NVIDIA Vera CPUs double throughput for Modal Sandboxes
• TNW: Modal says Nvidia's Vera CPU doubles throughput for its AI sandboxes
• Yahoo Finance: Nvidia wins CPU business from Modal, previously exclusive to AMD and Intel