Former Google Employee Cites U.S. Power and Data Center Limits for AI GPUs

A former Google employee said U.S. data-center capacity and power are limiting how quickly AI providers can deploy GPUs for customers that need low-latency service.

The employee said some companies have seen traffic multiply fivefold while available capacity remained inadequate. Customers with the strictest latency requirements need GPU facilities in the United States, the employee said. Deploying workloads in Singapore, Malaysia or Australia is technically possible, but many customers will not accept the added delay.

The constraint extends beyond accelerator supply. In the employee’s example, a model of about three terabytes would ideally load from a file system onto GPUs in roughly 10 minutes. That was an ideal target for that example, not a universal service requirement. The example highlights the need for storage and networking that can move model files to computing systems quickly.

The employee divided GPU cloud providers into three groups. Large cloud platforms were described as dependable but expensive, with contracts often requiring at least three years. The employee said customers are seeking Nvidia’s B300 systems, while some already view B200 systems as outdated.

Specialized GPU cloud providers, often called neoclouds, make up another group. The employee estimated there are 20 to 30 such operators and said the company where they worked placed about 80% to 90% of its capacity with them. That reliance, the employee said, requires more engineering work to coordinate providers.

The employee cited one customer deployment as an example of reliability trouble: 90 copies of a model used eight GPUs each, or 720 in total, and 20 GPUs went offline without warning early one morning. The employee speculated that a network problem or overheating in Nvidia Blackwell GPUs may have caused the disruption.

Commercial terms can also differ. U.S. neoclouds often require 20% to 30% upfront, with remaining fees paid monthly or quarterly, the employee said. Customers may negotiate discounts at the end of a contract for service losses, while Asia-Pacific operators typically do not require deposits.

The employee expects GPU availability to improve over the next one to two years as Nvidia expands shipments, but said power constraints may keep the number of GPUs running in U.S. data centers from rising much. The employee also pointed to Meta and xAI offering some idle GPU capacity for rent through their own cloud services, a model they expect more operators to adopt.