Google unveils new Cloud TPU VMs to simplify training ML models

Early adopters are using the preview version in everything from healthcare analytics to quantum chemistry.

This week Google announced a new infrastructure option for its cloud platform. The new feature will enable enterprises to more efficiently provision instances with Tensor Processing Units, Google’s internally developed AI chips.

Google’ Cloud customers could already provision instances with TPUs before the announcement. But those instances didn’t run in the same physical server enclosure as the TPUs. Rather, they connected to them remotely via a network link. This remote connection slowed down processing, as applications had to send data to a TPU over the network and then wait for the results to return.

This network-based slowdown is removed with the new offering. The new instances attach directly to Google’s TPU AI chips. This direct connection obviates performance slowdowns and latency due to the vagaries of the network.

How the new TPU’s simplicity and flexibility lead to cost savings

Alexander Spiridonov Product Manager, Google AI, detailed the new offering in a blog post. “Now you can write and debug an ML model line-by-line using a single TPU VM,” he said. Users can then scale it up on a Cloud TPU Pod slice to take advantage of the super-fast TPU interconnect, he added.

“You have root access to every TPU VM you create, so you can install and run any code you wish in a tight loop with your TPU accelerators. You can use local storage, execute custom code in your input pipelines, and more easily integrate Cloud TPUs into your research and production workflows.”

Google’s new Cloud TPU system architecture is simpler and more flexible that those previous, says Spiridonov. In addition to major usability benefits, users can also achieve performance gains because their code no longer needs to make round trips across the datacenter network to reach the TPUs.

He adds that customers may also see significant cost savings. “If you previously needed a fleet of powerful Compute Engine VMs to feed data to remote hosts in a Cloud TPU Pod slice, you can now run that data processing directly on the Cloud TPU hosts and eliminate the need for the additional Compute Engine VMs,” he explained.

Tip: What is Google Anthos? Is this the modern cloud infrastructure you’re looking for?