How to Deploy Artificial Intelligence in the Public Cloud, Data Centre and Edge

High performance computing: “We see a lot of adoption of GPUs in the high performance computing (HPC) space,” says Traves. “That may mean putting GPUs in existing HPC nodes to allow for additional computational capabilities. It may be adding dedicated GPU nodes into an existing HPC cluster to run those types of workloads.”

Data science: In data science and machine learning, there’s something called RAPIDS, which NVIDIA brought to market. It helps accelerate data science and machine learning workflows in ways that weren’t possible a few years ago.

Cloud computing: You can get instances with GPUs on demand in the cloud (pending availability). “If they are available, you can run your workloads on them as needed, and release that expensive asset when you don’t need it,” says Traves. “You don’t have to dedicate money into infrastructure; you can simply make it available on demand in one of the public clouds, assuming your data can be there.”

Data analytics: GPU accelerated analytics and interactive visualization provide deeper insights, dynamic correlation and predictive outcomes much faster.

“The first thing is understanding where is your data and how much data do you have,” says Traves. “Data tends to be a bit of an anchor sometimes. Your computational throughput needs to be in proximity to where your data is for it to be efficient.”

“When you’re dealing with data, and you’re trying to provide insight into it, we also have to look at the security issues around it. In many cases, customers may not be able to or may choose not to move data to a public cloud because they’re concerned about what could potentially be exposed as a result. Understanding what needs to change within the data becomes a concern.”

High availability of the data is also a concern. “If it goes down, it doesn’t really hurt anything from a production standpoint, but it means there are dozens of researchers and data scientists who don’t have the infrastructure they need to run workloads on and potentially the jobs that were running would have to re-run and potentially lose days of work,” says Traves. Having your workloads span multiple locations and data centres is an important design consideration, while leveraging the public cloud for customer-facing applications would also be a good idea.

And don’t forget about power and cooling. “As these systems get larger and larger and your requirements grow, you quickly find that there are challenges with feeding enough power and cooling into GPU servers, a traditional data room or even a data centre,” says Traves. Even many large organizations can’t cope with the amount of power and cooling required from these systems and are looking to purpose-built colocation facilities.

How much data do you have? Can you put in on a USB key or a hard drive? Is it in a data mart or data warehouse on premises, or does it live securely in the cloud?

“If you’re going to continually do training, then as your data changes, you want to re-run your training to get a better model,” says Traves. “So the more often that data changes, the more continuous that process needs to be.”

Are you sharing raw data across the organization, as well as the results? You need access to those data sets in a realistic timeframe as well. “Where are we going to process this data, where is it coming from? Is it in a data centre, in the public cloud? Am I generating it out of a vehicle in an IoT device? And how much processing do I want to do at the edge, as opposed to shipping that data back to a central location?” asks Traves.

With autonomous vehicles, “I may need to make a decision now, or the car crashes. So having some context of that ability to collect new data and communicate with peers, and then being able to transfer that data back to a central location makes a lot more sense.”

“The ability to leverage the tools that your public cloud provider has is really important,” says Traves. “The data needs to be there for that to actually happen. Once it’s there, and you’re happy with it from a security perspective, being able to scale your needs up and down as demand dictates is very powerful. You don’t need to invest in equipment; you can invest in instances when you need them.”

“And that’s really powerful for inferencing. It’s a little less powerful in the case of training, because typically you want those assets running at 100 percent for days on end. And anybody who’s running 100 percent utilization in a cloud instance knows that costs a lot of money. That’s where we may look to run things in data centres.”

Data centres are also a good place for data you have to protect to meet governance requirements. Or if you’re dealing with petabytes of data, it might make sense to build data centres.

“It makes even more sense to look at a colocation facility as an extension of your data centre,” says Traves. “A colocation, you can treat as your own data centre, but it bypasses the power/cooling issue, as it becomes an issue for them to deal with. It gives you the high-speed on ramp to the public cloud, which means you can leverage their tools and services to run against your data, but you can run your workloads locally on those systems at 100 percent utilization and not have to worry about getting dinged by the public cloud provider.”

“At the edge, real-time decision making is really important,” says Traves. “You may be in a disconnected state, so you need to deal with a model you have, along with whatever contextual data you’re gathering. You typically manage those as a fleet and you want the fleet to be able to communicate with each other, exchange data and leverage what’s happening close to the actual sensors themselves, without having to go back to HQ somewhere.”

In any case, most AI implementations are going to provide data across these three locations, so your data won’t live in just one place.