As healthcare organizations become more experienced with using artificial intelligence in their workflows, they are looking to optimize operations with the technology infrastructure that best meets their needs. For a growing number of health systems, this means running AI workloads in hybrid environments. In fact, IDC predicts that by 2028, 75% of enterprise AI workloads will be deployed on hybrid infrastructure.
According to Sana Gutierrez, senior manager of the data and artificial intelligence practice at CDW, the future of the data center is hybrid. This allows healthcare organizations to place workloads where they can best meet their needs, whether in the cloud, on-premises, or in a hybrid environment.
For example, the public cloud is a good option for AI workloads that may involve experimentation or need burst capacity, while a private cloud is well suited for workloads involving sensitive data. Similarly, on-premises clusters of GPUs can handle predictable, large-scale inferencing workloads.
One of the key benefits of hybrid infrastructure is that it allows healthcare IT teams to control costs and deal with GPU scarcity while maintaining data sovereignty and supporting demands for compliance with data regulations such as HIPAA and the European Union’s General Data Protection Regulation. When that data is kept close to processing capabilities, it can also minimize latency.
Mariano Carro, principal field solutions architect for Microsoft hybrid infrastructure at CDW, notes that whether an AI workload is running in the cloud or on-premises is going to depend on the organization’s specific needs for artificial intelligence. Most of the time, some resources in the cloud are needed for training and high-level compute, but once training is complete, workloads may be moved on-premises to improve performance or protect data privacy.
They often revisit where applications run based on changing cost structures, performance needs, and governance requirements. Eryn Brodsky, server and storage practice lead at CDW, says that workload placement is going to take into consideration things like latency as well as accessibility.
Related: Healthcare mergers surge in 2025
Organizations need to think about what AI outcomes they want to leverage the data, and they need to have quick access to it while ensuring proper access.
This leads to movement between environments over time. Many organizations begin their AI journeys in the cloud, drawn by its flexibility and access to large-scale compute, but as workloads stabilize and costs become clearer, some shift those workloads back into their own environments.
According to Gutierrez, the future of the data center is hybrid. “There are organizations that are going to decide to begin within a cloud environment, and that posture can completely change. They need to understand how they want to take workloads, whether they’re in the cloud, on-premises or in a neocloud, and put them in the right place to gain maximum benefit.”
As healthcare organizations look to build out infrastructure that can support AI into the future, they are strategizing around meeting specific business and clinical needs, where to place workloads and how to align their AI efforts with their desired outcomes.
As they handle these decisions, platforms that unify management across environments are becoming increasingly valuable. Microsoft’s Azure Local offers one such option, enabling organizations to build and scale AI infrastructure while maintaining flexibility in where workloads run. With centralized oversight, IT teams can more easily control resources, enforce policies, and ensure workloads are running in the most effective locations, much like health IT teams do with endpoint management.
Ultimately, the value of hybrid infrastructure comes down to how well it aligns with business and clinical goals. Healthcare organizations that take a thoughtful approach to workload placement, cost management, and performance optimization are better positioned to realize meaningful returns from their AI investments. When organizations get AI and accelerated compute right, there is a material benefit to the bottom line of any organization, according to Gutierrez.