Cloud Computing: Public or Private? An Honest Decision Framework

"Public or private cloud?" is one of those questions that gets asked as if there is a single correct answer. There is not. After moving thousands of workloads in both directions over two decades — lifting on-prem systems into AWS and Azure, and just as often pulling runaway cloud bills back into private cloud on VMware, Proxmox, and Hyper-V — I can tell you the honest answer is almost always "both, and the split depends on the workload, not the slogan."

Here is the framework we actually use when a customer asks us to help them decide.

The three variables that actually matter

Ignore the marketing comparisons. When we scope a cloud decision, we evaluate three things per workload:

1. How variable is the demand? Does utilization swing by 10x over a week or a day, or is it a steady hum?
2. How predictable are the compliance and data residency requirements? Are you regulated, audited, or subject to sovereignty rules?
3. What is the actual three-year total cost, including the people who have to run it?

Every other factor — performance, availability, developer experience, security — is real but secondary. Those three variables account for most of the decision most of the time.

When public cloud wins

Public cloud is the right answer when demand is genuinely bursty or unpredictable. If you run a batch job once a month that needs 400 vCPUs for six hours and nothing the rest of the time, buying and racking 400 vCPUs worth of private infrastructure is an expensive mistake. AWS Spot, Azure Low Priority, and GCP preemptible instances exist for exactly this reason, and they are shockingly cheap when the workload can tolerate interruption.

Public cloud is also right when the service you need is a managed service you genuinely cannot replicate — Lambda for event-driven glue code, BigQuery for ad-hoc analytics on terabytes of data, Cognito or Entra ID for auth, S3 or Blob Storage as a durable object store. These are the kinds of services where the hyperscaler has invested thousands of engineer-years building something you would spend the next decade trying to match on-prem. Don't try. Use them.

Finally, public cloud wins when you need new capacity fast. Need 50 servers in a region you've never operated in because you just signed a customer there? That happens on a credit card, not a purchase order and a four-month colocation contract. Elasticity in time-to-provision is a real advantage and worth paying a premium for.

When private cloud wins

Private cloud — by which I mean VMware, Proxmox, Hyper-V, or OpenStack on hardware you own or lease in a colocation facility — wins on steady-state workloads. If your web tier runs at 30 to 70 percent CPU 24/7, if your databases have predictable IO patterns, if your file servers are full of data that grows at a predictable clip, the per-unit cost of running that on owned hardware is 3x to 5x cheaper than the equivalent public cloud compute over three years. I have run the numbers on this for a lot of customers and the math is not close.

Private cloud also wins when data egress costs would kill you. Public cloud providers have turned egress bandwidth into a tollbooth. If you have a workload that generates a lot of outbound traffic — media streaming, large file distribution, backup targets for external customers — your monthly bill can double because of bandwidth alone. Colocation bandwidth is a flat commit. Hyperscaler bandwidth is a meter running in the background.

And private cloud wins on sovereignty and predictability. When an auditor asks you to prove exactly where a particular dataset lives, which subcontractors have had access to it, and what the physical security chain looks like, the answers are much easier to document for infrastructure you own than for infrastructure shared with millions of other tenants.

The workloads that should never go to public cloud

A short list, because this one is underdiscussed:

• Anything where egress bandwidth is a large fraction of the workload. Run the math. It's usually worse than you expect.
• Long-running stable VDI farms with known seat counts. VDI in public cloud is expensive. VDI in private cloud on properly sized hardware is not. This is a space we have strong opinions about because we've been running VDI at scale since before it was called VDI.
• Applications with licensing that penalizes you for cloud deployment. Oracle, in particular, can turn a workable on-prem license into a financial nightmare in public cloud.
• Workloads where the regulator has not blessed the provider's region. This is shrinking but still matters for defense, healthcare, and certain financial workloads.

The workloads that should never stay on-prem

Equally short, equally underdiscussed:

• Any workload whose peak demand is more than 3x its average. The economics don't work. You're paying for headroom that sits idle.
• Disaster recovery targets for small shops. Cloud storage and compute-on-demand for DR beats a second physical site for almost everyone under a certain size.
• Globally distributed workloads you do not have the staff to run. If your team cannot staff a second region, let a hyperscaler staff it for you.
• Applications whose entire value depends on a managed service the hyperscaler sells. Don't try to replicate a managed ML platform on-prem.

The hybrid answer most customers end up with

In practice, the sensible architecture for most mid-market organizations is a hybrid that looks roughly like this: steady-state production workloads on private cloud in a colocation facility, with identity and endpoint management living in the public cloud (usually Microsoft's Entra ID and Intune because it's already where the M365 tenant lives), bursty and dev-test workloads in public cloud, backups and DR targets in public cloud object storage, and a handful of genuinely cloud-native workloads — event glue, data warehouse, SaaS integrations — running on whichever hyperscaler the rest of the stack points to.

This is boring, and it does not make for a good conference keynote. But over three years it delivers meaningfully better TCO than an all-in public cloud strategy for most workloads we see, and it keeps the benefits of public cloud where those benefits are real.

The question to actually ask

Stop asking "public or private." Start asking, workload by workload: is the demand variable enough that elasticity is worth paying for, are the compliance constraints loose enough that a shared platform is acceptable, and does the three-year math honestly come out ahead?

Those three questions, asked honestly, will give you the right architecture most of the time. And the architecture you end up with will not fit on a single marketing slide — which is, in my experience, how you know it's the right one.