Evaluating a Managed Cloud Provider: Four Things That Predict Success

I have sat on both sides of the managed cloud provider evaluation. I have been the customer trying to pick a partner, and for 23 years I have been the provider answering RFP questions from customers who were trying to pick one. After all that, I am confident about one thing: the questions most evaluation frameworks focus on are not the questions that predict whether the engagement will work.

People ask about certifications, customer logos, years in business, and data center locations. Those are fine table-stakes filters, but they don't tell you what the next three years will feel like. The four signals below do.

One: How They Answer Ambiguous Questions

Send the provider a question that doesn't have a single right answer. Something like "we have a two-node SQL Server Always On cluster running a line-of-business app that has a hard cutover window of thirty minutes once a quarter. Should we move it to your managed private cloud or keep it where it is?"

There is no right answer to that question without more information. A good provider will tell you that. They will ask about RPO and RTO, licensing model, storage IOPS profile, network path to the application users, and whether the app has been tested against a storage migration before. A bad provider will jump straight to "we can handle that, let us send over a migration quote."

The reason this matters is that your environment is going to throw ambiguous problems at the provider constantly. That is the nature of operations work. If they cannot sit with ambiguity during the sales process, they are going to fake confidence in production, and faking confidence in production is how outages happen. You want a provider whose instinct is to ask more questions, not one whose instinct is to answer faster.

The other thing this test reveals is the provider's internal seniority. A shop with experienced engineers on the front line will give you the question-asking answer. A shop with junior engineers and a sales team reading from a script will give you the confident answer. You are about to hire operators. Hire the shop with the operators.

Two: Who Owns the Relationship Internally

Every managed provider has somebody whose job is to keep you happy. The question is whether that person has actual authority or is just an inbox.

A provider that assigns you a technical account manager with no engineering authority is selling you a ticketing experience. When something goes wrong, the TAM will log a ticket, an engineer will pick it up when their queue rotates there, and the TAM will update you with whatever the engineer said. The TAM is a translator, not a decision maker. The time from "something is wrong" to "somebody empowered is working on it" can be measured in hours.

A provider that assigns you a principal or senior engineer as your relationship owner is selling you something different. That person knows your environment, has escalation authority, can pull other engineers in directly, and does not need to wait for a queue to rotate. When something breaks, they are already on it.

Ask during evaluation: "Who is my day-one point of contact and what is their title? How many other customers do they support? Can I talk to two of those customers before we sign?" The answers tell you exactly what the relationship is going to feel like when you need it most. If the relationship owner is supporting thirty accounts, you are getting thirty-way attention. If they are supporting five, you are getting real attention.

Three: The Shape of the Runbook

Ask the provider to show you an anonymized runbook for a scenario they handle often. Backup failure recovery, failed patch rollback, certificate renewal, that kind of thing. How detailed is it? Is it updated with dates? Does it reference specific tooling and specific commands, or does it describe steps in vague narrative?

A mature provider has runbooks that read like Swiss train schedules. Step numbers, expected outputs, decision points, escalation thresholds. Every runbook has a version history because the environment keeps changing and the runbook keeps getting updated. New engineers onboard against the runbook and can handle a real incident within weeks because the runbook does not assume prior tribal knowledge.

A less mature provider has runbooks that were written once and never revisited, or worse, runbooks that live in somebody's head. When you ask to see one you will get a PDF that looks like it was formatted in 2018, or you will get a verbal walkthrough that changes depending on which engineer you ask.

This matters because your environment will inherit the runbook culture of your provider. If they write runbooks, you will get runbooks. If they rely on tribal knowledge, you will inherit tribal knowledge, and tribal knowledge evaporates the moment the engineer who has it takes a vacation or finds another job.

Four: Honesty About What They Don't Do

Every provider claims to do everything. Almost no provider actually does. The ones worth working with will tell you what they don't do and either introduce you to a partner who does it or help you scope the work so it stays in their lane.

This is the single most underrated signal in managed cloud evaluation. A provider who says "we run your VMware environment, but we don't write your custom PowerShell automation — that's something you'll either keep in-house or we'll bring in a scripting partner for" is telling you the truth about the boundary of their skill set. A provider who says "yes, we handle PowerShell, custom automation, database administration, application tuning, identity federation, SIEM engineering, and penetration testing" is telling you that whoever shows up will be learning on your budget.

Ask the question directly: "What should we not hire you for?" Watch the reaction. The reaction is more useful than the answer. A confident, competent provider will have a ready list of things they steer customers away from. A provider who squirms and tries to claim everything is the provider who will absorb any scope you hand them, do a mediocre job on most of it, and blow out the budget.

What the Four Signals Tell You Together

These four signals are really asking the same underlying question from different angles: is this a provider who operates with intellectual honesty, or is this a provider who operates with sales enthusiasm? The technology piece is table stakes — if a provider cannot run VMware or Azure or a backup platform in 2024, they will not survive in the market. What separates the providers who are worth a three-year contract from the ones who are not is whether the humans on the other side of the table are telling you the truth about what the engagement is going to look like.

You will notice I did not mention price. Price matters, but price is almost always a lagging indicator of the four signals above. A provider with mature runbooks, experienced front-line engineers, empowered relationship owners, and an honest sense of their own limits is not the cheapest option in any RFP. They are not trying to be. They are also not going to surprise you in month nine with a scope change or an incident you have to explain to your board. The cheapest provider in the stack almost always becomes the most expensive provider in the stack once the hidden costs get tallied — second-guessed decisions, rework on bad architecture, time your internal team spends babysitting the provider instead of doing their own jobs.

Four Takeaways

1. Test how a provider answers ambiguous questions. Ambiguity is the shape of operations work. Hire a provider who is comfortable asking more questions before committing to an answer.
2. Insist on an empowered relationship owner, not a translator. You want the person who knows your environment to have authority to fix it when it breaks, not to forward your ticket.
3. Read a real runbook before you sign. Runbook culture inside the provider becomes runbook culture in your environment. Mature runbooks are a leading indicator of mature operations.
4. Ask what they don't do. The provider who gives you an honest list is telling you the truth about everything else. The provider who claims to do everything is telling you the truth about nothing.