Guides
Part of Campaign management tools: methods, tools and useful context
Campaign management tools case studies: evidence and results
Campaign management tools case studies work only when the outcome is falsifiable. Four composite situations, what each one turned on, and how to read a vendor's.
Most published case studies about campaign software are unreadable as evidence. They report that a team saved time, they attach a percentage nobody can check, and they never mention the accounts where the same rollout was abandoned. The four situations below are composites, written to show the decision path rather than to prove a result. None of them describes a real agency, and no figures are given, because a figure you cannot verify is worse than no figure at all.
What to take away
- A case study is useful only when it names the constraint. Without one it is a testimonial with a chart on it.
- The interesting cases are the ones where the tool was the right purchase and the rollout still failed.
- Ask any vendor case study one question: what would have had to happen for this to be written up as a failure?
What has to be in a case for it to be worth reading
Four elements, and the absence of any one of them makes the story unusable.
| Element | What it looks like when present | What its absence hides |
|---|---|---|
| The constraint | The team could not do X because of Y | Whether the result would transfer to you at all |
| The counterfactual | What they were doing before, in detail | Whether the tool caused the improvement or the rewrite of the process did |
| The cost that was paid | Migration, training, the month of double entry | The real price, which is never the subscription |
| The part that did not work | A workflow they kept in a spreadsheet | That the account has been edited into a success |
Every case below turns on the same underlying question that room turns on: where the authoritative copy of a fact lives, and who is allowed to change it.
The migration that succeeded and the reporting that broke
A team moved campaign records into a new system, kept the old spreadsheet running for a quarter as a safety net, and then found their quarterly client reports no longer matched. Nothing had been lost. Two systems had simply been counting a campaign as live on different dates, and the report was pulling from the older one.
What it turned on: nobody had written down which system owned the fact "this campaign started". The fix was a paragraph, not a feature. Deciding that ownership per field, before cutover rather than after, is the substance of a rollout plan.
The product that fit the work and not the week
An agency chose a tool that matched its process closely and still saw usage collapse within two months. The work had not changed. What had changed was that logging it now took a separate step at a moment when nobody had a spare minute, so people did it in batches on Fridays, from memory.
What it turned on: the tool asked for the record at the wrong point in the day. A system that is only true once a week is not a single source of truth, and a team will quietly go back to whatever is faster. This is the failure mode that no demo produces, because in a demo the data entry is the task.
The seat count that grew faster than the work
A team bought seats for the people running campaigns, then added client reviewers, then freelancers, then a second reviewer per account because the first one went on leave. Nobody made a bad decision. Each addition was obviously reasonable, and at renewal the total was unrecognizable.
What it turned on: the meter and the workload grew on different curves. The habit that prevents it is boring, which is why it is rare: someone reviews the seat list monthly and removes people who have not signed in. The way meters are built into these quotes, and which ones grow when you succeed, is covered in the guide to tool pricing structures.
The client who asked for the account back
A retainer ended and the client asked for everything: briefs, approvals, contracts, message history. What came out was a set of exports for the objects the vendor models, and nothing for the conversations, which is where most of the reasoning lived.
What it turned on: nobody had tested the export until they needed it. The obligation was in the client contract and the capability was never in the tool. Both parties were acting in good faith and the record still could not be handed over.
Why the published ones read so well
Case studies are collected from customers who stayed. The accounts that churned are not asked for a quote, and the rollouts that were quietly abandoned never reach the marketing team at all. What you read is filtered by survivorship bias before anyone edits a word of it, which is why the pattern across vendor case studies is so consistent and so uninformative.
Three questions turn one back into evidence. Which of your customers of this size did not renew, and what was different about them? What did the first ninety days actually cost the team in hours? Which part of the workflow is still outside the product? A vendor who can answer all three is telling you something. One who answers none is showing you a brochure with a customer's name on it.
The habit of writing your own version of these, honestly, is worth more than reading anyone else's. Keep a short note after each rollout: what you expected, what happened, what you would test earlier next time. The general discipline of writing a purchase decision as a claim that could turn out wrong is set out in the guide to comparing suppliers, and the underlying chain of jobs these tools sit across is mapped in the guide to campaign management software.
Common questions
Are composite cases not just made up?
They are illustrations, and they are labeled as such. The alternative is inventing a named agency and a percentage, which would be worse: a fabricated specific reads as evidence and is not.
How do we tell a real case study from a written one?
Look for a number that is against the vendor's interest. Real accounts contain a cost, a delay or a workaround. Written ones contain only gains.
Should we ask for a reference from a similar agency?
Yes, and ask the vendor to introduce you to one that left. The refusal is informative, and the occasional agreement is the most useful call you will have.
What should we record about our own rollout?
The date, the thing you expected to improve, and the measure you would have accepted as proof. Write it before the tool arrives, because afterwards you will remember having expected whatever happened.