The Removal Test — Adasight

Part One

The technique, stated plainly.

Before you spend another week optimizing something — a page, a feature, an email, a whole workflow — ask a sharper question first.

"Has anyone questioned the assumption that customers actually care about this?"

Dr. Simon Jackson · Cherto

The technique

Test removing it — not improving it.

If a team has been grinding on something for a while with little movement, the fastest diagnostic isn't another round of tweaks. It's running an experiment that removes the thing entirely and measuring what happens. The result — whichever direction it goes — tells you something a dozen more tweaks wouldn't have.

Part Two

Why it works.

Every feature, page, and process sits on top of an assumption nobody's actually tested — usually some version of "customers need this." Most optimization work happens without ever checking if that root assumption is true.

The mindset shift

You're not designing experiments to produce winners.

You're designing experiments that test the hypothesis. A clear negative result — engagement drops when you remove it — tells you the thing was valuable, and spending more time on it is worth it. A clear positive or neutral result tells you the opposite: stop investing, redirect that time elsewhere. Either way, you've replaced a guess with evidence.

→

Negative result (metrics drop when removed): the thing was doing real work — now you know continued investment is justified.

→

Neutral or positive result (nothing changes, or things improve): the assumption was wrong — redirect the time and effort somewhere it'll actually matter.

Part Three

Two real results from removing things.

Both outcomes below count as wins — not because both removals stuck, but because both tests replaced a guess with a decisive answer.

Result: keep it

Canva's rarely-used templates

Usage data showed most templates were opened rarely. Engineering wanted to cut them to reduce maintenance load — a reasonable-sounding assumption. The team ran the removal test anyway. Engagement dropped significantly. It turned out the unused templates still gave customers contrast, making it easier to decide what they *did* want. The assumption ("unused = unnecessary") was wrong, and the test proved it before the cut happened.

Result: cut it

Booking.com's stuck page section

A team spent nearly a year optimizing one part of the booking flow — copy, design, layout, even the machine learning behind it. Nothing moved the needle. Someone finally asked if customers cared about that section at all. They ran the removal test. It was the biggest win the team saw all year — the section had been quietly costing effort for zero return.

→

If a year of tweaking hasn't moved a metric, the next test should be removal, not another tweak.

Part Four

Using it as a pushback tool.

The removal test isn't only for things that have already stalled. It's most valuable earlier — as a way to slow a team down before they sink months into an ambitious, complicated idea built on an unproven assumption.

The classic trigger: a team proposes something complex — personalization is the textbook example. Today they have one static version of something. Tomorrow they want to build machine-learning-driven personalization at scale. Before any of that gets built, the sharper question is simple: "Have you tried just turning it off — or not sending it at all — first?"

How to apply this

Slow the team down before the build starts.

When a team pitches a complicated solution, ask what happens if you remove or simplify the current version first — before any of the complex build begins.

Make the cost of skipping this concrete: name the people, time, and money the ambitious version will burn if the underlying assumption turns out to be wrong.

If full removal feels too risky to propose, suggest a smaller step instead — two segments instead of full personalization, one milestone instead of the whole vision.

Frame it as respect for the team's time, not doubt in their idea — you're trying to protect them from months of work on an unvalidated premise, not block the idea itself.

Adasight x Cherto webinar: 5 Lessons Learned from 1000s of Experiments

Watch The Full Session On YouTube

Watch 5 Lessons Learned From 1000s of Experiments

Gregor Spielmann (Adasight) × Dr. Simon Jackson (Cherto)

This technique is one part of a longer conversation — including why teams should test everything, and the metric bucket framework for catching cannibalization.

Watch on YouTube →

Next Step

One technique is one piece. Here's how to see the whole picture.

Adasight's Experimentation Gap Analysis is a structured look across the four places programs typically get stuck — with a clear roadmap for closing what's holding yours back.

Data

Can your team even get to the data needed to form a hypothesis?

Insights

Do results turn into a clear "why" — or end in a dashboard?

Experimentation Practice

Is there a clear success metric and process behind every test?

Is your experimentation data structured enough to actually feed AI?

See the four gaps →

What the audit delivers:

A current-state assessment across process, tooling, and skills.

The specific gaps keeping tests from producing trustworthy results.

A prioritized roadmap ranked by impact and effort — not a generic checklist.