Your cart is empty
You may check out all the available products and buy some in the shop
Return to shop
Optimisation & Testing
Learn Faster. Improve Faster. Keep Moving.
Conversion optimization is not about filling a backlog with ideas months in advance. It is about learning what customers respond to, adapting quickly and turning evidence into better digital experiences.
PrettyDamnGreat combines research, UX, experimentation and rapid iteration to continuously improve customer journeys, funnels and commercial performance.
Some ideas win. Some fail. Both are useful when they help us understand the audience faster.
How I Approach Experimentation
A Backlog Should Be Useful. Not Impressive.
A six-month list of experiments can look strategic while becoming outdated after the first few tests.
If an early experiment disproves an important assumption about customer behaviour, many hypotheses built on that assumption may no longer be relevant.
That is why I usually plan roughly one to three months ahead rather than creating a rigid backlog for an entire year. Priorities remain fluid and are updated as new evidence appears.
The objective is not to push as many experiments live as possible. The objective is to understand the audience faster and invest more quickly in behaviour that proves valuable.
The Principle
Research → Test → Learn → Adapt
Keep the backlog short and flexible
Use previous tests to challenge future assumptions
Move winning behaviour forward quickly
Stop weak ideas instead of defending them
Keep exploring where customer behaviour changes
Experimentation Mindset
If Every Test Wins, You Are Probably Not Learning Enough.
Safe ideas often produce safe insights.
Best practices can be useful starting points, but continuously testing obvious interface tweaks teaches very little about what actually drives customer behaviour.
I prefer to understand how far an organization is willing to experiment, where the internal boundaries are and which bigger assumptions are worth challenging.
Losing experiments are not wasted experiments when the setup was sound. They remove assumptions, sharpen our understanding and influence what should be tested next.
The goal is to fail intelligently, learn quickly and adopt winning behaviour before the market, technology or customer changes again.
You do not build a stronger optimization program by avoiding failure.
You build one by learning faster from it.
The Optimization Cycle
From Evidence to Experiment to Decision.
01
Understand
Start with research, previous experiments, analytics,
behavioural data, customer feedback and business context.
02
Form the Hypothesis
Connect an observed problem or opportunity to a proposed
change and define what behaviour or business outcome we
expect to influence.
03
Prioritise
Balance evidence, expected impact, effort, risk, technical
feasibility and what we still need to learn about the audience.
04
Design & Build
Create the test variant with internal or external UX designers,
developers or, where useful, rapidly prototype the direction
directly before development begins.
05
Run & Measure
Launch the experiment with clear metrics, controlled traffic
allocation and an appropriate sample and runtime.
06
Decide & Learn
Move the winner forward, remove an unsuccessful variant or
translate an inconclusive result into the next question.
Experiment Quality
A Test Needs Enough Evidence. But It Should Not Run Forever.
Runtime, traffic and significance should be considered before the result becomes convenient.
01
Plan Before Launch
Estimate whether the available traffic and conversion volume can realistically detect the expected effect before starting the experiment.
02
Usually 2–4 Weeks
Where traffic allows it, I generally aim to keep experiments within roughly two to four weeks to reduce unnecessary exposure to changing campaigns, seasonality and other external influences.
03
Validate Important Wins
For high-value decisions, my preference is to repeat a successful experiment where possible before assuming the observed uplift represents durable behaviour.
For experiment planning and result evaluation I use established statistical tools, including the Speero A/B Test Calculator, alongside the reporting and statistical model of the experimentation platform being used.
Testing Velocity & Quality
More Tests Is Not the Goal. More Reliable Learning Is.
A healthy experimentation program combines enough velocity to keep learning with enough statistical discipline to make those learnings useful.
01
Design Tests That Can Reach an Answer
Before launch, I look at traffic, baseline conversion and the effect size we realistically need to detect. This helps determine whether the experiment has enough sample size to produce a meaningful result within an appropriate runtime.
That is more useful than launching an idea first and hoping statistical significance appears afterwards.
02
Keep New Questions Entering the Cycle
Depending on traffic, development capacity and test complexity, the goal is typically to develop and test multiple new hypotheses each month rather than wait for a large quarterly backlog to be completed.
Every result feeds directly into what gets researched, challenged or tested next.
The compounding value of CRO does not come from stacking wins.
It comes from stacking what you learn about your customers.
What Can Be Tested?
From Small Behavioural Changes to a Completely New Experience.
01
Focused A/B Tests
Content, hierarchy, navigation, search, filtering, calls to action, persuasion, product presentation and other targeted journey improvements.
02
Journey Experiments
Larger changes to funnels, checkout flows, information architecture, merchandising, segmentation or customer journeys.
03
Split URL Tests
When a full page or website redesign is being considered, the existing and redesigned experiences can be tested as separate URLs with controlled traffic before moving fully to the new experience.
A redesign does not have to become an all-or-nothing launch.
Where the technology and traffic allow it, the new experience can
earn its way to 100%.
Experimentation Technology
Platform-Agnostic. Evidence-Led.
The experimentation platform is infrastructure. The value comes from the questions we ask and what we do with the answers.
Experimentation
Extensive experience with platforms including VWO, AB Tasty and Optimizely, alongside newer or lighter-weight setups such as Varify.io and GrowthBook.
Research & Analytics
Experimentation is connected to analytics, behavioural research and customer feedback rather than managed as an isolated testing program.
Your Existing Stack
If the organization already has a suitable testing or analytics environment, the preference is generally to work with what is already available rather than replace technology unnecessarily.
Optimization Does Not Have a Finish Line.
Customers change. Competitors change. Technology changes. Traffic sources change. Expectations change. A journey that performs well today is not guaranteed to remain the strongest version indefinitely.
As long as there is sufficient traffic, access and commercial value, continued research and experimentation can keep revealing new ways to improve the experience.
Compounding Knowledge
The Longer You Learn, the Less You Need to Guess.
Every properly designed experiment adds another piece to the picture. Winning tests reveal behaviour worth building on. Losing tests remove assumptions. Inconclusive tests tell us where the evidence is still weak.
Over time, that accumulated knowledge can make prioritisation sharper, future hypotheses stronger and optimization increasingly focused on what customers have actually shown us rather than what the organization assumes.
6/10
Experiments Win
Across PrettyDamnGreat experimentation work, approximately six out of ten experiments have produced a winning variant.
8/10
Reach Statistical Significance
Around eight out of ten tests reach statistical significance, supported by checking expected sample requirements before launch rather than simply waiting for a result afterwards.
Optimisation & Testing FAQ
Before We Start Experimenting.
How far ahead do you plan experiments?
Usually around one to three months. I prefer a shorter, flexible backlog because new experiment results can change the assumptions behind tests planned further ahead.
How long should an A/B test run?
It depends on traffic, conversions, detectable effect and the experiment itself. Where the required sample can be reached, I generally prefer roughly two to four weeks rather than allowing experiments to run indefinitely.
Do you only test small UX changes?
No. Experiments can range from focused content or interface changes to major customer journey changes and full page or website redesigns using split URL testing.
What happens after a winning test?
The winning behaviour can be implemented or rolled out, while the insight becomes input for future optimization. For strategically important tests, repeating the experiment can provide additional confidence that the result is durable.
Are losing tests bad?
Not necessarily. A well-designed losing experiment can be extremely valuable because it disproves an assumption and helps determine what should be explored next.
Do we need an experimentation platform already?
Not necessarily. The appropriate setup depends on traffic, technical environment, budget and experiment complexity. If suitable tooling already exists, I can usually work within the client's existing stack.
Keep Learning
The Fastest Route to Better Performance Is Better Evidence.
If you have traffic, questions and room to improve, we can turn those questions into experiments and those experiments into a better understanding of your customers.
Discuss an optimization program → Start with Research & Audits →