The A/B test is not a creative tool

Creative Strategy

The A/B Test is Not a Creative Tool

Why the pursuit of “perfect” data is creating a world of safe, invisible mediocrity.

One marketing team in London ran 6,241 tests on a single font color last year. The team wanted to find the best shade of blue. The team tested light blue. The team tested dark blue. The team tested blue that looked like the sea. The team tested blue that looked like the sky.

6,241

Individual Tests on a Single Color

The industrialization of creative indecision through iterative testing.

The results came back. The results were clear. The users liked the blue that they had seen before. The users liked the safe blue. The team changed the color. The team felt proud. The team had data. The data was the truth. The truth was boring.

The Weight of the Wrench

I fixed a toilet at three in the morning. The water was on the floor. The water was cold. I used a wrench. The wrench was heavy. I turned the nut. The nut did not move. I turned the nut harder. The nut moved. The water stopped.

I went to the cemetery at dawn. I work at the cemetery. I am a groundskeeper. My name is Blake R.-M. I saw the sun. The sun was white. The sun was not a predator. The sun was just the sun. I had a shovel. The shovel was old. The handle was smooth. I pushed the shovel into the grass. The grass was wet. I turned the dirt.

The creative director sits in a chair. The chair is black. The chair has wheels. The director moves the chair. The director looks at a screen. The screen shows two pictures. One picture is a man. The man is jumping. The other picture is a man. The man is sitting.

The director likes the man jumping. The director thinks the jump is bold. The jump shows energy. The director wants the energy. The director waits. The director cannot choose. The director must run a test. The test takes two weeks. The test gathers numbers. The numbers are the boss.

🏃♂️

The Jumping Man

(The Risk)

🪑

The Sitting Man

(The Data)

The numbers come back in a spreadsheet. The spreadsheet is long. The spreadsheet says the sitting man is better. The sitting man got three percent more clicks. Three percent is a small number. The director looks at the sitting man. The sitting man is safe. The sitting man is quiet.

The director uses the sitting man. The director kills the jump. The jump is dead. The jump is in a grave. I dig graves every day. I know what goes inside them.

Testing the Invisible

Data measures the past. Data tells you what happened yesterday. Data tells you what people did. Data does not tell you what people will do. Data does not know about the leap. The leap is a choice. The leap is a risk. The leap is the thing that no one has seen.

You cannot test the thing that does not exist. You can only test the things that are here. The testing makes the things look the same. The testing makes the world flat. The world is not flat. The world has holes. I dig the holes.

The algorithm wants the sitting man. The algorithm likes the pattern. The algorithm sees the pattern and repeats the pattern. This is a loop. The loop is a circle. You walk in the circle. You think you are moving. You are just walking. You are not going anywhere. You are staying in the same place.

The place is safe. The place is mediocre. Mediocrity is a slow death. It is the death of the brand. It is the death of the voice. A brand needs a voice. A voice needs conviction. Conviction is a hard thing. Conviction is the nut that will not turn. You need a wrench. You need to pull. You might break the wrench. You might hurt your hand. You might fail. But you make a choice.

A choice is a human thing. A computer does not make a choice. A computer runs a calculation. A calculation is not a voice. A calculation is a noise. The director drinks coffee. The coffee is cold. There are crumbs on the desk. The crumbs are from a sandwich. The director clicks the mouse. The mouse makes a sound. It is a small sound.

The director looks at the YouTube channel. The channel has many videos. The videos all look the same. The thumbnails are all the same. The titles are all the same. The data said this was good. The data said this would work. The channel is growing slowly. The growth is a flat line. The line is not a mountain. The line is a floor.

Growth Visualization

DATA-OPTIMIZED (STAGNANT)

The shovel hits the dirt that the algorithm forgot to measure. The director needs a mountain. A mountain is a big risk. To build a mountain, you need people to see the first rock. You need the first rock to stay in place. People look at the rock. If the rock has no weight, people keep walking. They do not stop. They do not look up.

Creating Social Proof

The signal is social proof. The signal is the view count. If the view count is zero, the signal is weak. If the view count is high, the signal is strong. The strong signal gives the bold choice a chance. It gives the jumping man a stage.

You can use achat vues youtube to create the signal. The views are the foundation. The views are the weight. The views keep the rock from blowing away in the wind. The wind is the testing. The wind is the data. The views let the bold work stand still long enough to be seen.

When the work is seen, the testing changes. The testing is not about the blue button anymore. The testing is about the idea. But the idea must survive the first hour. The idea must survive the first day. Most ideas die in the first hour. They die because they have no momentum. They die because the data-driven crowd is waiting for someone else to look first.

I saw a headstone today. The headstone was from . The name was carved deep. The carver did not test the font. The carver did not ask for clicks. The carver had a hammer. The carver had a chisel. The carver made a mark. The mark is still there. The dirt is still there. The name is still there.

The carver had conviction. The carver did not have a spreadsheet. The director wants to be the carver. The director wants to make a mark. The director looks at the spreadsheet again. The spreadsheet is a wall. The wall is high. The director cannot see over the wall. The director stays behind the wall. The work is safe behind the wall. The work is invisible behind the wall.

The Valley of Risk

Optimization is a trap. It is a trap for the mind. You optimize the small thing. You lose the big thing. You optimize the thumbnail. You lose the story. You optimize the title. You lose the truth. You find the local maximum.

LOCAL MAXIMUM

THE MOUNTAIN (THE RISK)

The local maximum is a small hill. You are on the small hill. You look at the sky. You see the mountain. The mountain is far away. You cannot get to the mountain from the small hill. You have to go down. You have to go into the valley. The valley is dark. The valley is the risk. The valley is where the data fails.

In the valley, you have only your eyes. You have only your hands. You have your wrench and your shovel. You make a choice because you believe the choice is right. You do not make the choice because the computer said it was three percent better. You make the choice because it is your choice.

I went home. I washed my hands. The dirt went down the drain. The water was warm now. The toilet was fixed. The floor was dry. I sat in my chair. I did not look at a screen. I looked at the wall. The wall was white. The wall did not have a button. The wall did not have a metric. The wall was just a wall. I liked the wall.

The director will go to the office tomorrow. The director will see the new test results. The results will favor the safe option. The director will have a choice. The director can follow the numbers. The director can follow the ghost. Or the director can close the spreadsheet.

Choose the Jump

The director can look at the jumping man. The director can push the red button. The director can take the leap. The leap is the only thing that lasts. The safe choice fades. The safe choice is forgotten. The safe choice is a leaf in the wind.

The bold choice is a stone. The stone stays. The stone is heavy. You need to give the stone a place to sit. You need to give the stone the views it needs to be a landmark. Then the people will come. They will see the stone. They will remember the stone.

Push the Red Button

“They will not remember the blue font. They will not remember the sitting man. They will remember the man who jumped.”

The dirt is waiting. The cemetery is quiet. The data is loud. I prefer the quiet. I prefer the shovel. I prefer the work that stays in the ground long after the server is turned off. The director should think about the ground. The director should think about the mark.

The mark is all we have. The test is just a distraction. The test is a lie that tells you that you are safe. You are never safe. You are only alive or you are not. The jumping man is alive. The sitting man is a ghost.

Choose the jump. Change the blue to red. Throw the spreadsheet in the trash. Dig a hole for the mediocre. Put it in the ground. Cover it with dirt. Walk away. Do not look back. There is more work to do.