Adwords has released a new feature called “Adwords Experiments”. Most people should have access in the coming weeks. I have been part of the Beta testing. I will offer some advice after some of my successful (and not so successful!) experiments.
What is Adwords Experiments?
Due to Adwords’ concepts of “account history”, you are rewarded for past performance. Thus, you must think twice about every change you make to the account as your changes have a permanent effect!
Have you ever had a “brilliant idea” that you couldn’t get your boss to sign off on? Adwords Experiments is great for cases like these. Make a change to a campaign, check it in a month, and then celebrate your great results. If your idea was horrible, “undo” the results without harming your account history!
Adwords Experiments allows you to make changes to your account that you can undo if you don’t like the results. Usually, we make changes to our accounts and then look in the account history (or our memory) to keep track of progress. This is adequate 95% of the time. For the other times, we can use Adwords Experiments.
How Does Adwords Experiments Work?
Adwords will test the control group vs your new experimental group and tell you the winner. If you want to keep the changes at the end of the experiment you just click a button and your changes are implemented. If not, your changes go back to the way they were before the experiment.
If you make a dumb decision, it will still have a negative effect on the company. It might negatively effect sales or CTR or whatever during the experiment. Using Adwords Experiments you won’t be permanently punished for trying to improve your account.
This is all that a marketer wants.
When Should I Use Adwords Experiments?
If you are making a risky change to the account, make an experiment. Testing new copy is not risky. Raising the price of a high performing keyword is not risky. Adding a bunch of new negative keywords to a campaign makes me nervous, so I would set up an experiment. Splitting up a high-performing adgroup into two makes me nervous, so I would set up an experiment.
The Positives of Adwords Experiments:
1) You can make risky changes to the account without fear of permanently damaging your account history.
2) One great feature of Adwords Experiments is that it automatically calculates statistical significance. I wonder why this has to be limited to experiments? It is such a useful feature! It would be great in many of Google’s products.
3) Certain variables can only be properly tested by using the Adwords Experiment feature. (One feature that comes to mind is determining ROI-based-on-keyword-position. Previously our testing has had to simulate this effect, or make inductions based upon time-shifting results. That is fine, and we are able to produce actionable-data, but this new way is a little better).
The Negatives- Unfortunately there are a lot of negatives. Perhaps some of them will be ironed out.
1) You can only run one experiment at a time at the campaign level. E-commerce sites tend to have tons of campaigns, so this will not be an issue for you if you are E-commerce. Lead-generation sites tend to have fewer campaigns. You may have set up your account so that you have a ton of adgroups and few campaigns. If this is the case you will be limited to one experiment at a time. (I would not recommend changing your account structure simply to get around this).
2) The interface is complex: make sure your first experiment is simple. Until you get the hang of the interface, don’t make complex experiments. Try messing around with numbers rather than keywords the first time out.
3) Remember that any changes make will have ramifications campaign-wide. A small change to one adgroup may effect another adgroup. Make sure you look for those. Some of the effects to other adgroups are non-intuitive, but that may just be an indication that I designed a poor experiment.
4) You can’t save your experiment history. If you want to save your results you will have to export them and keep a log.
-Make BOLD changes to your account so that your results will be statistically significant. Bid much higher or much lower.
-Make your first experiment simple.
-Most of your testing can still be done without using this feature. You can still experiment in Adwords without using “Adwords Experiments”.