Fine Tune A+ Content with Amazon’s Manage Your Experiments

Many brands are trying to master the best copy, images and modules to use in A+ Content. Without being able to compare the data, how do you really know if you have mastered the best possible page to drive maximum results? While brands can do split testing, this is done very manually and not so easily given the time it takes Amazon to approve the new pages. Plus, there can be a number of additional factors that can influence or taint the results.

Amazon recently announced a new tool for sellers available in Seller Central in the US called Manage Your Experiments (MYE). This tool allows sellers to A/B test their A+ Content to compare two pieces of content to see which one provides the greater results. This helps sellers drive more sales and learn what works for future content. This feature is only available to sellers enrolled in the Brand Registry and is also available for all vendors in Vendor Central.

Editor's note: You can now run A/B tests on A+ Content, images, titles, bullet points, and descriptions. Also, A+ Premium Content is available for vendors.

Split testing A+ Content is great if you are undecided on what modules to use or whether you should be using lifestyle imagery versus a product image or whether you are focusing on the right content.

Getting Started With Amazon's Manage Your Experiments

Before getting started, you need to ensure that you have different pieces of A+ Content to make comparisons. This content must be approved by Amazon and applied to the same ASIN. You also need to ensure these ASINs are eligible, which is based on the level of traffic the pages get. For Amazon to be able to test the two pages it needs a sufficient amount of data and this is only possible with high traffic ASINs.

Once you have your two pieces of content approved, you need to create an experiment. To do this go to the Manage Your Experiments dashboard via this link and click on "Create a New Experiment". Creating an experiment is self explanatory and easy to do. You just need to be clear on the content you want to compare and the duration of the experiment.

Interpreting The Results

Once the experiment has finished you will be able to review the results and see which piece of content drove more sales. Results are updated once a week so you can get a sneak preview of the performance to date. A word of caution, don’t base any future decisions on the results until the experiment is complete. Results can always change over time.

The results are cumulative and focus on unit sales. Amazon then takes that data and decides which version they believe is the winner and how confident they are with their decision. For example they could have an 80% confidence rate, which means Amazon is 80% confident that the selected version of content will deliver greater results. How Amazon calculates their confidence rates is rather complex. If you want the finer details, you can go to the Experiment Results help section.

Some experiments may see a very low confidence rate or be inconclusive. This can happen if the variants in content were so small they didn’t impact customer behavior or there was not enough traffic to make a solid analysis. It can also occur if the changes made might not be of importance to the customer.

To help get a more accurate result, when dividing the two versions, Amazon customers are randomly assigned one view and will only see that content for the duration of the experiment as long as they are logged in under the same account.

Get the Best Out of Your Experiments

For Amazon to make a confident assessment, make sure there are clear differences in the A+ Content you're testing that could influence customer behavior. Also, while the results are exciting to see, make sure that you are not too reactive. Allow the test to run through the scheduled duration even if the results are heavily weighted to one version and consider running multiple experiments to get a wider set of data before making any concrete decisions and changes.