Are You Guessing - Or Testing?

By Robert F. Abbott
Posted Thursday, October 28, 2004

How to use the Internet to test your marketing tactics. It's quick, easy, and inexpensive to try out your ideas.
Having trouble deciding which slogan to use on your Web page? Or what to call your newsletter? Or what to charge?

Well, you’re on the Web now, and that means you can do quick, easy, and inexpensive testing to get answers. I’ve done it a number of times and it’s been a useful experience each time. Here are three examples: Change and Measure; A Banner Example; and Online Questionnaire.

Change and Measure A key objective for one of my websites is to get visitors to subscribe to my newsletter, Abbott’s Communication Letter. When I started working to improve the ratio, the conversion rate was about 9%; that is, about one in 11 visitors subscribed.

In a bid to improve that ratio, I added a strong ‘Because’ section at the top of the index page, explicitly listing the benefits of subscribing. Again, I measured responses, getting several hundred of them before drawing any conclusions. And what did I find? Adding the ‘Because’ section improved the ratio, from 9% to 15%; now one in 6.6 visitors were subscribing.

To do this kind of testing, vary just one detail at a time, measure the results (with between 100 and 400 responses), and if the change worked, make it permanent and start over to test another change.

A Banner Example Here’s another challenge that seemed natural for testing: Which of two slogans should I use on the front cover of my book, A Manager’s Guide to Newsletters: Communicating for Results?

Solution: I created banner ads for each slogan, and tested them with the help of Mike Scanlin of Four Corners Effective Banners. Each banner got 3,000 impressions and when the test was done I had my answers.

To test with banners, start with a control banner and change just one thing on a second banner. Then review the click-through statistics to find out which works best. Of course, you can then take the best one and start testing it.

Online Questionnaire Here’s another type of testing: an online pricing questionnaire. I wanted to sell more copies of the electronic version of my book and wondered what the optimal price might be.

At the same time, Internet entrepreneur Ken Evoy developed a way to test prices using the power of the Internet. His online surveys use the Internet to ‘gather, analyze and deliver the data' at (http://myps.sitesell.com/acl.html) .

In this test, visitors to my website were given information about the book (but not the price) and asked to complete a quick, six-question survey. In exchange, they got a free report and a big discount if they bought the book.

When I had received enough responses to the survey, I was able to see clearly the optimum price and the effects that different prices would likely have on sales and profits.

How to Test If you try to test a lot of concepts at once it gets complicated. Believe me, I’ve tried. So change just one detail at a time, measure the results, compare with the original, keep or reject the change, and try another.

How many responses should you get? Try for 400 responses if you want to assess the opinions of more than 10,000 people, even millions of people. Does 400 seem too few to measure what millions of people might do? Well, assuming your test is reasonably well designed and you’re getting a fairly representative sample of the population, 400 provides a commonly accepted level of accuracy. For example, political pollsters can accurately forecast the voting intentions of literally millions of people after talking to just 400 of them.

Can you get by with fewer than 400 responses? Sure. You won’t have as much confidence in the results you get, and that’s okay if the stakes aren’t high or if you’re surveying a smaller population.

The important thing is that you test, not guess, because testing is the foundation of a successful initiative. And, using the power of the Internet makes it quicker, easier, and cheaper than ever before to test. That’s a deal you don’t want to pass up.

About the Author
Robert F. Abbott, writes and publishes a free newsletter which delivers communication ideas to your inbox every month. .