Conducting an A/B test

In creating an A/B test, we need to decide several things, and then we need to put our plan into action. We'll walk through those decisions here and create a simple set of web pages that will test the aspects of design that we are interested in changing, based upon the behavior of the user.

Before we start building stuff, though, we need to think through our experiment and what we'll need to build.

Planning the experiment

For this chapter, we're going to pretend that we have a website to sell widgets (or rather, looking at the Widgets! website).

The web page in this screenshot is the control page. Currently, we're getting 24 percent click-through on it from the Learn more! button.

We're interested in the text of the button. If it read Order now! instead of Learn more!, it might generate more click-through. (Of course, actually explaining what the product is and what problems it solves might be more effective, but one can't have everything.) This will be the test page, and...