By Art Hall

My volume is shrinking, my response is eroding and my resources are dwindling with each passing year. How do I keep the test pipeline producing winners against the headwinds of all these economic pressures?”

All too often I hear marketers raise these concerns and ask this question. Every year marketers are asked to improve results with less resources. For direct mail marketers, the challenges are even more daunting with rising postal and paper costs reducing the number of tests performed each year.

Testing is the lifeblood of any successful direct marketing program. To produce the next winner, it may take up to 10 unsuccessful attempts: consuming valuable time, money and resources in the process. With rapidly changing consumer preferences, traditional A/B and even multivariate testing are becoming less viable solutions.

Follow the leader

Most marketers apply the Warren Buffett axiom when testing, “It’s good to learn from your mistakes. It’s better to learn from other people’s mistakes.” Instead of investing to uncover new control packages, marketers often take the path of least resistance by riding the coattails of their competitors.

But there is a better way to become the leader of the pack without using brute force and risking your career on a string of unsuccessful tests, which I discuss later on.

The key is to discover that breakout creative before your competitors. Yet testing for the breakout package is risky business when only two out of 10 tests are successful. Marketers error on the side of traditionalism by making conservative improvements to the control, which yields only incremental results.

Break from the past

Unfortunately, the success of the past has trapped many talented direct marketers in decades-old testing strategies. They are:

  • A/B testing, the most common strategy, involves changing one or more elements of a direct mail package and creating multiple versions for testing. For example, you can take the control package and change the outer envelope, along with the copy while keeping all the components inside the same;
  • Multivariate testing allows multiple elements to change at one time. For example, you might test three envelopes, two offers and two inserts all within the same direct mail campaign. Each element requires a test cell for each version in the test; and
  • Focus group testing is a form of qualitative research consisting of interviews in which a group of people are asked about their perceptions, opinions, beliefs and attitudes toward a product, service, concept, advertisement, idea or packaging.

Removing testing guesswork

Since traditional testing is difficult and costly, finding an inexpensive and reliable alternative seems next to impossible. When marketers are introduced to scientific testing that is up to 98% accurate without incurring production or postage costs, it sounds too good to be true. But the experience of marketers with it demonstrate its effectiveness.

Diagnostic testing allows many parts of a mail package to interact through a simulation to produce a hybrid solution that is more than or different from the combination of its parts.

Using conjoint analysis—the next generation of predictive analytics—a panel representing the target audience is asked to rate and rank a controlled number of direct mail components, e.g. envelope, offer and letter format. The beauty is that anything can be tested without expending valuable resources when using diagnostic testing. As one marketer said, “Instead of guessing at what makes my tests work, this solution will show which cannons are firing and which ones are not.”

For marketers, understanding exactly what and how different direct mail attributes increase lift means package development can be optimized for consumer preference in days rather than months at a fraction of the cost. The nirvana of direct mail has arrived. The days of subjective creative development is becoming a distant past. Direct mail “title testing,” where the person in the room with the highest title makes the final decision, is fading away.

Powerful results

The ability to run direct mail diagnostics through a virtual lab has finally arrived. Conjoint analysis typically used for product development has now emerged into direct mail testing. This new technology allows marketers to test over a thousand permutations of 20 cells in a single test, even before entering into the mail. Instead of a 20% success rate, direct marketers are now achieving 90% (+) results.

Most mailers need test cells of 100,000 or more to ensure reliable results. At an average cost of $0.45 per piece with conjoint analysis, marketers are realizing significant savings and generating a pipeline of statistically reliable tests.

Through conjoint analysis and diagnostic testing, the data has the last word and valuable resources that focus on those attributes make a difference.

Art Hall, Ph.D. is senior marketing consultant, Quad/Graphics. Art is a direct marketing veteran with expertise in marketing, technology and finance. He educates and encourages Quad’s direct marketing clients to leverage the power of their customer data into highly personalized life-stage and personal marketing solutions. For more information about connecting qualitative with quantitative data to create better direct marketing results, visit

Previous post

How to do direct mail right

Next post

New Canada Post vision must find right balance

Direct Marketing

Direct Marketing

Lloydmedia, Inc is based in Markham, Ontario, Canada, and is a multi-platform media company which delivers a total audience of more than 100,000 readers across four national magazines, three industry directories, and a range of events and online marketing.

No Comment

Leave a reply

Your email address will not be published. Required fields are marked *