Direct Marketing Magazine - December 1996

• Do You Need Early Warnings?
• What's in an Early Warning Sytem?
• Performance
• Watchwords in Implementation
• An Added Bonus
• Concluding Comments

A small but growing number of direct marketers use the DEW line idea in their businesses by pretesting their offerings a few weeks or months before a major selling season. With this input, they scramble for more supplies of hot items, head off production for slow-movers, adjust prices, and generally manage operations in coming months with fewer surprises. The idea is particularly helpful for companies with short, intensive selling seasons such as Christmas.

When it comes to advance testing, direct marketers have a big advantage over others. To conduct a test, all they truly need is the printed mailing. They avoid any need for warehousing, manufacturers' minimums, handling of goods, and the like.

Do You Need Early Warnings?

Suppose you had a radar that could detect customers' interests in your products a month before drop-date. What value would such alerts have for you? Obviously, it depends on many factors. For Christmas toys, the selling season is short; customers will not wait for backorders and competition is intense. Under these conditions, an extra 4-6 weeks of advance planning should have high payoff, particularly in a better ability to "chase" hot items. For ongoing staple items, such as office supplies, underwear, or most blue jeans, the risks of surprise are low, and so will be the value of advance alerts. Companies that sell fashion apparel, short-season gift items, or long lead time items have shown the greatest interest in getting advance readings on their market. An early alert can tip the balance between a crisis and a striking success, as this cataloger found:

A maker of branded apparel introduced a "fun" product unrelated to its regular line -- a teddy bear. A mid-level manager came up with the idea, thinking customers might like to give it as a Christmas present. Other managers liked the idea, but lacking any experience in the "stuffed animal" category, they wrestled with the question of how many to make. Finally, they ordered an amount everyone agreed was conservative.

By good luck, a few weeks later the company conducted a survey on new products that included, among other items, this teddy bear. Customers "bought" it at more than five times the forecast level. Inventory buyers scrambled to increase supplies, and when Christmas was over, they had sold almost seven times the original forecast. Without the early alert, managers felt they may have met one-third of that demand.

What's In An Early Warning System?

Let's assume you want to set up an early warning system for a new line of products to be offered in next fall's mailings. Your company will be introducing them in a few months, and your mission is to plan production so that when they come out, you can meet the demand. There are three main forecasting issues:

1. Market and economic conditions generally. Will people be in a spending mood?

2. Customers' acceptance of the line as a whole.

3. Customers' item-by-item preferences within the new line.

The first issue, overall conditions, I will save for a future discussion. This article will focus on the relative appeal of different products. Research shows, by the way, that uncertainties in product appeal create far more costs for most direct marketers than do fluctuations in overall demand.

If a supply of the product is available, the most accurate test is a "live" test -- that is, mailing the actual offer to a sample of customers and waiting for orders. Some direct marketers, thinking no other option was available, have taken this approach but at considerable cost. Catalogers have told us of new product tests that cost hundreds of thousands of dollars for just a single product line. Obviously, such costs limit live tests mostly to large firms.

Some companies use a 6 or 12-month variation of the live test. They include test items as part of a regular (but small-circulation) mailing, with the aim of analyzing results and "rolling out" the successful items to their full list 6 or 12 months later. Accuracy suffers in these tests, though, because of the long time until roll-out and because the items' presentations, or the competing products in the catalog, may change substantially by the time of roll-out. And costs, while lower than the full tests above, still limit this approach to relatively few items.

More often, supplies of product will not yet be available and the test must proceed with a survey or mock order form. Concern about the realism of surveys, however, has caused some companies to rule out such tests without ever trying them. "If test customers aren't spending real money, how can their responses be relied upon?" or "How can the purchases of a few early-bird customers tell us much about the masses to follow?"

To be sure, "artificial" measurements, such as surveys, do introduce their own biases, but these distortions usually have a statistical regularity. Some aspects of human nature are, after all, fairly predictable. The chief biases that seem to appear in direct mail pretests are:

• Wishful thinking -- When they are not spending real money, respondents tend to "buy" more items in surveys than they actually do. Not surprisingly, this bias is greatest for high-price items and lowest (or even reversed) for low-price items.

• Weather factors -- When customers answer questions about sweaters in July, they will, no surprise, overstate their interest in lightweight cottons and understate their interest in wool and other warm designs.

• Date-sensitive items -- Items that sell at certain holidays, such as Easter or Christmas, tend to be underrepresented in tests conducted months in advance. Customers' procrastination on gift items seems to increase with each passing year.

In each of these cases, the key to useful accuracy is calibration of the test. Repeat the test cycle a few times and you will have enough information to adjust out these measurement errors in the future.

You can watch this process in action every couple of years on election night. Based on exit polls, with proper adjustments such as voting patterns in ethnic districts, the TV networks are able to call the winners of most races by early evening, long before much of the vote has been tallied.

Performance

Illustrated above is the outcome of a customer contact test for a specialty apparel catalog. This company mailed a preview of one of its catalogs to a sample of customers, together with a questionnaire, about a month before publication of the full catalog. Based on customers' responses, they then updated the item forecasts prepared earlier by inventory buyers.

In this chart, the heavy line shows the pattern of buyers' forecast errors without the test. The dashed line shows the pattern of errors if forecasts were based purely on the test. Even with its inherent biases, such as respondents not spending real money, this test reduced the standard deviation of forecast errors by about one-third. It also reduced the number of large errors (more than three times forecast or less than 1/3 of forecast) from about 9% of the items to virtually none. The third, lighter line shows the forecast errors that remained after the test results were adjusted to remove predictable measurement biases. In this adjusted form, the test removed about three-fourths of the variance between the original forecasts and the actual sales, across dozens of items.

This test reached about one-fifth of one percent of the audience for the catalog. Though overall estimates of the test's payoff were not computed, the savings in just two items, unexpected hot-sellers, more than paid for the test. Less tangible benefits included better customer satisfaction and fewer strains with suppliers. Inventory buyers reported, as a major benefit, the added confidence they felt when the test confirmed the appeal of items for which they previously had felt they were going out on a limb.

Watchwords in Implementation

In building an early warning system, the following watchwords may help:

1. Match your design to the task. If all you need is an early indication of interest in a few new items, an advisory panel of customers may do the job. If the financial stakes are high, however, because perhaps the new items will cannibalize sales from current ones, then you will probably want a quantitative test with a statistical sample of consumers. Similarly, if you are introducing a whole line of products, a statistically designed test will yield more reliable information.

2. Measure everything (or almost everything). Keep detailed records of who received the test, their RFM cells, dates of the mailings and responses, different versions that were mailed, and so on. Much of the value in your tests will come from comparing the results with various benchmarks, such as previous tests, and comparing responses for different products or customer groups. You may want to see if the test has any impact on recipients' prior buying behavior. Good measurements also enable controlled experiments, such as trying different prices. Usually, such measurements add little or no costs to the project.

3. Give careful thought to the mailing package, not only the test items, but companion items on facing pages, cover promotions, order forms, introductory letters, and the like. In the case of catalogs, this means including competing products, such as others in its own department or products customers typically choose as substitutes. Though an item's appeal to customers will show clearly in a pretest, competing products or near substitutes in the same catalog may significantly shade the results up or down, hurting the test's accuracy.

4. Take advantage of all your information sources. As useful as they may be, customer contacts are only one among several sources of information reaching you about a new product. Others include the opinions of product developers and anyone else who has seen or worked with the product during its development stages (such as suppliers), past histories of analogous products, and your own intuitive feelings about the item. If you keep a record of these sources' predictions, you will, over time, be able to calibrate their accuracy and thus know how much weight to give them in the future.

5. Include some calibration points in your test. If you are testing the whole catalog, it inevitably will include a number of "old classics" whose demand is relatively predictable. If you are testing a solo mailer or a fractional catalog, plan to give more attention to the selection of other items you will include in the test and how you will interpret their results.

6. Stick to one mission. Quite often in customer tests there is a strong temptation to try myriad variations of the product -- to "tweak" its price, features, ad copy or whatever, to arrive at the combination that most appeals to customers. Unfortunately, this tweaking comes at a price, in added sample or diminished forecasting accuracy. Such testing usually serves a marketing purpose more than a forecasting one, so you will need to consider its cost -- and value -- in that light.

7. As you gain experience with early warning measures, you can expect several benefits from the learning curve. First, the mechanics of conducting the measurements become easier, based on lessons from experience. Second, you can compare the results of past tests with the products' actual sales results, to better calibrate the measurement biases in the tests. From this analysis, you can improve the correction factors to apply in future years. Or, you might adjust the test procedure to reduce such effects. Finally, inventory planners grow to accept the tests and act more decisively on the information.

An Added Bonus

An added bonus of our pretesting work has been insight into the effects of display on performance of direct mail items. Not surprisingly, the pretest findings frequently disagree with buyers' initial forecasts. In exploring these discrepancies, the explanations frequently came back "lousy shot" or "it's just a great picture." Over the years these discussions have led to a variety of buyers' theories about the linkage between display and demand. When tested statistically, about half of these theories have proved accurate. They have been incorporated into the adjustment models or pretests as well as buyers' practices in pre-season forecasting. Other useful insights relate to the competition among items in a department and promotional pricing.

Concluding Comments

Perhaps the most important lesson of all from early warning systems is that it does not take long for organizations to catch on to their value. Once people break the mindset that, "We can't do it," or "Nobody would believe it," or whatever, the possibilities expand exponentially. We have seen companies increase their testing programs by ten-fold or more over the course of a few years. This growth takes place not only in scale but also in their degree of sophistication, such as looking for ways to test items earlier in the development cycle. Functionally, they are adding longer-range radars to their DEW lines.

With intensifying competition and a faster pace of change in business, early warning systems will almost certainly grow in number and importance. Just like the Allies in the 1950s, aggressive companies will always want to gain an edge over their competitors through better intelligence about their markets. As the DEW line proved four decades ago, looking beyond the horizon can make the difference between last-minute panics and being able to sleep at night. If your company does not yet have an early warning system, maybe now is the time to start thinking about how you might build one.