Working Paper

The Need: Accurate Early Forecasts

When direct mail marketers send out a mailing, they often need to watch its response closely in the first few days, to quickly project how well it will perform versus plan. Mail order catalogs, publishers, continuity clubs, and fund-raisers, are example firms with this need. Important decisions often hinge on this early interpretation of a mailing's response, for example, adjustments to inventories, changes to earnings or cash flow projections, or retargeting future mailings.

Besides such "logical" reasons, managers often have an emotional stake as well. Many mailings are experiments in some way, such as new products, new offers, different timing, or different list selections. The people who designed these experiments want to see how well their ideas worked -- the sooner the better. We frequently see CEOs watching the daily numbers as intensely as anyone else.

The Problem: Timing Uncertainty vs. Demand Uncertainty

Despite such attention, it often takes weeks for a clear answer to a mailing's success. Why? Because in the early days, variations in the timing of customer responses play havoc with the interpretation of the daily orders. Differences in postal delivery are familiar to everyone, but other factors also affect customers' timing, for example:

• At certain times of the year, customers take longer to make up their minds.
• Sales or special promotions usually cause shoppers to buy quickly.
• Some mailing lists, or segments of the house file, respond faster than others.
• Successive mailings to the same customers may interact with one another.

There are many such factors, and their influence varies from mailing to mailing. Forecasters have trouble keeping all factors in mind at once and adjusting consistently for them. Thus, the "timing uncertainty" during the early days of a live mailing often exceeds the demand uncertainty. In many lines of business, a mailing must reach 40% - 50% of its total response before its projections are reliable. In the meantime, forecasters' projections often swing up and down as they "home-in" on the target (see sketch).

Many users of these projections, such as inventory planners, have learned to distrust the early numbers and wait until the picture is clearer. Obviously, this delay hurts their ability to react and leaves more "fires" to put out once they do.

Reducing the Timing "Uncertainty"

There is less uncertainty in the timing factors than meets the eye. When each factor can be isolated separately, its effects tend to be consistent from one mailing to another. Most of the "uncertainty" comes from forecaster overload, trying to weigh many factors operating all at once. This problem presents a good application for modeling.

The details of the modeling vary from one modeler to another. Some work with the mailing's "percent done curve," others work with the falloff rate of responses, or other approaches (see nearby illustration). Within these broad options, numerous detail choices remain, such as modeling the daily curve directly, fitting a statistical distribution to the data, or combining several models. Each has its pros and cons for different applications.

Operationally, modeling these timing factors requires building a history file of the daily responses from past mailings, together with their timing factors such as mail date, mailing size, adjacent mailings, list selections, and specifics of the offer. Typically, it is useful to have 2 - 3 years of history for analysis.

The Issue of Interpretation

Modeling will reduce the timing uncertainties but not eliminate them completely. Variations remain, and forecasters are left with a tricky problem of interpretation. Suppose a mailing was forecast for 1,000 responses by Day 2 and by that time it actually reaches 1,200. Do we conclude that it is going to finish 20% above plan? Or, do we just say the response was early this time, and the mailing is holding to its original plan? The truth probably lies somewhere in-between, but where?  Answering that question requires a bit of mathematics, but it is easily handled, in our case, by an add-in for Microsoft Excel.

Putting the Pieces Together: A Convenient, Compact System

Each day, the Reforecasting Live system uses a pre-fitted model to project a mailing's percent done for that day, based on information up through the previous day's orders. The reforecasting module then compares this expectation with the actual orders for the day to yield a new projection for the life of the mailing. The result is a convenient, compact system for reforecasting live mailings. It requires practically no effort in daily use, and it typically converges near its final level within a few days after customers begin to receive the mailing. In one company, this was a full two weeks sooner than it had previously waited for reliable readings, in another about 10 days sooner. Two additional benefits were the absence of up-and-down swings and released time for analysts to work on other things. An attachment shows several examples of projections generated by the Reforecasting Live system.

The technology in Reforecasting Live has also been applied for projecting individual products  to guide inventory replenishment  (a process often called "in-season" item forecasting).  In that application, labor savings are often as important as forecast accuracy, since the software relieves buyers from days and days of unrewarding work.