Testing and measuring data-driven mail campaigns

The value of testing

Data can help you understand the effect of your targeting and creative on any communications or campaigns. To improve marketing performance marketers are encouraged to ‘test and learn’.

The easiest way to start testing is using an A/B test. It's best to keep this simple: only test one element per campaign for example ’10% off’ vs. ‘£5 off when you spend over £30’.

There are many ways to measure a direct marketing campaign, here are a few of the simple methods.
 

Direct measurement

Direct measurement is where it’s possible to get a one-to-one match on responders. Such as using promotional codes and measuring redemptions. Match-backs are also commonly used, this is where a mailing list of addresses is matched back to a transactional file (customer orders) to understand who has responded or purchased from the mailing list. It's best to measure both coupon redemptions and complete a match-back as sometimes a customer who responds doesn’t always remember to redeem a coupon!

Indirect measurement

Indirect measurement is where a response is inferred. An example of this would be to take two similarly performing store locations, then complete leaflet distribution (door drops) around one store and not the other. You would then look at the sales or footfall uplift over the campaign period using a baseline of both stores a few weeks before the activity as a comparison. You should be able to see a clear uplift in these metrics after the marketing activity drops, this would be your inferred or indirect response. It's important to understand that this will only work if you use a suitable volume, as you would not be able to tell the impact if you sent just 100 leaflets around a Supermarket Megastore!  

Fallow/control groups

Fallow, holdouts, or control groups is where you ringfence a small percentage of your customer database who will - for test purposes - not receive any marketing or campaign activity over a period of time that you’re trying to measure. The effect of this inactivity can then be used to justify the marketing efforts over the long-term.

Some other more complex methods of measurement include Econometrics, multi-touch attribution and Marketing Mix Modelling.

Ideally you would want to use all the measurement methods to get as clear a view as possible on how effective your campaign has been. Reviewing the performance of a campaign from as many angles as possible may highlight something otherwise missed and ultimately lead to better performing campaigns.

For more information on measuring direct mail performance, see Mail Measurement and Attribution