Analytics Strategist

March 2, 2009

the first 3 insights on attribution analytics

Looking at micro attribution from a conversion modeling framework, there are a few insights we can contribute right away without getting into the details.

1)  The sampling bias

If your attribution analysis used only data from convertors, then you have an issue with sampling bias.

As a first order question for any modeling project, understanding the data sample, therefore the potential sampling bias, is crucial.  How is this relevant to the attribution problem?

Considering a hypothetical, but commonly asked type of, question:

What is the right attributions to banner and search given that I know the conversion path data:
    Banner -> conversion:   20%
    Banner -> search -> conversion: 40%
    Search -> conversion: 25%
    Search -> Banner -> conversion:  15%

Well, what could be with the question?  A standard setup for a conversion model is to use conversion as the dependent variable for the model with banner and search as predictors. The problem here is, we only have convertor cases but no non-convertor cases.  We simply can’t perform a model at all.  We need more data such as the number of click on banner but did not convert.

The sampling bias issue is actually deeper than this.  We want to know if the coverage of banner and search are “biased” for the data we are using, an example is when banner were national while search were regional. We also need to ask, if the future campaigns will be run in ways similar to what happened before – the requirement of modeling setup mimicking the applying context.

2) Encoding sequential pattern

The data for micro attribution is naturally in the form of collection of events/transactions:
User1:
    banner_id time1.1
    search_id time1.2
    search_id time1.3
    conversion time1.4
User2:
    banner_id time2.1
User3:
    search_id time3.1
    conversion time3.2

Some may think that this form of data makes predictive modeling infeasible. This is not the cases.  There are many predictive modeling are done with transaction/event type of data: fraud detection, survival model, to name a couple.  In fact, there are sophisticated practice in mining and modeling sequential patterns that are way beyond what being thought about in common attribution problem discussion. The simple message is:  this is an area that is well researched and practices and there have been great amount of knowledge and expertise related to this already.  

3) Separating model development from implementation processes

Again, the common sense from the predictive modeling world can shed some light on how our web analytics industry should approach attribution problem.  All WA vendors are trying to figure out this crucial question: how should we provide data/tool service to help clients solve their attribution problem. Should we provide data, should we provide attribution rules, or should provide flexible tools so that clients can specify their own attribution rules. 

The modeling perspective says that there is no generic conversion model that is right for all clients, very much like in Direct Marketing we all know there is no one right response model for all clients – even for clients in the same industry. Discover Card will have a different response model than American Express, partly because of the differences in their targeted population, their services, and partly because of the availably of data.  Web Analytics vendors should provide data sufficient for client to build their own conversion models, but not building ONE standard model for clients (of course, they can provide separate modeling services, which is a different story). Web Analytics vendors should also provide tools so that clients’ modeling can be specified/implemented once it’s been developed.  Given the parametric nature in conversion models, none of the tools from current major Web Analytics vendors seem sufficient for this task.

That is all for today. Please come back to read the next post: conversion model – not what you want but what you need.

Advertisements

2 Comments »

  1. Hi Huayin.

    I can see how micro attribution analysis might be similar to conversion modeling and where the conversion modeling framework applies. But, there isn’t a complete overlap. As I understand it, attribution analysis is a backward looking task whereas conversion modeling (as in the direct marketing context) has a backward-facing component (the modeling part, where you use prior data attached to a prior event with outcomes that have already happened) and a forward-facing component (the implementation part where you use new data to predict a future outcome).

    In macro attribution, I can clearly see both backward and forward facing pieces – modeling and forecasting.

    In the micro attribution case, I can see the backward-facing piece, a historical analysis to determine the influence of prior tactics on conversion from prior events. Unlike in macro attribution analysis, where the business objective is to estimate what will happen several steps ahead if certain changes are made, I don’t see a similar business objective with micro attribution. With the latter, as I see it, the business objective is to understand what contributed and how much to outcomes from a past event. This can be accomplished via modeling or maybe inaccurately, via reporting/tracking analysis etc.

    However, when you talk about an implementation phase with micro attribution, is that the same thing as using a model to target an individual for a desired outcome? If so, then the line between attribution analysis and predictive modeling seems to disappear. I am sure there are many real differences in the way this might work for an online application – different attributes, different history periods, different data capture intervals, probably real-time implementation (can’t see batch implementation working here), etc. etc. And if the implementation phase is using a targeting model, then hasn’t the solution changed from attribution to targeting? Has the business objective changed?

    I am still thinking about what the implementation phase of a micro attribution analysis might be like and how it is different/not different from response modeling.

    Satindra Chakravorty.

    Comment by Satindra Chakravorty — March 3, 2009 @ 6:30 pm

  2. Satindra, thanks again for the thoughtful questions and comments. I am going to put my response to it in my next post: “attribution: what you want may not be what you need”. many thanks!

    Comment by huayin — March 4, 2009 @ 7:05 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: