Analytics Strategist

March 4, 2009

attribution: what you want may not be what you need

.. or should I say, what you need may not be what you want?

Attribution problem, particularly the macro attribution problem, is traditionally asking for a way to partition the success metrics to each marketing efforts, in the form of percentages, so that the relative contributions can be measured. The hope is that these percentages can be used, aside from figuring out how to distribute bonus, to guide the optimal budgetting decision.  However, the promise of using attribution as an optimization methodology is flawed. Attribution is basically an additive model of business process where the success can be partitioned as if they are mutually independent. It is problematic when the actually relationship between marketing efforts and their success metric is non-linear – due to either the presence of interaction (in the form of synergy or cannibalization) effects or the intrinsic quantitative relationship.

When the data-driven relationship/model is shown to be non-additive and non-linear, it may not be intuitively clear of how to use it for attribution, i.e. coming up with the percentages.  On the other hand, the non-linear non-additive model should just be what you need for operational optimization, such as budget optimization decision.  This is because true optimization follows the equalization principle of marginal returns, rather than the averages. Attribution percentage is not as necessary for optimization precisely because it is based on the average. It still useful in many cases when the averages and marginals are highly correlated.

This is the basic idea of this post, and the reasons I was asking everyone to come back and read. For fundamentally, you need to optimize your operation; not just a set of percentages for distributing credits.  

The other point I want to make is that the percentages that every attribution is trying to get at, or every attribution rules is trying to produce, only make senses with some types of causal interpretation.  If the data show that one factor/effort has no real influence on success/conversion, then it is conceptually not justifiable to attribution anything to it. Again, the right interpretation of an attribution is based on the affirmation of a causal relationship. This is another reason for why the statistical modeling is fundamentally important.

I am not saying that the conversion model I mentioned is the typical “conversion model” we used in the DM context; it does not has to be exclusively predictive modeling. The special type of conversion model that we are building is a causal model based on empirical data. Much of the predictive modeling techniques do apply, but still there are some differences.  For example, if it is for prediction purpose, proxy variables are as valid as any other variables. It may not be automatically acceptable when building a model that require causal interpretation. 

Correct attribution rules, should be based on sound conversion model. Its implementation in web analytics tools can facilitate the reporting process for insight generation and monitoring purposes. It will have a similar role as what it does now. What I am arguing about it that it should be build based on sound data-driven conversion model, not simply intuitions. My point goes a little further in that I am also arguing the use of conversion models (be it linear or non-linear, additive or non-additive), but not the attribution percentages.

In sum, conversion model will provide what you needed, which is the ability to optimize your operation, but may not be what you wanted with attribution; those percentages that we all like to see and talk about are ultimately less critical than what we thought.  

Please come back and read the next post on a deep dive example of conversion modeling approach to attribution

Comments?

March 2, 2009

the first 3 insights on attribution analytics

Looking at micro attribution from a conversion modeling framework, there are a few insights we can contribute right away without getting into the details.

1)  The sampling bias

If your attribution analysis used only data from convertors, then you have an issue with sampling bias.

As a first order question for any modeling project, understanding the data sample, therefore the potential sampling bias, is crucial.  How is this relevant to the attribution problem?

Considering a hypothetical, but commonly asked type of, question:

What is the right attributions to banner and search given that I know the conversion path data:
    Banner -> conversion:   20%
    Banner -> search -> conversion: 40%
    Search -> conversion: 25%
    Search -> Banner -> conversion:  15%

Well, what could be with the question?  A standard setup for a conversion model is to use conversion as the dependent variable for the model with banner and search as predictors. The problem here is, we only have convertor cases but no non-convertor cases.  We simply can’t perform a model at all.  We need more data such as the number of click on banner but did not convert.

The sampling bias issue is actually deeper than this.  We want to know if the coverage of banner and search are “biased” for the data we are using, an example is when banner were national while search were regional. We also need to ask, if the future campaigns will be run in ways similar to what happened before – the requirement of modeling setup mimicking the applying context.

2) Encoding sequential pattern

The data for micro attribution is naturally in the form of collection of events/transactions:
User1:
    banner_id time1.1
    search_id time1.2
    search_id time1.3
    conversion time1.4
User2:
    banner_id time2.1
User3:
    search_id time3.1
    conversion time3.2

Some may think that this form of data makes predictive modeling infeasible. This is not the cases.  There are many predictive modeling are done with transaction/event type of data: fraud detection, survival model, to name a couple.  In fact, there are sophisticated practice in mining and modeling sequential patterns that are way beyond what being thought about in common attribution problem discussion. The simple message is:  this is an area that is well researched and practices and there have been great amount of knowledge and expertise related to this already.  

3) Separating model development from implementation processes

Again, the common sense from the predictive modeling world can shed some light on how our web analytics industry should approach attribution problem.  All WA vendors are trying to figure out this crucial question: how should we provide data/tool service to help clients solve their attribution problem. Should we provide data, should we provide attribution rules, or should provide flexible tools so that clients can specify their own attribution rules. 

The modeling perspective says that there is no generic conversion model that is right for all clients, very much like in Direct Marketing we all know there is no one right response model for all clients – even for clients in the same industry. Discover Card will have a different response model than American Express, partly because of the differences in their targeted population, their services, and partly because of the availably of data.  Web Analytics vendors should provide data sufficient for client to build their own conversion models, but not building ONE standard model for clients (of course, they can provide separate modeling services, which is a different story). Web Analytics vendors should also provide tools so that clients’ modeling can be specified/implemented once it’s been developed.  Given the parametric nature in conversion models, none of the tools from current major Web Analytics vendors seem sufficient for this task.

That is all for today. Please come back to read the next post: conversion model – not what you want but what you need.

February 26, 2009

micro attribution analytics is conversion modeling

If you are surprised by the title statement, you are in the majority.  

This is actually a very strong statement and I did not make it lightly. It is saying that micro attribution is an area of data analytics that can be defined and studied with rigorous statistical methodologies. In short, it is more of a science than art or common sense.  Micro attribution problem is more like a response modeling or risk modeling problem than the problem of finding out a fair rule for distributing year-end bonus.  

Does this sound the same as how others describe attribution problem and solutions? 

It is certainly different from those who think the solution to attribution problem is about tracking.  Tracking is important because it provides you the data, but in itself they do not tell you what factors or customer experience have more or less influence on conversion.

It is also different from many who think about “last click”, “first click” etc. when they speak about attribution models.  Those are not data analytics models or statistical models that I was referring to.  One is about intuition-based smart rule; the other is about data-driven behavioral modeling.  The smart rule vs. modeling debate was long over in Direct Marketing, but it is just beginning in web analytics and online, right here in the micro attribution problem.   

It is also different from the many who think this is all about metrics (because of the claim that there is no right solution to attribution :).  It is not about averaging the attribution of first-click and last click. It is not about using engagement metrics as a proxy either. 

It is definitely not the same as those who think we need to wisdom-of-the-crowd type of solution.  The percentage of you who think early keywords should get 15% attribution for “assist” maybe right, but it has no bearing to me.  I do not believe that there is an average truth in any of these, for reasons that I do not believe one retailer’s offer-X response model shouldn’t be used for a loyalty campaign of a telecom company.

It is categorically different from those who hold that there is no right answer to the attribution problem. I agree that there is no perfect model that has no model prediction errors, but that is not a refutation for statistical modeling.  Statistics is founded on imprecision in data and never afraid of counter examples.

It is an approach of simplification, not of complication and certainly not a proposal to bring in psychology, media-logy or astrology into the picture.  In that regard, it could be a spoiler for the fun party we had so far.

Still, it is really just a claim at this point.  Please come back to read the next post: (TBD)

Blog at WordPress.com.