Analytics Strategist

March 4, 2009

attribution: what you want may not be what you need

.. or should I say, what you need may not be what you want?

Attribution problem, particularly the macro attribution problem, is traditionally asking for a way to partition the success metrics to each marketing efforts, in the form of percentages, so that the relative contributions can be measured. The hope is that these percentages can be used, aside from figuring out how to distribute bonus, to guide the optimal budgetting decision.  However, the promise of using attribution as an optimization methodology is flawed. Attribution is basically an additive model of business process where the success can be partitioned as if they are mutually independent. It is problematic when the actually relationship between marketing efforts and their success metric is non-linear – due to either the presence of interaction (in the form of synergy or cannibalization) effects or the intrinsic quantitative relationship.

When the data-driven relationship/model is shown to be non-additive and non-linear, it may not be intuitively clear of how to use it for attribution, i.e. coming up with the percentages.  On the other hand, the non-linear non-additive model should just be what you need for operational optimization, such as budget optimization decision.  This is because true optimization follows the equalization principle of marginal returns, rather than the averages. Attribution percentage is not as necessary for optimization precisely because it is based on the average. It still useful in many cases when the averages and marginals are highly correlated.

This is the basic idea of this post, and the reasons I was asking everyone to come back and read. For fundamentally, you need to optimize your operation; not just a set of percentages for distributing credits.  

The other point I want to make is that the percentages that every attribution is trying to get at, or every attribution rules is trying to produce, only make senses with some types of causal interpretation.  If the data show that one factor/effort has no real influence on success/conversion, then it is conceptually not justifiable to attribution anything to it. Again, the right interpretation of an attribution is based on the affirmation of a causal relationship. This is another reason for why the statistical modeling is fundamentally important.

I am not saying that the conversion model I mentioned is the typical “conversion model” we used in the DM context; it does not has to be exclusively predictive modeling. The special type of conversion model that we are building is a causal model based on empirical data. Much of the predictive modeling techniques do apply, but still there are some differences.  For example, if it is for prediction purpose, proxy variables are as valid as any other variables. It may not be automatically acceptable when building a model that require causal interpretation. 

Correct attribution rules, should be based on sound conversion model. Its implementation in web analytics tools can facilitate the reporting process for insight generation and monitoring purposes. It will have a similar role as what it does now. What I am arguing about it that it should be build based on sound data-driven conversion model, not simply intuitions. My point goes a little further in that I am also arguing the use of conversion models (be it linear or non-linear, additive or non-additive), but not the attribution percentages.

In sum, conversion model will provide what you needed, which is the ability to optimize your operation, but may not be what you wanted with attribution; those percentages that we all like to see and talk about are ultimately less critical than what we thought.  

Please come back and read the next post on a deep dive example of conversion modeling approach to attribution

Comments?

Advertisements

6 Comments »

  1. Huayin – this is great.

    I like the way you describe the shortcoming of the attribution process because of it’s attempt to partition casue in an additive way. I think I knew this in the back of my mind but it’s clear now.

    I also agree with your other point about empirically establishing a causal effect BEFORE attribution, without assuming one exists.

    Finally, I think it’s clear to me that conversion modeling doesn’t directly answer the attribution question; however, it is necessary to inform any attempts to addressing attribution and even beyond – optimization.

    I will stay tuned for your next post.

    Satindra Chakravorty.

    Comment by Satindra Chakravorty — March 4, 2009 @ 9:09 pm

  2. You said it better than I did. Thanks!

    Comment by huayin — March 5, 2009 @ 2:13 am

  3. […] but rather a failure for the purpose of attribution ( I have discussed this extensively in my post: Attribution, what you want may not be what you need).  The interaction between traditional media and digital media however, is of a different nature […]

    Pingback by Media Mix Modeling, the new challenges « Analytics Strategist — March 16, 2009 @ 4:35 am

  4. […] If we can’t find the right way to do it, all is not lost. Perhaps we should ask ourselves a question: is this really what we needed, or just what we wanted? This is the subject of one of my recent post: what you wanted may not be what you needed. […]

    Pingback by attribution of interaction effect: the somewhat wrong logic of doing it « Analytics Strategist — March 17, 2009 @ 4:24 pm

  5. Huayin, great post.

    I agree with your first point about taking non-linearity into consideration.

    I am not sure if I understand your second point correctly. Attribution requires affirmation of causal interpretation? Shouldn’t we include channel inputs into the model, run the model, and then decide whether there is correlation between individual channel inputs and performance outputs? Causal relationships cannot be affirmed without real tests. Could you please explain your ‘conversion model’ in detail? What are the differences between this model and predictive modelling?

    Thanks,
    Minyu

    Comment by Minyu Bian — June 22, 2012 @ 10:41 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: