The Principles of Attribution Model

(Disclaimer:  some questions and answers below are totally made up,  any resemblance to anything anyone said is purely coincidental)

How do we know an attribution model, such as Last Click Attribution, is wrong?

• it is incorrect - surprise surprise, a lot of people just make the claim and be done with it
• it does not accurately capture the real influence a campaign has on purchase – but how do you know it?
• it only credit the closer – isn’t this just a re-statement of what it is?
• it is unfair to upper funnels and only awards to lower funnel – are you suggesting that it should award to all funnels, why?
• it leads to budget mis-allocation so your campaign is not optimized – how do you know?
• it is so obvious, I just know it – what?

How do we know an attribution model, such as a equal attribution model, is right?

• it is better than LCA – intuition?
• it gives out different credits than LCA so you can see how much mis-allocation LCA does to you campaign – different from LCA is not automatically right
• we tested and it generate better success metrics for the campaign – sound good, how?
• it is fair – what does that mean?

How do we find the right attribution model?

• try different attribution models and test the outcome – attribution model does not general outcome to campaigns directly
• play with different models and see which one fit your situation better – how do I know the fitness?
• use statistical modeling methodology to measure influence objectively – what models? conversion models?
• use predictive model for conversion – why predictive models? what models? how to calculate influence and credit from the models?
• test and control experiment - how many test and control, what formula to use to calculate credit?
• you decide, we allow you to choose and try whatever attribution weights you want – but I want to know what’s the right one?
• the predictive models help you with optimization, once we get that, you do not care about attribution – but I do care …
• shh … it is proprietary: I won’t tell you or I will kill you! - ?

The Principle of Influence

Three principles are often implicitly used:  the “influence principle”,  the “fairness principle” and the “optimization principle”.

The influence principle works like this: assume we can measure each campaign’s influence on a conversion, the correct attribution model will give credit to campaigns proportional to their influence.  The second principle is often worded with “fairness”, but very much the same as the first principle:  if multiple campaigns contribute to a conversion, giving 100% credit to only one of them if “unfair” to others.  The third principle, the optimization principle, in my understanding, is more about the application of attribution (or the benefit of it) and not about the principle of attribution.

The principle of influence is the anchor of three; the fairness and optimization principles are either a softer version or a derivative of it.

Now we have our principle, are we close to figuring out the right approach to attribution model?  We need to get closer to the assumption of this principle.  Can we objectively measure (quantify)  influence?  Are there multiple solutions or just one right way to do this?

If influence principle is the only justification of attribution models, then quantitative measurement methodology such as probabilistic modeling, some time it is called algorithmic solution which I think is a misnomer,  will be the center technology to use.  It leave no room for arguing just on the ground of intuition alone.  Those who offer only intuition and experience, plus tools for clients to play with whatever attribution weights are not attribution solution provider, but merely a vendor of flexible reporting.

Those of the intuition and experience school like to frame attribution model around the order and position of touch points:  the first/last/even and the introducer/assist/closer. (how many vendors are doing this today?)  They have troubles in providing quantitative probabilistic solution to attribution issue.  The little known fact is that it is analytically flawed:  the labeling of “last touch” and “closer” are only known post-conversion, and therefore not usable inside probabilistic modeling framework.  In predictive modeling and data mining lingo, this is known as the “leakage problem”.  (search on Google, or read Xuhui’s article that mentioned this).

Unfortunately, we have a problem with the data scientist camp as well but of different nature; it is the lack of transparency with metrics, models and process details.  Some vendors are unwilling to open up their “secret sauce”.  Perhaps, but is that all?  I will try to demystify and discuss the “secret sauce” of attribution modeling.

Attribution Model vs Attribution Modeling

Attribution is a difficult topic, growing into a mess of tangled threads.

I hope this post, and subsequent ones, will help to untangle the messy threads.  I like to start with simple stuff, be meticulous with the use of words and concepts and be patient; after all, haste makes waste.

When an advertiser records a conversion or purchase, some times there are multiple campaigns in the touch point history of the conversion; how do we decide what campaign(s) responsible for the conversion and how should the conversion be credited to each of these campaigns?  This is the attribution problem; a practical issue first raised in digital advertising but in itself a general analytical challenges.  It is applicable to many marketing/advertising contexts, cross channels or within a particular channel for example.

Micro vs Macro

Notice that Attribution is a “micro” level problem: it dealt with each individual conversion event.  In contrast, Marketing Mix Model (or Media Mix Model) deals with “macro” level problem: crediting conversion volume to each channel/campaigns in aggregate.  There are similarity between the two when viewed from the business side; they are quite different analytic problems, different in all major aspects of the analytic process: from data to methodology to application.

Attribution Model vs Attribution Modeling

Advertisers implement business rule(s) to handle this “attribution”, or credit distribution, process.  These rules are generally called “attribution rule” or “attribution model”; examples of it are Last Click Model, First Click Model, Fractional Attribution Model etc..  Rules and models are interchangeable in this regard, they serve as instruction set for the execution of attribution process.

There are no shortage of attribution rules or models being discussed. Anyone can come up with a new one, as long as it does partitioning credit .  The challenge is finding the right one, to choose from too many of them.  In other words, it is the lack of justification for the approach, process and methodology of Attribution Rules/Models that is a problem.

Now comes the Attribution Modeling – a statistical model-based approach to quantify the impact of each campaigns on each individual’s conversion behavior.  It is a data-driven algorithmic approach; it is hot and cool, with an aura of objectivity around.  It is often treated as the secret sauce to unlock attribution and optimization and covered with a proprietary blackbox.

Let me slow down a bit.  I have discussed two important concepts here: Attribution Model and Attribution Modeling. The former refers to the attribution rules; the later refers to the process of generating/justifying the rules. I understand that everyone do not agree with my use of the words, or the distinction between the two; but I think this is a critical distinction, for untangling the threads in attribution discussion.

Domain Expert vs Data Scientist

There are generally two camps when it comes to the generation/justification of attribution model.  The first is the “domain experts” and the second the “data scientists”.  Domain experts take issues with attribution models by pointing out the good and bad, arguing on behalf of common sense, experience and intuition; it is qualitative, insightful and at times interesting but generally pessimistic and fall short when it comes to build a rigorous data-driven solutions. The general principle for justifying attribution is one of two: the influence and the fairness.  The influence principle attributes credit based on the influence of the campaign on conversion, whereas the fairness is often stated generally.

The fairness principle is not a concern for the data scientists camp; in fact, it is all about modeling/quantifying the impact or influence. After all, if you can do the attribution based on precise measurements of influences of each touch points, what other principle do you need? Of course, the problem is often about the promise and less about the principle.  In contrast to the domain experts, the data scientists approach is quantitative, rigorous and data driven. You can argue with the choice of a specific modeling methodology, but the resulting model itself does not require common sense or past experience to justify.

Principle of Attribution:  Influence, Fairness, Optimality

A third principle in picking the right attribution model is optimality, for lack of better word.  Do right attribution models lead to optimal campaign management?  Some argue yes.  Does the reverse statement true? Can optimality be a principle in choosing or justifying attribution model?  These are some of things I will discuss and debate about in my next writeup.

Thanks for reading!

Attribution Model and Attribution Modeling do not mean the same thing

With great frustration (to myself and many others who speak about attribution model before), I am making a plea here:  please make it clear what do you mean when you write or speak about attribution model!  For those who do not have the patient to think over, pick one from the two most common uses of it:

A:  Attribution Model as a reference to the process or rules about crediting marketing/advertising success to individual campaigns. Names for such commonly used credit allocation rules:  Last-Click, First Click and even distribution etc.

B:  Alternatively, people use attribution model to means the statistical modeling methodology and/or processes in producing the credit allocation rules above – this could be all kinds of control/experiment testing, regression modeling, bayesian statistical modeling etc..  There are arguments about whether the right model has to be causal model, explanatory model or predictive models.

A or B, which one are you? In other words, which one do you mean when you utter “attribution model”?

I am A; and I use “attribution modeling” for B.  This is the best I can do, after quite sometime struggling with it.

I believe this is a serious matter.  To quote Confucius: “If language is not correct, then what is said is not what is meant; if what is said is not what is meant, then what must be done remains undone.”

Funny analogies of wrong attribution models

Few topics are near and dear to my heart as Attribution Modeling is.  I first bumped into it a more than 4 years ago; and my first written piece on attribution is a linkedin Q&A piece answering a question from Kevin Lee on duplication-rate  (in August 2007).  Since then, my interest in attribution gets real serious, resulting in a dozen’s attribution related blog posts.  The interest never died after that, although I have not written anything the last three years.

I am back on it with a vengeance! Consider this as my first one back.

I want to start on a gentle note though.  I am amused about people still debating about First Touch vs Last Touch attribution as viable attribution models, a bit out of the question in my opinion.  I want to share some funny analogies for what could go wrong with them.

Starting with Last Touch Attribution Model, a football analogy goes like this: “relying solely on a last click attribution model may lead a manager to sack his midfielder for not scoring any goals. Despite creating countless opportunities he gets no credit as his name isn’t on the score-sheet. Similarly a first click attribution model may lead the manager to drop his striker for not creating any goals, despite finishing them. – BrightonSEO presentation slides

There are a lot of good analogies like this that are derived from team sports.  This analogy is applicable not only to Last Touch, but to all single touch point attribution models.  The funniest one I heard is about First Touch Attribution, from none other than the prolific Avinash Kaushik: “first click attribution is like giving his first girlfriend credit for his current marriage.” - Avinash quote

Analogy is analogy, it does not do full justice to what’s been discussed.  However, what we should learn at least this much: if your attribution model is solely based on the sequencing order of touch points, you are wrong.  Those who propose Last, First, Even, Linear or whatever attribution models, watch out!

A good attribution model needs a disciplined development process, and better yet, a data-driven one.  The less the assumptions made about the values of touch points the better – we should learn to let empirical evidence speak for itself.

Do you have any interesting analogy, or thought?

The three generations of (micro) attribution analytics

For marketing and advertising, attribution problem normally starts at the macro level: we have total sales/conversions and marketing spends.  Marketing Mix Modeling (MMM) is the commonly used analytics tool providing a solution using time series data of these macro metrics.  

The MMM solution has many limitations that are intrinsically linked to the nature of the (macro level) data that’s been used.  Micro attribution analytics, when the micro-level touch points and conversion tracking is available, provides a better attribution solution.  Although sadly, MMM is more often practice even when the data for micro-attribution is available; this is primarily due to the lack of development and understanding of the micro attribution analytics, particularly the model-based approach.

There has been three types, or better yet, three generations of micro analytics over the years: the tracking-based solution, the order-based solution and the model-based solution.

The tracking-based solution has been popular in the multi-channel marketing world.  The main challenge here is to figure out through which channel a sale or conversion event happens. The book Multichannel Marketing – Metrics and Methods for On and Offline Success by Akin Arikan is an excellent source of information for the most often used methodologies – covering customized URL, unique 1-800 numbers and many other cross-channel tracking techniques.  Tracking normally implemented at the channel-level, not individual event levels.  Without the tracking solution, the sales numbers by channels are inferred through MMM or other analytics. With proper tracking, the numbers are directly observed.

Tracking solution essentially a single attribution approach to a multi-touch attribution problem. It does not deal with the customer level multi-touch experience.  This single-touch attribution approach leads natrually to the last-touch point rule when viewed from a multi-touch attribution perspective.  Another drawback of it is that it is simply a data-based solution without much analytics sophistication behind it – it provides relationship numbers without a strong argument for causal interpretation.  

 The order-based solution explicitly recognizes the multi-touch nature of individual consumer experience for brands and products. With the availability of micro-level touch point and conversion data, order-based attribution generally seeks attribution rules in the form of a weighting scheme based on the order of events. For example, when all weights are zero except the last touch point, it simply reduced to the LAST touch point attribution.  There has been many such rules been discussed; with constant debate about the virtual and drawbacks of each and every one of the rules.  There are also derived metrics based on these low-level order-based rules, such as the appropriate attribution ratio (Eric Peterson).

Despite the many advantages of order-based multi-touch attribution approach, there are still methodological limitations. One of the limitations is that, as many already know, there is no weighting scheme that is generally applicable, or appropriate for all business under all circumstances. There is no point of arguing which rule is the best without the specifics of the business and data context.  The proper rule should be different depending on the context; however, there is no provision or general methodology for the rule should be developed.  

Another limitation of the order-based weighting scheme is: for any given rule, the weight of an event is determined solely based on the order of event and not the type of event.  For example, one rule may specify the first click getting 20% attribution – when it maybe more appropriate to give the first click 40% attribution if it is a “search” and 10% if it is a “banner click through”.

Intrinsic to its intuition-based rule development process is that it does not have a rigorous methodology to support any causal interpretation which is central for right attribution and operation optimization.

Here comes the third generation of attribution analytics: the model-based attribution.  It promises to deliver a sound modeling process for rule development, and provides the analytical rigor for finding relationships that can have causal interpretation.  

More details to come.  Please come back to read the next post: a deep dive example of model-based attribution.

Related post: Micro and Macro Attribution

attribution: what you want may not be what you need

.. or should I say, what you need may not be what you want?

Attribution problem, particularly the macro attribution problem, is traditionally asking for a way to partition the success metrics to each marketing efforts, in the form of percentages, so that the relative contributions can be measured. The hope is that these percentages can be used, aside from figuring out how to distribute bonus, to guide the optimal budgetting decision.  However, the promise of using attribution as an optimization methodology is flawed. Attribution is basically an additive model of business process where the success can be partitioned as if they are mutually independent. It is problematic when the actually relationship between marketing efforts and their success metric is non-linear – due to either the presence of interaction (in the form of synergy or cannibalization) effects or the intrinsic quantitative relationship.

When the data-driven relationship/model is shown to be non-additive and non-linear, it may not be intuitively clear of how to use it for attribution, i.e. coming up with the percentages.  On the other hand, the non-linear non-additive model should just be what you need for operational optimization, such as budget optimization decision.  This is because true optimization follows the equalization principle of marginal returns, rather than the averages. Attribution percentage is not as necessary for optimization precisely because it is based on the average. It still useful in many cases when the averages and marginals are highly correlated.

This is the basic idea of this post, and the reasons I was asking everyone to come back and read. For fundamentally, you need to optimize your operation; not just a set of percentages for distributing credits.  

The other point I want to make is that the percentages that every attribution is trying to get at, or every attribution rules is trying to produce, only make senses with some types of causal interpretation.  If the data show that one factor/effort has no real influence on success/conversion, then it is conceptually not justifiable to attribution anything to it. Again, the right interpretation of an attribution is based on the affirmation of a causal relationship. This is another reason for why the statistical modeling is fundamentally important.

I am not saying that the conversion model I mentioned is the typical “conversion model” we used in the DM context; it does not has to be exclusively predictive modeling. The special type of conversion model that we are building is a causal model based on empirical data. Much of the predictive modeling techniques do apply, but still there are some differences.  For example, if it is for prediction purpose, proxy variables are as valid as any other variables. It may not be automatically acceptable when building a model that require causal interpretation. 

Correct attribution rules, should be based on sound conversion model. Its implementation in web analytics tools can facilitate the reporting process for insight generation and monitoring purposes. It will have a similar role as what it does now. What I am arguing about it that it should be build based on sound data-driven conversion model, not simply intuitions. My point goes a little further in that I am also arguing the use of conversion models (be it linear or non-linear, additive or non-additive), but not the attribution percentages.

In sum, conversion model will provide what you needed, which is the ability to optimize your operation, but may not be what you wanted with attribution; those percentages that we all like to see and talk about are ultimately less critical than what we thought.  

Please come back and read the next post on a deep dive example of conversion modeling approach to attribution. 

Comments?

the first 3 insights on attribution analytics

Looking at micro attribution from a conversion modeling framework, there are a few insights we can contribute right away without getting into the details.

1)  The sampling bias

If your attribution analysis used only data from convertors, then you have an issue with sampling bias.

As a first order question for any modeling project, understanding the data sample, therefore the potential sampling bias, is crucial.  How is this relevant to the attribution problem?

Considering a hypothetical, but commonly asked type of, question:

What is the right attributions to banner and search given that I know the conversion path data:
    Banner -> conversion:   20%
    Banner -> search -> conversion: 40%
    Search -> conversion: 25%
    Search -> Banner -> conversion:  15%

Well, what could be with the question?  A standard setup for a conversion model is to use conversion as the dependent variable for the model with banner and search as predictors. The problem here is, we only have convertor cases but no non-convertor cases.  We simply can’t perform a model at all.  We need more data such as the number of click on banner but did not convert.

The sampling bias issue is actually deeper than this.  We want to know if the coverage of banner and search are “biased” for the data we are using, an example is when banner were national while search were regional. We also need to ask, if the future campaigns will be run in ways similar to what happened before – the requirement of modeling setup mimicking the applying context.

2) Encoding sequential pattern

The data for micro attribution is naturally in the form of collection of events/transactions:
User1:
    banner_id time1.1
    search_id time1.2
    search_id time1.3
    conversion time1.4
User2:
    banner_id time2.1
User3:
    search_id time3.1
    conversion time3.2
Some may think that this form of data makes predictive modeling infeasible. This is not the cases.  There are many predictive modeling are done with transaction/event type of data: fraud detection, survival model, to name a couple.  In fact, there are sophisticated practice in mining and modeling sequential patterns that are way beyond what being thought about in common attribution problem discussion. The simple message is:  this is an area that is well researched and practices and there have been great amount of knowledge and expertise related to this already.  

3) Separating model development from implementation processes

Again, the common sense from the predictive modeling world can shed some light on how our web analytics industry should approach attribution problem.  All WA vendors are trying to figure out this crucial question: how should we provide data/tool service to help clients solve their attribution problem. Should we provide data, should we provide attribution rules, or should provide flexible tools so that clients can specify their own attribution rules. 

The modeling perspective says that there is no generic conversion model that is right for all clients, very much like in Direct Marketing we all know there is no one right response model for all clients – even for clients in the same industry. Discover Card will have a different response model than American Express, partly because of the differences in their targeted population, their services, and partly because of the availably of data.  Web Analytics vendors should provide data sufficient for client to build their own conversion models, but not building ONE standard model for clients (of course, they can provide separate modeling services, which is a different story). Web Analytics vendors should also provide tools so that clients’ modeling can be specified/implemented once it’s been developed.  Given the parametric nature in conversion models, none of the tools from current major Web Analytics vendors seem sufficient for this task.

That is all for today. Please come back to read the next post: conversion model – not what you want but what you need.

micro attribution analytics is conversion modeling

If you are surprised by the title statement, you are in the majority.  

This is actually a very strong statement and I did not make it lightly. It is saying that micro attribution is an area of data analytics that can be defined and studied with rigorous statistical methodologies. In short, it is more of a science than art or common sense.  Micro attribution problem is more like a response modeling or risk modeling problem than the problem of finding out a fair rule for distributing year-end bonus.  

Does this sound the same as how others describe attribution problem and solutions? 

It is certainly different from those who think the solution to attribution problem is about tracking.  Tracking is important because it provides you the data, but in itself they do not tell you what factors or customer experience have more or less influence on conversion.

It is also different from many who think about “last click”, “first click” etc. when they speak about attribution models.  Those are not data analytics models or statistical models that I was referring to.  One is about intuition-based smart rule; the other is about data-driven behavioral modeling.  The smart rule vs. modeling debate was long over in Direct Marketing, but it is just beginning in web analytics and online, right here in the micro attribution problem.   

It is also different from the many who think this is all about metrics (because of the claim that there is no right solution to attribution .  It is not about averaging the attribution of first-click and last click. It is not about using engagement metrics as a proxy either. 

It is definitely not the same as those who think we need to wisdom-of-the-crowd type of solution.  The percentage of you who think early keywords should get 15% attribution for “assist” maybe right, but it has no bearing to me.  I do not believe that there is an average truth in any of these, for reasons that I do not believe one retailer’s offer-X response model shouldn’t be used for a loyalty campaign of a telecom company.

It is categorically different from those who hold that there is no right answer to the attribution problem. I agree that there is no perfect model that has no model prediction errors, but that is not a refutation for statistical modeling.  Statistics is founded on imprecision in data and never afraid of counter examples.

It is an approach of simplification, not of complication and certainly not a proposal to bring in psychology, media-logy or astrology into the picture.  In that regard, it could be a spoiler for the fun party we had so far.

Still, it is really just a claim at this point.  Please come back to read the next post: (TBD)

micro and macro attribution

What is attribution analytics? I was thinking of quoting a definition from Wikipedia but could not find it. So here’s my version of it:

Attribution analytics is a set of data analytics techniques used to determine the proportional contribution of marketing campaigns to conversions.  

There are two general types of attribution problem:  micro attribution focus on attribution of individual level conversion to marketing campaign touch points; macro attribution focus on attribution of aggregated level sale/conversion to marketing campaigns activities. Both types of attribution are trying to answer the question of how much credit each marketing campaigns should get; the data analytics behind them are in fact very different.

Marketing Mix Modeling (MMM) is one type of macro attribution analytics.  

The data used for macro attribution are commonly time-series data:  sale volume over time and marketing campaign spend data over the same time periods (may also include a few periods prior to that).  MMM is typically statistical/econometric model that use marketing spend data, and others, to predict the sale volume. The estimated relationship will then applied back to the data to figure out the proportional contribution of each marketing campaigns to total sale (or total incremental sale). MMM is a well developed analytics technique that has been around and practiced for decades. Its pros and cons are also well understood.

Macro attribution is mostly used for the purpose of evaluating the effectiveness of marketing/advertising channels, budget allocation and optimization.

Micro attribution on the other hand, uses individual level event data.  When individual is exposed to multiple marketing events (touch points), which events should get credit and how much? The explosive growth of individual level event data and cross channel matching methodology/technology is perhaps the major factor behind the rapidly growing interest in micro attribution. There has been a great deal of misunderstanding regarding to what the right analytics framework for this, and what are appropriate analytics tools for it. Some think that it is all about coming up with smart business rules and protocols while others believe that more rigorous statistical modeling is needed.

Micro attribution is mainly used for tactical campaign optimization.  However, when multiple channels/media events are involved (such as email, banner, ppc and seo etc..) its implication on macro level budget allocation is inevitable. Hence the organizational complications come with it as well. 

Individual event level data is clearly a better source for attribution – even if the interest is only at the aggregate level.  Multichannel marketers have been wrestling with attribution problem for a long time.  Many of the innovation are in the data capturing area, such as custom 1-800 numbers and custom URL etc.. Those are important measurement processes, designed to capture micro event level data to avoid having to do the less precision and less effective macro attribution.  However, in most cases only the last touch point is recorded – because it often happens at the same time as conversion event.  It is a situation that fit neither the micro nor the macro attribution problem above.  From the micro attribution perspective, because it has only one (last) touch point, it is simply a dummy LAST touch point attribution rule.  From the macro attribution perspective, because we know which touch point a sale/conversion was attributed, there is no need for any modeling at all.  Hence the popularity of the use of these types of attribution measurement devices.

The calling for using micro attribution is due to its optimization potential for business operation.  If attribution for the purpose of attribution, i.e. to satisfy political or procedural reporting need, then the last touch point attribution is a perfect solution.  However, it fails mainly because of its inability to capture and understand today’s consumer experience – that is multi-channel, multi-touch in nature.  One touch point analytics paradigm is outdated for understanding today’s consumers. The future is multi-touch, the future is micro attribution.  Hence it is extremely important to understand the right analytics framework for micro attribution.

Please come back to read the next post: micro attribution analytics is conversion modeling.

attribution is the hottest topic these days …

Why?  there are a few signs:

If web analytics is your profession or your passion, considering these:

    Avinash Kaushik has several posts in recent month:  Measuring Value of “Upper Funnel” Keywords,   Measure Latent Conversions & Visitor Behavior  and this one from almost a year ago: Conversion / ROI Attribution.

    Eric Peterson  has been busy with webinars and presentations, all over attribution problem.  He use a new metric as a foundation for attribution analysis: “Appropriate Attribution Ratio”.  

We all know that all the major web analytics vendors are working hard trying to figure out the right attribution modeling tool to offer.  My recent meeting with a major web analytics vendor also convinced me; it is all about attribution data service and attribution modeling.

Attribution is also the most talked about topic in SEM/SEO today. I can’t think of a search conference does not have sessions focus on attribution; and every SEM tool makers are tooting its solution for atrribution management. 

From ad-servers: Atlas’s Engagement Mapping and DoubleClick’s Exposure to Convertion Report.

Another variation of name is Multiple Attribution Protocol.

Forrester Research, who lead research in perhaps everything online marketing, has three researches on attribution within the last six months: Attribution by Emily Riley, Search and Attribution by Evan Andrews and Multicampaign attribution by John Lovett et al.  

It is a hot topic and an area not short of differing approaches.  Anyone who is seriously interested should also read Barry Parshall’s fighting post on the right and wrong approaches for attribution modeling.