Q: I will begin with this question, what you do NOT want to talk about today?
A: I do not want to waste time on things that most people know and agree with, such as “Last Touch Attribution is flawed”
Q: Why is attribution model such a difficult challenge, that after many years we still seem to just begin scratching the surface of it?
A: No idea.
Q: Let me try a different way, why is it so hard to build an attribution model?
A: It is not. It is NOT difficult to build an attribution model – in fact, you can build 5 of them in less than a min: Last Touch, First Touch etc… J It is difficult to build good attribution modeling – a process that produce methodologically sound attribution model.
Q: “Attribution modeling” – is this the kind of tool already available through Google Analytics.
A: No. Those are attribution model specification tools – “you specify the kind of attribution models to your heart’s content and I do reporting using them”. They do not tell you what IS the RIGHT attribution model. An attribution reporting tool does not make an attribution modeling tool.
Q: “Methodologically sound” – that seems to be at the heart of all attribution debates these days. Do you think we will ever reach a consensus on this?
A: Without a consensus on this, how can anyone sell an attribution product or service?
Q: On the other hand, isn’t “algorithmic attribution” already a consensus, that everyone can build on it?
A: What is that thing?
Q: All vendors seem to take the “algorithmic attribution” approach, possibly adding additional phrases, such as “statistical models” and data-driven etc. Isn’t that sufficient?
A: How? They never show how it works.
Q: Do you really need to get into that level of detail, the “Black Box” – the proprietary algorithm that people legitimately do not release to the public?
A: There is no reason to believe that anyone has a “proprietary algorithm” for attribution. Unlike predictive modeling, a domain of technology that can be “externally” evaluated without going inside the Black Box, attribution modeling is like math, a methodology whose validity needs to be internally justified. A Black Box for attribution sounds like an oxymoron for me. You do not see people claim that they have a “proprietary proof” of Fermat’s Last Theorem. (Ironically, Fermat himself claimed the proof on the margin of a book without actually showing it, but everyone knows he never intended it to be like that).
Q: Why then do people claim to have but do not show their algorithmic and/or modeling approach?
A: It is anyone’s guess. I see no reason for that; it hurts themselves and it hurts the advertising industry, particularly online advertising industry. I suggest, from today, every vendor should either stop claiming that they have proprietary attribution modeling/model or get out of the “Black Box” (the new empire’s cloth?) and prove the legitimacy of their claim.
Q: Ok, suppose I say, I build a regression model to quantify which channels impact conversion and by how much, then calculate the proportional weights based on that and partition the credits according to the proportions. What would you say?
A: How?
Q: You are not serious, right? I am giving you so much details – how much more do you want?
A: The program and process sounds like it will work, and it is quite CLEAR that it is going to work to non-practitioners’ eyes. But you know and I know that it does NOT work. Having built conversion models does not solve the attribution problem. Attribution problem comes down to the partitioning of credit, i.e. how much of the conversion credit to be partitioned and how much given to each channels. The logic has to be explicitly presented and justified. The core challenge has been glossed over and covered up, but not solved.
Q: Please simply it for me.
A: There is no automatic translation available from conversion models to attribution models – the process of doing that, which is attribution modeling has to be explicitly stated.
Q: You defined attribution problem as partitioning credit to channels – are you talking about only Cross-Channel Attribution? If I want to focus only on Digital Attribution, or even Publisher Attribution only, is what you said still relevant?
A: Yes. I am talking about it from data analytics angle – you can just replace the word “channel” with others and the rest will apply.
Q: Ok, what if the conversion model I use is not regression, but some kind of Bayesian models?
A: It does not matter. It can be Bayesian, Neural Net or a Hidden Markov Model. As long as it is a conversion model. The automatic translation is not there.
Q: Does it matter if the conversion model is predictive or descriptive?
A: It should be a conversion model – there are multiple meanings of “predictive model”; it is essentially predictive models, but need not handle “information leaking” type of issues as a predictive model should.
Q: Does it need to be “causal” model, and not a “correlational” model?
A: Define causal for me. Specifically, do people know what they mean by “correlational” model? Do they know multivariate models and dependence concepts?
Q: I assume we know. Causal vs. correlational are just common sense concepts to help us make the discussion around “model” more precise …
A: But neither are more precise concepts than statistical modeling language. Even philosophers themselves begin to use statistical modeling language to clarifying their “causal” framework …
Q: Now I am confused. Where are we right now?
A: We are discussing statistical models and attribution modeling …
Q: Ok, should we use statistical models when we do attribution?
A: We have to. Quantifying the impact of certain actions on conversion should be the foundation for any valid attribution process; there are no more precise ways to do that than developing solid statistical models for conversion behavior!
Q: Not even experimental design?
A: Not even that.
Q: But what is the right statistical model? Some types of regression models or some Bayesian models or Markovian models?
A: It does not have to be any one of them, and yet, any one of them may do the job.
Q: If that is true, how can one justify the objectivity of the model?
A: A conversion model provides the basis for what reality looks like – to our best knowledge at the moment. There can be different types of statistical methodologies to model the conversion behavior, and that does not create problems with the objectivity of the model output. We have seen this in marketing response models, where the modelers have the freedom to choose whatever methodology (type of models) they deem appropriate and yet it does not compromise the objectivity of its results.
Q: But attribution is different; when building marketing response models, what is important is the score, not the coefficients or any “form factors” of the model. In attribution, those form factors are central, and not scores, to derive the attribution formula.
A: That’s exactly the problem that needs to be corrected. Attribution formula should NOT be built on the “form factors” of the conversion model, but rather on the scores of the conversion models!
Q: Explain more …
A: If you can’t claim that linear regression model IS the only right model for conversion behavior, you can’t claim those regression coefficients, the “form factors” of the regression models, are intrinsic to the conversion behavior. Thus, any attribution formula built on top of that cannot be justified.
Q: And the conclusion, in simpler language …
A: Conversion model is needed for attribution, but attribution model is not the conversion model. Attribution model should be built on top of the “essence” part of the conversion models, i.e. the scores, and not the form factors. Attribution modeling is the process of translating conversion modeling results to attribution model.
Q: What is that saying about the offering from current vendors?
A: They often tell us that they build conversion models, but reveals nothing about their attribution modeling methodology.
Q: What if they say that, they are hiding their proprietary attribution technology in Black Box? Are they just covering up the fact that they have nothing in there, and they do not know how?
A: Anyone’s guess. The bottom line is, anyone claiming anything should acknowledge the right to doubt from their audience.
Q: It is common to see companies hiding their predictive modeling (or recommendation engine technology) in Black Box … why not attribution?
A: Predictive modeling, or even recommendation modeling, are things that can be externally tested and verified. You can put two predictive model scores, and test out which one has more predictive power without knowing how they build the models. Attribution modeling is different; you have to make explicit how and why your way of allocation is justified – otherwise, I have no way of verifying and validating your claim.
Q: We are not in the faith business …
A: Amen.
Q: Ok, big deal. I am an advertiser, what should I do?
A: Demand anyone who is selling you attribution products/services, to show you their attribution stuff. It is ok if they hide the conversion model part of it, but do not compromise on the attribution modeling.
Q: I am in the vendor business, what should I do?
A: Defend yourself – not by working on defensive rhetoric, but by building and presenting your attribution modeling openly.
Q: If I am an agency, what should I do?
A: Attribution should live inside the agency. You can own, or rent it; you should not be fooled by those who like to make you think attribution modeling is a proprietary technology – it is not. Granted that you are not a technology company, but attribution modeling is not a proprietary technology. If you have people who can build conversion model, you are right up there with those “proprietary” attribution vendors.
Q: If attribution modeling becomes an “Open” methodology, what about those attribution vendors? What they will own and why advertisers and agencies wouldn’t build themselves?
A: That’s my question too J
Q: Are vendors going to be out of business?
A: Well, they can still own the conversion modeling part of it … and there are still predictive modeling shops out there, in business …
Q: Somehow, you sound like you know something about this “open secret” already J Can you share a little on that?
A: Can we talk tomorrow? I need to leave for this “Attribution Revolution” conference tonight …
Analytics Strategist 
Leave a comment