Vadim, there is a bigger issue than what this example covers. I will discuss it in a separate post.

]]>Huayin, thank you for clarification. I would love to chat with you – we have come up with some very interesting things. Re: your example, I think it is impossible to get around the causality.

If you imagine a standard A/B testing experiment, P(c|A,B) == P(c|A,nB) implies that there is no change in the conversion probability when you replace ads run by pub B with the PSA’s. The same can be told in the language of counterfactuals. This in turn by definition means that the incremental value of publisher B is zero, and there nothing unfair in not giving this pub any credit. Please let me know whether you agree and how we could talk. I really appreciate you writing this blog.

]]>Vadim, thank you for stopping by!

The notations nA is understood as counterfactual. You are correct in pointing out the error I made in the example, really appreciate that! I apologize.

I think if I used the following assumption: P(c|A,B) == P(c|nA,B) == P(c|A,nB), I can hopefully still point out the problem, when we will have C_a == C_b == 0. Also, if I assume P(c|nA,B) == 0.99 * P(c|A,B) and P(c|A,B) == P(c|A,nB), then the simple crediting formula will lead to A getting 100% of credit, suggesting a bit of unfairness to B.

Thanks again!

]]>In fact, there is clear contradiction in the example:

If, as you state, P(c|A,nB) == P(c|nA,nB) and B does not add anything when A is present: P(c|A,B) = P(c|A,nB), then P(c|A,B) = P(c|nA,nB), and the entire campaign has no incremental value.

The conclusion is not that a simple attribution formula should work but that if treated properly, causal-type “conversion model” should not produce obvious problems. How to properly distribute credit is a different matter.

]]>