Analytics Strategist

November 16, 2011

Ad exchange, matching game and mechanism design

Over the years, I have learned some interesting things in this new ad:tech industry, particularly around the Ad Exchange and RTB ad auction market model.  I want to share some of my thoughts here and hope you find them interesting to read.

Ad Exchange is not like a financial exchange

The “exchange” in the name is suggestive of a financial stock exchange market, and interesting observations can be made based on this analogy.  However, there are some fundamental differences such as the lack of liquidity in ad impression and information asymmetry.  Jerry Neumann has blogged about this topic profusely;  it is still a topic of great interest today, as seen in a recent article by Edward Montes.

In fact, the differences are easy to understand. The harder part is, like Jerry asked,  If the ad exchange aren’t exchanges, what are they?  or I should add, what should they be like?

Publisher’s preference is the missing piece

The analogy with financial exchange (stock and future) is not a good analogy partly because of its inability to fully model advertiser preference. Not all impressions are of the same value to an advertiser and not all advertisers give the same value to an impression. The commodity auction model as embedded in ad exchange does better, because it allows advertisers to bid based on any form of evaluation – a chance for advertiser to fully express its preference over audience, contextual content and publishers’ brand.

Still, there is a problem for the current auction model: after collecting all the bids from advertisers, it takes the highest bidder to be the winner, as if price is the only thing publishers care about.  In reality, not all bids with the same price are of the same value to a publisher.  Publishers care about brand safety and contextual relevancy as well; in fact, the quality of user experience may mean more to publishers than advertisers!  In sum, publishers care about the quality of the ads above and beyond the bid price.  Unfortunately, the current ad exchanges lack the proper mechanism allowing publishers to articulate their full preferences.  This results in lost of market efficiency and lost of opportunities to remove transaction frictions.  This is a design flaw.

Display marketplace is still far from perfectly efficient and this design flaw does not help.  The recent developments of Private Marketplace are piecemeal attempts to overcome this design issue.  Some market movements in late merge and acquisition attempts can be understood from this angle.

Where can we look for design idea on how to handle this issue?  – paid search and game theory!

The quality score framework from paid search

In many ways, paid search is just like ad exchange, with Google plays one of a few “publisher” roles.   In both markets, advertisers are competing for ad-view through auction bidding;  if we equate audience in display to keywords in search, then the bidding processes is quite the same:  search advertisers do extensive keyword research, look at past performance along side of other planning parameters such as time of the day etc. to optimize their bids;  similarly display advertisers look at the audience attributes, the site and page contents, past performance and planning parameters as they perform bid optimization.

The bidding processes in both markets are similar;  the differences lie in the post-bidding ad evaluation.

After all bids are collected, ad exchange today simply select the highest bidder.  In case of paid search, bids are evaluated on price, ad relevancy and many other attributes.  Google has mastered the evaluation process with its Quality Score framework.  This difference in having a Quality Score framework vs not is not a small thing.  As anyone familiar with the history of paid search know, the quality score framework played a pivotal role in shaping the search industry when Google introduced it around the turn of the century.  The post-bidding ad evaluation for display may just be a critical piece of technology and have potentially significant impact on the efficiency and the health of the display market.

The need for a non-trivial post-bidding ad evaluation calls for an extra decision process (and algorithm) to be added, either at ad exchange or at publisher’s site, or both.  In this new model and with this extra component, ad exchange will send the full list of bidding to the publisher instead of picking a winner based on price alone.  It is then up to the publisher to decide which ad will be shown.  With millions of publishers, large and small, this seemingly small change may be a trigger for more inside this industry where technology is already orders of magnitude more complex than paid search.

The matching game analogy

With full preferences being taking into account for both advertisers and publishers, ad exchange looks less like a commodity marketplace and more like the matching game.  It will be interesting to look at market efficiency from the perspective of mechanism design in game theory, which is another way of saying operational market process.

Matching advertisers with publishers under a continuous series of Vickrey auctions is our setup for the discussion – the best model I can think of that mimic the matching game setup;  it shouldn’t be too surprising to anyone that matching game is an interesting analogy to ad exchange.  As a game theory abstraction of many practical cases, matching game includes college admission and marriage market.  Let’s take the marriage market as an example.

Using a simplistic description, a marriage market involves a set of Men and a set of Women.  Each man has a preference vector over the set of women (a ranking of women);  similarly each woman has a preference vector over the set of men (a ranking of men).  A matching is an assignment of men to women such that each man is assigned to at most one woman and vice versa.  A matching is unstable if there exist a man-woman pair not currently matched to each other but both prefer match to each other than their current match – the pair as such is called a blocking pair.  When there is no blocking pair exist, a match will be called a stable match.

Clearly, stability of a match is a good quality: a stable match is not vulnerable to any voluntary pairwise rematch (translating into ad exchange language, a stable match is one such that no pair of advertiser – publisher currently not matched to each other have incentive to switch and form a new match).  A matching is male-optimal if no two males have incentive to switch partners. Female-Optimal is defined similarly.  A stable matching that is both male-optimal and female optimal looks like an perfect efficient market; we hope to find a mechanism that lead to the unique stable matching as such – something we can then mimic to implement for a future ad exchange model.

Unfortunately, there is no unique stable matching for matching game in general (in this case, having too many good things may not be a good thing).  There is also no unique optimal matching that is optimal from both men and women’s perspective.  We learned that Male Proposals Deferred Acceptance Algorithm, sort of like the current auction process in ad exchange in which advertisers played the male roles, produce Male-Optimal stable matching.  If we switch the role of men and women, a similar algorithm exists that produces Female-Optimal matching.  The two algorithms/mechanisms lead to two distinctly different results.  You can read more about algorithmic game theorycomputational game theory, specifically on matching game and mechanism design if interested.

So, why are we looking into this and what we’ve learned from it?  Below is my translation, or transliteration to be more appropriate, from game theory speak to the ad:tech domain.

We all like to believe that there is an efficient market design for everything, including the exchange marketplace for ads. Our believe is justified for all commodity marketplace by the general equilibrium theory.  Unfortunately, there is no equivalence of a “general equilibrium” or universal optimal stable match for a marriage market, which implies that there is no universal optimal advertiser-publisher matching in ad exchange.  If this is the case, the search for an optimal market mechanism for ad exchange will be a mission impossible.

However, there exist one-sided optimal condition, advertiser-optimal and/or publisher-optimal matching.  It is also easy to find the corresponding mechanisms that lead to those one-sided optimal stable matching.  The auction market as currently implemented in ad exchanges, with the addition of post-bidding evaluation process, is similar to the mechanism leading to advertiser-optimal matching.

The future seems open for all kinds of good mechanism design.  Still, I believe that there is a “naturalness” in the current style auction market.  It is quite natural for the auction process to start from the publisher side, by putting the ad impression on auction, because it is all start with the audience requesting a webpage – a request send to a publisher. It is not easy to imagine how advertiser can set up a “reverse auction” starting from the demand side, within a RTB context. We can never rule out the possibility, and it may work potentially for trading the ad future.

Conclusion:

I am reluctant to draw any conclusions – these are all food for thought and discussion.  I’d love to hear your comments!

August 26, 2011

What’s wrong with BS-ing?

Filed under: misc — Huayin Wang @ 2:27 pm

Professionals who are BS-ing are lemon professionals.  They are fake or counterfeit of real professionals.

What’s wrong with fake product or any counterfeit?  Read on the market for lemon.

Is BS-ing still “not good, but ok” because a lot of people do it (even worse, relying on it for a living)?

I hope we all get serious about BS-ing, it is not a small thing.

January 2, 2010

a decade in data analytics …

Filed under: misc, Web Analytics — Tags: , , , — Huayin Wang @ 10:53 pm

I was reading an article The Decade of Data: Seven Trends to Watch in 2010 this morning and found it a fitting retrospective and perspective piece.  I have been working in data analytics for the past 15 years, so naturally I went searching for similar articles with more of a focus on analytics, but came back empty handed 😦

I wish I could write a similar post, but feel the task is too big to take.  A systematic review with vision into the future would require much more dedication and effort than I could afford at this point.  However, I do have a couple of thoughts and went ahead to gather some evidence to share.  I’d love to hear your thoughts; please comment and provide your perspectives.

The above chart shows search volume indices for several data analytics related keywords over the last six years.  There are many interesting patterns.  The one caught my eyes first is the birth of Google Analytics: Nov 14, 2005.  No only did it cause a huge spike in the search trend for “analytics”, the first day “analytics” surpass “regression”, it become the driving force behind the growth of web analytics and analytics discipline in general.  Today, more than half of all “analytics” searches are associated with “Google Analytics”.  Anyone who writes the history of data analytics will have to study the impact of GA seriously.

I wish I could do a chart on the impact of SAS and SPSS on data analytics in a similar fashion, but unfortunately it is hard to isolate SAS searches for statistics software vs other “SAS” searches.  When limited to the “software” category, it seems that SAS has about twice the volume of SPSS, so I used SPSS instead.

Many years ago, before Google Analytics and the “web analyst” generation, statistical analysis and modeling dominated the business applications of data analytics.  Statistician and their predictive modeling practice were sitting in their ivy tower.  Since the early years of the 21st century, data mining and machine learning became a strong competing discipline to statistics – I remember the many heated debates between statistician and computer scientists about statistical modeling vs data mining.  New jargons came about, such as decision tree, neural network, association rule and sequence mining.  To whomever had the newest, smartest, most math grade, efficient and powerful algorithm went the spoils.

Google Analytics changed everything.  Along with data democratization came the democratization of data intelligence. Who would’ve guessed that today, for a large crowd of (web) analysts, analytics would become near-synonymous with Google Analytics and building dashboard, tracking and reporting the right metrics the holy grail of analytics?  Those statisticians may still inhabit the ivy tower of data analytics, but the world is already owned by others – the people – as democracy would dictate.

No question about it, data analytics is trending up and flourishing as never before.

comments?  Please share your thought here.

March 29, 2009

where is the “deep dive example” of attribution analytics?

Filed under: misc, Random Thoughts — Tags: , — Huayin Wang @ 2:23 pm

First of all, this is Sunday morning. I am not going  to write anything that requires siginificant works from my thinking mind — but …

I have been wondering, for the last couple of days, why the “deep dive example” piece that I promised hasn’t come out yet. The reason is just getting too complicated: lack of motivation, busy at work, shifting focus on other things … and last but not least: twitter!

To be sure, I do not mean to join the fashion of crediting / blaming twitter for everything, including the latest recession. I just want to point out a plain and simple fact, that I have been looking, reading random things from twitter or related to twitter so much that it EATS up most of my FREE time; and worse yet, I notice some subtle changes in me, a little ADD like symptom; it erodes my concentration, cuts my usual chain of thoughts into short pieces and stires up my urge to surface, to verbalize anything and everything. 

It is a little sad to see how twitter is winning over the world of advertising, PR, News and celebrites; but it is truly scary to hear story about twitter’s invasion to education

I need to ban myself from twitter for a little while and see if the damage is irreversable.

March 14, 2009

reading notes : 2009 Digital Outlook

Filed under: Advertising, business strategy, misc, reading — Tags: , — Huayin Wang @ 6:04 pm

With six hundreds (in 5 days) tweets from readers of the 180 pages 2009 Digital Outlook from Razorfish, this report is certainly captured the attention of many working in marketing/advertising. It is an exciting read and I will share a couple of my notes here.

Clark Kokich’s introduction sets up the story line really well.  

The opening paragraphs went to the key point directly.

 “I spent the first 30 years of my advertising career focused on saying things. What do we need to say to persuade people to buy our product or service? How do we say it in a unique and memorable way? Where do we say it? How much will it cost to say it? How do we measure consumer reactions to the things we say to them?”

Now, after 10 years in the digital space, I find myself spending my time talking to clients about building things. What do customers need to make smart decisions? What applications do we need to build to satisfy that need? Where are our customers when they make a decision?”

He then described the new role agency need to play: ” .. it’s about the actual role they should be playing in setting business strategy, designing product and service offerings, delivering service after the sale, creating innovative distribution channels and developing new revenue models.”

These are great insights.  Ad agencies are expert of creative messaging – “saying things”; the new challenge is about shifting the focus away from that and go beyond. This is a tremendous challenge indeed, one that would require new skills and “deep collaboration between creative, technology, media, user experience and analytics”.

February 26, 2009

micro attribution analytics is conversion modeling

If you are surprised by the title statement, you are in the majority.  

This is actually a very strong statement and I did not make it lightly. It is saying that micro attribution is an area of data analytics that can be defined and studied with rigorous statistical methodologies. In short, it is more of a science than art or common sense.  Micro attribution problem is more like a response modeling or risk modeling problem than the problem of finding out a fair rule for distributing year-end bonus.  

Does this sound the same as how others describe attribution problem and solutions? 

It is certainly different from those who think the solution to attribution problem is about tracking.  Tracking is important because it provides you the data, but in itself they do not tell you what factors or customer experience have more or less influence on conversion.

It is also different from many who think about “last click”, “first click” etc. when they speak about attribution models.  Those are not data analytics models or statistical models that I was referring to.  One is about intuition-based smart rule; the other is about data-driven behavioral modeling.  The smart rule vs. modeling debate was long over in Direct Marketing, but it is just beginning in web analytics and online, right here in the micro attribution problem.   

It is also different from the many who think this is all about metrics (because of the claim that there is no right solution to attribution :).  It is not about averaging the attribution of first-click and last click. It is not about using engagement metrics as a proxy either. 

It is definitely not the same as those who think we need to wisdom-of-the-crowd type of solution.  The percentage of you who think early keywords should get 15% attribution for “assist” maybe right, but it has no bearing to me.  I do not believe that there is an average truth in any of these, for reasons that I do not believe one retailer’s offer-X response model shouldn’t be used for a loyalty campaign of a telecom company.

It is categorically different from those who hold that there is no right answer to the attribution problem. I agree that there is no perfect model that has no model prediction errors, but that is not a refutation for statistical modeling.  Statistics is founded on imprecision in data and never afraid of counter examples.

It is an approach of simplification, not of complication and certainly not a proposal to bring in psychology, media-logy or astrology into the picture.  In that regard, it could be a spoiler for the fun party we had so far.

Still, it is really just a claim at this point.  Please come back to read the next post: (TBD)

February 24, 2009

get out of group think

Filed under: Business, Random Thoughts — Tags: , — Huayin Wang @ 5:24 pm

Sometimes our over-confidence of our own expertise may prevent us from finding the right solution for a problem.  After all, we do not know what we do not know.  The recent web 2.0 movement that arms us with all the capabilities to listen, publish, and connect to our peers and people “like me” may have actually exasperated the situation.

The new challenge now is: how to get out of groupthink.  Popular opinion is not necessarily the right thing to spread around, and popular support is not a confirmation of getting something right.

Sometime it helps to step back from our narrow vision and awareness and immediate interest and local network of friends, to realize that the world is bigger than we thought and all the smart people are not in our profession and solutions to our problem may already be there in the open. 

February 20, 2009

The desire to last forever …

Filed under: Business, Random Thoughts, spirituality, Technology — Tags: , , , — Huayin Wang @ 5:29 pm

and to do things that could last forever .. the desire that used to be a synonym for ego is perhaps one of the most important, and subtle, force for why we do not see reality as it is when it is right in your face.  It is perhaps the one force that comes so natrually for us in preventing us from going with the flow of nature.

I used to see this when it comes to spiritual matters, not knowing that this is so applicable to business as well.

December 16, 2008

migrate from blogger to wordpress

Filed under: misc — Tags: — Huayin Wang @ 4:56 am

I have just consolidated all posts from 6 of my prior bloger blogs to wordpress – feeling light and fit 🙂

oh ya.

August 9, 2008

Small is beautiful

Filed under: misc — Tags: , , — Huayin Wang @ 5:05 pm

a powerful little webserver written in Rebol

« Newer PostsOlder Posts »

Blog at WordPress.com.