Why technology professionals are paid so high: Understanding compensation using game theory
Recently, a LinkedIn post (featured above) by Zach Wilson, a Tech Lead at Airbnb, went viral. What was surprising about this post was how quickly his compensation escalated from $170k to $575k. But most conversations about the post were centered around a different aspect of his revelation: how high $575k was compared to what most people made in the USA.
Some reactions to the post were quite bad. A school teacher wrote:
“As a public school special education teacher, this made me physically ill. I don’t even make $50K.”
For a lot of us, the fact has been well-known: professionals in the high tech industry are paid insanely high compared to most people. So I decided to make a post explaining the economics behind this, which I believe should be obvious to everyone at this point in the history of technological innovation. This post is my attempt to make the economics behind compensation, and the related notions of value, reward, innovation, economic progress, as clear as possible.
Now, you could learn about how employee salary and bonus are decided at a financial accounting class (such as Financial Accounting by MITx, which I audited and would highly recommend if you want to learn about valuing companies and similar things). But what I wish to explain here is a somewhat deeper aspect of the whole deal, so that I can give the reader a thinking tool that generalizes to many other important matters. I base this on the theory for which the 2012 Nobel Prize in Economics was awarded.
Partly, the answer to the question of why tech compensation is so high is quite simple: the contribution of these professionals end up making hell lot of money for the companies they work for. As Zach pointed out in reply to comments:
“The data pipelines I wrote for Netflix saved them millions by making smarter AB test rollout decisions. The security infrastructure I built reduced security engineers time to troubleshoot issues a lot. This also saved the company millions.” — Zach
But this is not the whole story. Another aspect that determines compensation is how “replaceable” you are, because, for instance, having a lot of people who can do the same thing as you decreases your bargaining power over your employer regardless of how valuable your service is. Indeed, this is the “supply” part of the economics equation that is behind all this.
Maybe, if you’re somewhat knowledgeable of economics you already know the obvious high-level answer: the supply and demand decides compensation. But to get a deeper look into how exactly these two criteria factor into one’s reward in our particular case (and cases similar to this), and to understand the incentive structure that is necessary for the progress of humanity (and your retirement!), we must turn to game theory.
Innovation and reward
We shall begin with a thought experiment. Suppose you were the first individual in the world to invent the bow and arrow: a staggering improvement over spears. Suppose this made hunting 3x efficient, i.e. a skilled hunter could catch 3x more animals with a bow and arrows as they do with a spear. Now suppose you found one person who is skilled enough to use your new invention. The question is this: if the hunter catches 3 animals using your bow and arrow how will you expect to split them among yourself?
Since the bow and arrows improves efficiency by 3x, without them the hunter would only have caught 1 animal instead of 3. So the contribution of your invention is 2 animals per 3 animals caught. But this doesn’t mean you will get 2 animals as your reward! Instead, a fair splitting will give you 1. This is because the hunter has as much claim over that 2 animal surplus as you do. Indeed, without him there would be 0 surplus. So the surplus is equally divided, and in total you get 1 and he gets 2.
This is fairly intuitive, but can get much complex when additional factors are added such as availability of more than one skilled hunters or alternative inventions. The branch of economics and mathematics that studies this sort of situations is called game theory. In fact, what explains the intuition we used here is Shapley value: a solution concept in cooperative game theory. In the Wikipedia page I linked you will see some scary-looking equations, but here I will try to make it all clear with simple examples.
In order to introduce Shapley value, let us consider the bow-and-arrow example again but now with two skilled hunters who can each either hunt 1 animal with a spear or 3 animals with a bow and arrows. We additionally impose that you have been only able to produce one set of bow and arrows. The first thing we do to understand the situation is to model it as a game involving three players: the two hunters and you the “innovator”. What Shapley value says is all players can expect to be rewarded their average marginal contribution to the surplus.
To understand what this means, let is imagine the three players entering the game one by one. We shall denote this by an ordered list, e.g. (H1, H2, I) representing two hunters and the inventor of bow and arrows respectively. In the ordering (H1, H2, I) the marginal contribution of H1 (“Hunter 1”) to the surplus is 0 since he does not have access to your (I) invention yet (He enters the game before you). On the other hand, in the ordering (I, H2, H1) he enters too “late”: all possible surplus is already created by the “coalition” of I and H2 (Note that there is only one set of bow and arrows). In fact, you can see that the only ordering in which Hunter 1 contributes to the surplus is (I, H1, H2), in which H1 creates value by using your invention of bow and arrows to hunt 2 more animals than he could with a spear. (Note: We can define the surplus as the total number of animals hunted instead of the addition to the number, and the final results we are going to obtain in a moment will be the same).
Now we have seen that in all possible orderings of the three players (which is 3! = 6), the total surplus Hunter 1 contributes is 2. So the average surplus is 2/6 = 1/3. This is the genius method Shapley invented to calculate how valuable each player is in a market game like ours. We see that, by Shapley value, the new expected total reward of a hunter is 1+1/3 = 4/3 for every 3 animals he hunts. Notice that this is less than the earlier 2 animals. On the other hand, you as the inventor of the bow and arrows can expect a reward of 4/3 animals instead of the earlier 1 (Do the calculation as an exercise!).
One can try many variations of this game, such as by including multiple inventors instead of a single one. The central theme one will observe in all of these has two components: (1) The more people there are who have the skills required to create surplus the less will be each of their reward, and (2) the more valuable and unique your contribution is the more will be your reward. The graph below shows how the value of players grow when you vary either the number of “hunters” or “inventors” while keeping the other fixed:
Our conclusion is, of course, entirely obvious and expected. But when it comes to the real world, a lot of people seem to have trouble accepting this for mysterious reasons. Maybe it is because they don’t understand the importance of the incentive structure behind this. So I shall discuss this in the next section.
The math aside, the simple point of emphasis of this section has been this:
If you can invent and innovate in a way that many can’t, and if as a result of this innovation you can enable others to create high value, then you will be rewarded like hell!
And this is exactly high performing engineers do. This is also what tier-S innovators like Elon Musk do. This is why they can command the kind of rewards they obtain. (One might argue “reward” is not an appropriate word because, for instance, the wealth of Elon Musk is tied to the valuation of his companies. But remember that valuation is in turn tied to the expected future cash flow, which is ultimately determined by the strategic games played inside the corporation: as an extreme example, in a world where employees manage to coerce corporations to distribute all their profits to them while also taking ownership of all the non-operating assets, the valuation will drop to 0. There are however better games than the one described above for understanding capital gains, but I avoid introducing them to keep the post simple).
Capital and the role of incentive in progress: paying the pied piper so you can retire
We discussed a hypothetical scenario in which you were the first individual in the world to invent the bow and arrow. Now, if you look at early human history, new inventions didn’t exactly make the inventors billionaires. This is indeed because there was no notion of money to begin with. Early humans didn’t even have a method for food preservation! So even if you collected piles of meat from a team of hunter subordinates, anything more than you and your friends could eat was going to rot. For this reason, it is sometimes quipped by evolutionary biologists that in the early periods you stored your wealth as fat in your friends’ body! (I forgot where I heard it from).
Now given this, there is no surprise that it typically took something in the order of hundreds of thousands of years for early humans to come up with each new innovation. There was no incentive! It was also not the case that humans could fund others to work on “something new”. There could not have been a Bell Labs in the prehistoric period, because there was no capital. Humans basically had to bootstrap capitalism and then let its magic work in order to get where we are now.
So what is the cause of the disparity between outcomes between individuals? It is simple: true innovators — those who can scale up production or improve efficiency to create millions of worth of value— are extremely rare. And any society that wants to progress must incentivize innovators by allowing them their fair share. So given we are a society that progresses, it is unavoidable that there is some disparity. Now there are many who question the very need for innovation and progress. This is indeed absurd but let me make it clear.
A fact we tend to forget is that it is because of the innovators that retirement is a possibility for individuals in our time. It is disappointing to me that we take this for granted. I think it is miraculous that people can retire. A hunter gatherer must give you a funny look if you mention to him this possibility. How do you hunt so hard that you can fund not doing anything for a third of your life?! It is an utter impossibility for him, but thanks to innovations, not for us.
So when you get your annual return on your investment, know that what makes it possible is the constant innovations happening around us. In our time it’s mostly technological innovations that makes compounding possible (The chances are that a significant portion of your retirement fund will be tracking the high tech industry). So you should, by all means, be glad that the innovators in the tech field are highly incentivized.
Answering the question
We now have some idea about how things work in this world. Now let us answer, for instance, the question: why do teachers get so little compensation compared to tech professionals? It is clear that the value teachers contribute is immense, because each of us has to pass through schooling in order to be able to generate any value in our modern world. And there is no schooling without teachers. So why are they paid so little?
Because teachers, as important as they are, are easily replaceable. This is why we are able to educate children at a cost that is much lower than it would be if we had to pay in proportion to the value of education. And in turn, this is why teachers are paid like they are. This is the reality. Whatever you feel about this is up to you.
Indeed it is not just to teachers that this applies, but to any profession. Assuming everything works properly, your compensation is a function of both the value you contribute and the rarity of your skills. Without the value being generated, there is no progress. And without progress, there is no financial freedom. This is a simple and rather obvious fact which all of us should understand so that we can prosper as a society without unnecessary tensions.
I am planning to write more posts like this in the future: demystifying things using math or more general stuff related to technology, computer science, engineering, data science, AI, and finance. If you are interested in reading my content, please consider following me.
