This week, Evgeny Kanashevsky, an economist from Zalando, Economics Phd of the University of Pennsylvania, spoke on our social networks.
At work, Zhenya is engaged in establishing causal relationships in online advertising using experimental and quasi-experimental methods and machine learning models.
We share with you the transcript of the broadcast.
My name is Evgeny Kanashevsky. Today we will talk about what IT companies pay economists for, how economists differ from ordinary data scientists, and answer interesting questions like “how much is a human life?” That economists do.
First, I'll introduce myself. I am currently working as an economist / data scientist for a large company Zalando. It is an online store that sells clothing, footwear, cosmetics in 16 European countries and plans to expand into new markets. Before joining Zalando in 2020, I was doing my PhD in Economics at Pennsylvania State University. I began to be interested in economics long before that, when I studied at MIPT and then also at the Russian School of Economics.
Before going to my PhD in Economics, I worked for 2 years in a contextual advertising agency in Moscow; really wanted to know more about what the economy is and how it works. To quench my thirst, I ended up taking a PhD. Now I hope to share this knowledge with you. I hope you will be interested and we will understand why business needs economists.
First, let's try to debunk some of the well-established ideas about economists, which, based on my experience, are present in Russia. Many of you have probably taken economics courses at university, and you remember that you went through market equilibrium, supply and demand curves. Some of you, perhaps, have gone through a different kind of macroeconomic balance - what does the GDP consist of, how to calculate it, what will happen to GDP if the state spends so much money, what taxes are collected from the population. You may have answered such questions in an economics course at university, and few of us have progressed further.
The conventional wisdom about economists is that they work in academic institutions (NES, HSE) or international organizations (IMF, World Bank, Organization for Economic Assistance and Development), and there they help countries break out of the poverty trap and carry out economic reforms. They can work in government organizations - central banks, ministries of economic development of their countries. If they are super-ambitious, then they can try to become an economic adviser to the President of the United States. The stereotypical view of economists in industry is that in industry they work in banking and finance.
I've often come across this default view of economists - that they only do these things. In fact, economics is much broader. She deals with all kinds of interesting people-related issues and that's why businesses hire them.
Let's see what questions economists are asking. Answers are given to these questions, lectures are given on them at the courses of economics, in the bachelor's degree.
One of the fundamental questions economists are looking for answers is: “Why are some countries rich and others poor? This is a question like "what came before - an egg or a chicken?" What came before: economic growth or democratic institutions? The countries of the Anglo-Saxon world are so rich because they have established democratic institutions (separation of powers, independent legislature, independent judiciary, separate from the executive branch) - and that is why they became so rich later, were able to grow economically? Or is it the other way around: these countries grew economically, and then, having become rich, decided to establish democratic institutions? This is a non-trivial question, it is very difficult to answer it. This question is essential for many countries: for example, China is a country whose economy is growing very rapidly.Will it be able to grow further at the same pace, and does China need these very democratic institutions, or will China be able to continue to grow in a sufficiently undemocratic environment?
This question has been relevant before. In the first half of the 20th century, the USSR developed very quickly, possibly faster than the countries of the Western world. Economists wondered if the USSR could continue to grow like this, a question that was extremely important strategically during the Cold War. Further history showed that the USSR was unable to continue its economic development, and the consensus among economists is that this was impossible due to the lack of democratic institutions and market mechanisms of the economy.
These are the questions that economists ask. These questions are complex because there are not so many countries in the world; it is impossible to look into a parallel universe where the USSR would have developed using market mechanisms of the economy and democratic institutions. In addition to questions on a global scale, economists ask more everyday questions that have good practical value. For example - how to profitably sell radio frequencies in several US states. Imagine: you own a certain radio frequency range (you represent the state). You want to sell them in the most profitable way - so as to bring the most money to the budget. The question arises: how to arrange a mechanism by which you will do this? Economists suggest using auctions to sell frequencies; it seems logical if you rememberhow works of art are sold, for example. But then the next question arises: how to arrange the auction in the best way in order to get as much money as possible into the budget?
This sounds abstract - but think about things like collusion, for example. Buyers at the auction can come to an agreement behind your back and say: let me buy these frequencies, and you - these, and you will place a low bid on some bands, and I - on others. And then they will pay as little as possible. The task of economists is to prevent such collusion and provide the state with as much money as possible in the budget, which will be used for the benefit of citizens.
Another example of an interesting question economists are asking is, "How do you set up a kidney transplant market to save as many people as possible?" The word "market" does not sound very good morally here unless you are an economist; Most likely, this word here is associated with the illegal market, with the sale of kidneys, which is a crime. But when I talk about the "market" now, I mean that there is a demand for kidneys: there are people who need kidneys, and there are people who are willing to offer their kidneys - because a person can live normally with one kidney.
Imagine a hypothetical situation: your wife (husband, girlfriend, boyfriend, parent) needs a kidney and you are ready to donate it. But the point is, compatibility is needed. Your loved one may simply not be suitable for your own kidney. What you can do: You can find the same pair of people (maybe from another city, region or country) who are incompatible with each other, but at the same time your kidney may fit a needy person from another pair - and vice versa. And thus, two people who previously did not have a chance to get a kidney, or they had to wait for a very long queue for a kidney (and this can take months or years), this chance appears due to the fact that we found two couples who are ready to exchange kidneys, in this kidney "market". We helped people.
Of course, the situation can be much more complicated. Maybe you have to turn a chain of very many moves - to attract a lot of pairs of people in which, when they are all together, it will be possible to find such exchanges so that all people in need from these pairs will receive a kidney. Only two pairs are optional. You can imagine an example with three pairs: a healthy person's kidney from one pair matches a needy person from the next pair, and so on. Such cycles can go up to 70 pairs, in fact. This is an example of a case in which economists devised an algorithm to organize the “market” and help people with kidney transplants. For this, she received the Nobel Prize in Economics in 2012 by Stanford University professor Al Roth.
Another interesting question that is not entirely obvious from a moral point of view is "how much is a human life." It is not clear from a moral point of view how one can evaluate it at all - after all, human life is priceless. But we can think like economists - pragmatically and practically, where we may need to evaluate human life.
Imagine you want to spend money to improve roads in Russia. Every year 15 thousand people die on the roads, approximately. You think how much you are willing to invest in improving roads - from the point of view of the state. A good way to think about it is how many lives would be saved by such an improvement in roads. For a good estimate of the costs of building roads and the benefits that can be obtained from building roads, just the cost of a human life is needed.
These estimates have been made; they were made for the United States, because there is good data there at very different levels. These estimates suggest that the cost of a human life ranges from $ 4 million to $ 9 million. Such assessments, of course, need to be projected onto Russia - we must measure how the price of human life differs in countries where people earn less and bring less value. This is a very rough estimate, but there are rules for recalculation. And, as rude as it sounds, the cost of human life in Russia is less than in America, simply because the economy in Russia is less productive. But, nevertheless, you can get some estimates and with the help of this solve practical issues - for example, how much are we willing to spend on improving road safety in order to lose fewer people.
So, I hope, it became roughly clear: economists are engaged in a variety of issues, but the common thing in these issues is that we everywhere study not objects of the material world (like physics, for example), but living people. Economists study the incentives of people - how people work, what decisions they make.
Where do economists work - in addition to international organizations, scientific institutions, government agencies and banks? Economists work in industry consulting, mainly in various consulting firms. More and more work in large and small IT companies. Just to name examples of the biggest companies - Google, Amazon, Uber, Facebook; The company I work for (Zalando) is an e-commerce company that has its own team of economists to solve problems about human behavior. Netflix is an example of a company that is not very large in terms of staff, but it also hires economists who solve very interesting problems.
The market for companies that hire economists is not limited to these examples. But big companies that hire a lot of people in principle are hiring more and more economists.
Then the question is: why do IT businesses hire economists? The monosyllabic answer is to offer the customer value. That is, we proceed from the assumption that businesses want to offer value to the customer, and for this they must understand how customers make decisions (to summarize, how people do it). This is what economists have been doing for the past 100 years, at least. They study how people make decisions in a wide variety of situations. Not only in the classic markets that you might think of from the introductory economics courses you have had, but in a wide variety of markets and in a wide variety of situations.
To answer in more detail the question "why IT firms are recruiting economists," we will analyze three points. First, why are economists researching how people make decisions in a wide variety of situations - how is it that they do it and ask strange questions ("what is the value of human life?", "How to arrange the market for kidney transplants?"). Secondly, we will analyze 4 types of tasks for which businesses need economists. Finally, we'll discuss why businesses need economists and not ordinary data scientists: what are the differences between them.
Point One: Why Do Economists Answer Questions About Human Behavior? A little history will do the trick here. Economists began studying market equilibrium as early as the 18th century (at least). They started asking questions: how the markets are arranged, how the demand is formed, how the supply is arranged. They were thinking about classic markets - let's imagine a cotton or grain market, for example. It had nothing to do with such "crazy" questions like the value of human life. However, in these classical markets, the equilibrium - the equilibrium price, the quantity of goods sold - depended on the decisions made by people and on what incentives people were guided by when buying and selling goods. When economists studied market equilibrium, they had to develop sufficiently advanced mathematical methods to better answer the questions that arose.By the 20th century, economists and mathematicians were developing mathematical methods to answer questions related to market equilibrium.
Maybe you've watched the movie A Beautiful Mind - this is a great example. This is a film about John Nash, a mathematician who won the Nobel Prize in economics for his articles on game theory. It is originally a mathematical concept, but economists are very active in applying the work of Nash and other mathematicians - and in general, very advanced mathematics - in the study of the market. This is partly why economists are capable of solving complex and bizarre questions.
At the beginning of the 20th century, statistics abstracted from the study of cause-and-effect relationships. Statistics deliberately began to care only about the distributions of data, about conditional distributions, about how the data is arranged, about how, knowing one variable, to predict another (i.e., say what is the probability that one event will happen while observing another event ). Statistics abstracted from such questions as "how does the presence of democratic institutions affect economic growth in a country in the next 100 years?" or “in one country people grow crops that have a very large plantation size and are therefore profitable to work with bonded labor - how will growth in this country differ from growth in countries that grow other crops - for example, cereals - with a smaller plantation , which is why there are many farmers in these countries,and slave labor in them would not be as profitable as on rice or sugar cane plantations; how will these facts affect the future economic growth of these countries? " (under the influence is meant a causal relationship).
Economists had to develop their own methods - econometric methods - to answer precisely the questions of cause and effect, rather than simple questions about correlations or conditional probabilities. This happened at the beginning of the 20th century. We rewind the last quarter of a century and understand that economists, since the 18th century, have been developing mathematical methods, rigorous mathematics; economists since at least the twentieth century have developed sophisticated statistical methods that help more confidently answer questions of cause and effect. And we understand that economics is the science that helps to answer questions about human behavior, and to do it as strictly as possible - using the highest possible standards for establishing causation.
What is special about human behavior and its study? The point is that we cannot experiment here. We cannot clone a person and check what would have happened to the exact same person who would have entered another university, for example, and received a lower quality education - how would this affect his future earnings. Therefore, economists had to develop mathematical and statistical methods to understand exactly how people's decisions affect their future and what will happen to them.
This brings us to why IT companies are hiring economists. I have identified 4 large areas where economists work in IT companies; this is not a complete list - economists do other things as well, but these areas are already formed.
The first area is the assessment of the demand for goods or services. Many businesses can be thought of here, but I will give the most striking examples. Amazon is one of the largest online stores (I can't say who is the largest - Amazon or Alibaba), a huge company with a huge staff, which is very important to assess demand: how many products will be sold, how price changes will affect sales. Uber is a paradise for economists because it has a lot of data: you can estimate the demand for taxis, you can answer questions like “how to set prices for Uber during rush hour in midtown Manhattan, how should these prices differ from prices at other times in this same location; how to make Uber receive revenue, and at the same time people can order a taxi and a taxi will come to them ”. That is, we see that there are many people here, and they have different incentives:the client has an incentive to take a taxi as quickly and cheaply as possible; Uber and a taxi driver want to make money.
A second example of a problem is auctions. Here the most striking examples are Google and Yandex, online advertising markets. Every time you type something into a search, there is a small auction and someone pays to show you an ad. From 70 to 90% of the revenue of the parent company of Google - Alphabet - comes from these auctions, that is, Google lives on advertising (Yandex too). Economists are working on how to design these auctions so that businesses don't spend extra money on advertising and users get relevant ads. All of these incentives are factored into the design of auctions by economists.
The third layer of tasks is experiments and quasi-experiments (A / B tests). A / B testing is done routinely and continuously in large companies. Economists here design systems for A / B testing; in fact, it's easy to do it, it's hard to do it well. How to do A / B testing correctly, measure results, spend a minimum of resources and get the most benefit - this is all done by economists at companies like Google, Facebook or Zalando. Netflix is well known for its experimentation and A / B testing platform.
The fourth layer is measuring downstream impact: how a product affects, say, the lifetime value that you get from one customer. Imagine you want to estimate how much a subscription service like Amazon Prime should cost: how much to charge subscribers to maximize subscriptions so people don't unsubscribe? To do this, you need to estimate how much value Amazon Prime brings to the customer - not in a day or a month, but over the years. And such tasks, when there is long-term planning and forecasting, long-term assessment, are solved by economists for business.
We see these tasks, and everything seems to be fine, but the question arises: how do economists differ from ordinary data scientists, because they also solve similar problems? Why are economists special, what is their comparative advantage? Let us explain with the example of the "ladder of causality". She has three levels. Using the example of three questions in the context of an offline store, I will try to explain why businesses need economists.
Imagine that you are the owner of a typical corner store and you want to predict the demand for beer and chips. You want to replenish the stock of goods for sale for the next week - this requires a demand prediction. Imagine the first case: the next week does not differ significantly from the previous one. Then, based on the history of previous purchases, we can build a model that predicts the demand for chips and beer for the next week. The question you are asking is, "How much beer and chips will people buy at current prices in our store, or other similar stores?" The phrase "at current prices" is very important here - that is, we are looking at the status quo.
We'll call this a Level 1 question: we don't care why the demand for goods was the same last week. A good predictive model we’ll build doesn’t have to explain why people bought so much beer and chips — it must predict the future with reasonable accuracy based on past data. Examples of such models: recognition of visual images (to distinguish a cat from a dog in the photo), detection of spam messages in e-mail, determination of a potential client's credit rating. In predictive models, cause and effect relationships are not important; based on the data you already have, you want to make a prediction for a new object - a picture, message, client. This is where machine learning techniques work great. We can teach advanced techniques and make very good predictions.
Let's move on to the second question and the second level of the ladder. Let's say we wanted to change the price of beer for the next week. The question we are asking is: “What will happen to beer sales next week if the price is raised? And what will happen to the sales of chips, which are possibly related to the sales of beer? " Why can't we just use cases from the past when the price of beer changed in the past? Precisely because, perhaps, in the past, the price has changed for other reasons. Now we want to raise the price ourselves; in the past, maybe all stores automatically raised it at the same time, due to changes in excise taxes. Or it could have happened for other reasons. But the predictive model is not able to know these reasons, and here we cannot simply use machine learning to tweak the price of beer, increase it somehow and predict.what will happen to the sales of beer and chips. Machine learning models don't work in terms of cause and effect. They just see the correlations that have been in the data in the past and see conditional probabilities: for example, the probability that a person will buy chips if they have already bought beer.
What to do in this case? We can do an experiment. We can only raise the price of beer in a separate store in our chain (suppose we have a chain of stores) and see what happens in this store next week with sales of beer and chips - and in other stores as well. Here we are doing a basic A / B test. How to carry it out correctly - these are the tasks that economists set in companies. This is the second level of questions about causation. Here, in contrast to the first level, we want to know what happens if we make a change - that is, we can feel the cause and effect, the change we are making, and the end result (sales of beer and chips).
The third level is even more difficult. Let's say we raised beer prices and saw that sales of chips fell. Why did sales of chips fall? Is it because beer has become more expensive - or maybe because we now sell corn sticks cheaper than chips? So we took it, didn't think, and changed the price of sticks. Or we simply cannot control everything: we have squids, pistachios, corn sticks, and the price for them also changes. It is difficult for us to conduct an experiment similar to the experiment from the second level: prices for a lot of goods change, some of them are replaced, some are supplemented by chips (like beer). And then we are faced with this question: "What would happen to the sales of chips this week if we did not change the prices of beer?" Here we want to understand what would happen if all other changes took place, except for the change in the price of beer.You need to think about a parallel reality, in which it was so, and somehow construct it.
This is a third level question: what would happen if. Our predictive model is incapable of answering it, incapable of constructing a parallel reality. This question often cannot be answered by experimentation. This is where economists are needed: over the past half century, they have been studying human behavior and asking just such questions - "what would have happened if this had not happened."
An example of such a question: in the 70s of the XX century in the Basque Country - a region of Spain - there was a strong separatist movement, there were many terrorist attacks. Economists wanted to assess what would have happened to the Basque Country - how it would have developed economically - if these terrorist attacks had not happened. How do economists do it? They build a parallel reality in which they build an imaginary peaceful Basque Country, and watch how it develops, what economic growth it has. The results suggest that terrorist activity in the Basque Country has reduced its economic growth by 10%. Here we can understand that these are the questions the economist has been asking for a long time, and they have developed many methods for answering such questions. This is why they are valuable to business.
So, we tried to explain why economists are valuable to business and how they differ from ordinary data scientists. The last question of our webinar is: "Why will businesses hire more economists?" We will answer with an example of an online advertisement.
Companies spend huge amounts of money on advertising - online and offline. In a nutshell, no one really knows exactly how advertising works. The status quo among marketers is that advertising works. Many economists have a different opinion, they believe that in 90% of cases it does not work - but the status quo is that it works "somehow" (and no one can say how exactly). The question "how exactly does advertising work" is exactly a question of 2-3 levels of the ladder of causality, and it is impossible to answer it well with the help of predictive models.
More specific examples of questions that businesses are asking: "What will happen to sales if you spend twice as much on advertising next month?" Of course, we can conduct an experiment - spend 2 times more on advertising. But how to understand then that advertising brought new sales? This could be due to seasonal sales in the same month, or due to other promotional activity. A company can have many advertising channels, especially a big one like Zalando, and budgets can vary for many of them. Let's say we want to increase the budget in one of them and understand what is happening with sales - but how can we understand what exactly this change led to additional sales? What if sales are deferred - the user saw an ad, but made a purchase only after a month?
The next question is how to answer such complex questions, given that users in Europe and other countries are less and less willing to share their data - cookies and others? It is increasingly difficult for businesses to track users. Tracking users outside of your site on an individual level is not possible in most cases; you cannot know what the user was doing on the platform where your ad was - Facebook or Instagram. Businesses ask these questions, and they hire economists for these questions.
At the end of our conversation about economists in business, I want to tell you an Italian parable. The owner of the pizzeria had two sons, and he sent them out to hand out coupons for free coffee - just to attract more customers to his pizzeria. And he arranged a competition between the sons. He said: I give you blue and red coupons to distinguish from whom the buyer came with the coupon, and I will buy a soccer ball to the one with more coupons at the checkout. One son went outside to hand out coupons to attract visitors. And the other son stayed at the pizzeria and, while his father was making pizza in the kitchen, began distributing coupons to people in line who were about to buy pizza. The question is: which of these sons won?
In fact, this parable also works in the advertising business. Does the business spend money on people who would buy the product you are selling anyway? Or is he spending money on those people whom he will really attract through advertising, and who in a parallel reality, where there is no advertising, would not buy your product? And now, when we talk about these parallel realities, when we need to think about and construct them, this is where we need economists. Therefore, the business is recruiting them.
I'll tell you what to do if you want to know more about economics. I can recommend books and podcasts. An excellent book to start with is the book of the former rector of NES Sergei Guriev, Myths of Economics. From there, I took an example today about the cost of human life. You can read in this book about the myths that exist about what economists do and how the economy works, and about debunking these myths. You can read about why some countries are rich and others are not, what is the consensus among economists and how much work they have done studying the history of countries, you can read in the book "Why Nations Fail". About more micro-level tasks, when we look not at countries, but at the behavior of individuals, and about how economists differ from normal data scientists who are engaged in prediction or classification tasks,you can read in the book "Mostly Harmless Econometrics" - this is about econometric methods, special statistical methods for studying cause and effect relationships. The Book of Why also focuses on causation, which is where I took the concept of a three-level ladder of cause and effect; it does a great job of explaining why businesses need more and more economists.
I also wanted to share podcasts and lectures. There is “Economics by ear” in Russian - an excellent podcast from VTimes. Also, the podcast "Economics and Life" is on the NES YouTube channel; in general, you can find many interesting lectures on it.
If you want to read or listen to in English, I can first recommend Freakonomics - this is the most popular podcast on economics, very interesting. It is from this that you can understand how crazy questions economists are ready to ask - and answer them using good mathematical and econometric methods. The podcast writers also have a blog on their website and some good books that I recommend, too.
Q: now in Russia the salary ceiling for economists is $ 1000 per month - what does it take to get a job at Google?
In fact, in order to get a job at Google as an economist, you need a PhD in economics - Google can only afford to choose from a PhD in economics. That doesn't mean you can't do economics at Google at all: you can do them as a data scientist and collaborate with economists. I know this from my colleagues who work at Google. It won't be so easy, but I think it's possible.
If you have any questions about what I have told, then I will answer the most interesting of them even later. Thanks to everyone who asked interesting questions and who was interested in this lecture.
