From the translator: On July 22, Slate Star Codex author and renowned rationalist Scott Alexander deleted his blog in an attempt to avoid de-anonymization by an NYT journalist (the blog was restored on September 11). This was the reason for me to read his texts, one of which I liked so much that I decided to translate it. The original text was published on Halloween 2018. In square brackets are links that add context to the non-US reader.
Scott Alexander
Sort by Inconsistency
Thank you for allowing me to post my story on your blog. Mainstream media sucks, nobody would believe me there.
It all started in September 2017. At that time I was working in a small start-up on internet advertising. You know, like on Facebook or Twitter. We teach advertisers how to get more clicks. Our startup, whose name I won't tell you, tried to add deep learning there. As soon as investors see "deep learning" - they immediately start to waste money. We teach the neural network to predict how many likes a topic will receive on Reddit. Then we ask her how many likes some of our advertising posts will get, and post one of the most requested ones. Here some people (not me) explains it better. Why Reddit? Likes and dislikes are easier to process than a bunch of Facebook reactions, plus demographic targeting by subreddits, plus there is an archivenearly two billion messages that can be downloaded for the training set. We trained the network to predict likes by post title.
Any predictive neural network can also work as a generative one. I taught the network to recognize dogs - run it the other way around to get a picture of the dog. Taught the network to predict likes by text - run it the other way around to get the text that will receive likes. We tried it, it was very funny. I don't remember the exact text, but for / r / politics the generated topic was “Trump is no longer our president. Transgender people are our new president. " For / r / technology, it was something about Elon Musk saving net neutrality. You could also generate headers for maximum dislikes, but that would be boring and like penis enlargement spam.
On reddit, you can sort posts by the degree of controversy. The exact algorithm is here , but in short, the algorithm multiplies the number of votes (likes + dislikes) by their balance (the ratio of likes to dislikes) to pop up posts on which people disagree with each other as much as possible. Controversial topics attract people, so we taught our grid to predict them too. The project was run by a brand new Indian woman with a long name who everyone called Shiri, and she was not doing very well, so Brad, our boss, asked me to help her. Shiri ran the grid on a two-billion archive of comments, and the result was highly controversial hypothetical scenarios around US policy. Everything went according to plan.
The Japanese, when developing biological weapons, tested it on Chinese prisoners. The Tuskegee Institute tested isolated syphilis strains in African Americans. We were either more ethical, or just dumber, and tested the Shiri Shears right on ourselves. We had a closed internal subreddit to discuss our affairs: Brad wanted us to get better acquainted with the platform itself. The problem was that when Shiri tested the neural network for controversy in our subreddit, the output was either a set of platitudes or obvious nonsense. Not even a hint of a controversial topic or room for disagreement. The topic we looked at then was about the architecture of our code. I will not go into detail, just imagine that you have taken all the worst architectural solutions in the world,hardcoded them in the most miserable way and rolled them out to users with a fat fig in their pockets. Shiri's scissors put forward as the most controversial topic a proposal to organize our code with this architecture. And so, when we had been discussing for about ten minutes where the bug could be, Shiri said that she did not understand how the program generates such obvious things.
Shiri did not speak English very well, and at first I thought that we did not understand each other. I corrected it. The program was generating obvious nonsense. She stood her ground. I still thought she had mixed something up and explained to her the difference between the words "true" and "false." It looks like she was offended. I tried to clarify if she really thinks that this is the worst architectural decision, this plan to use all the poorest patterns together so that they can never be fixed, will be correct for our product? She was sure of it. Moreover, she was surprised that I thought differently. She thought that we were already working in this direction (of course, we were not), and abandoning this architecture would lead to a complete rewrite of everything and make our code much worse.
I already began to doubt my own adequacy, and we went to Blake and David, our senior developers and "voices of reason". They were discussing some of their affairs, but I interrupted them and gave them the Scissors thesis. Blake naturally asked why we distract them with this blunt slag. However, David got confused after Shiri and began to say that this was the right decision. All four started arguing. I was still convinced that both Shiri and David misunderstood the question (although David's first language was English and the topic was crystal clear). David, meanwhile, felt that he was being discredited more and more, said that he didn’t understand anything, and Blake and I were just two shitty programmers who didn’t understand a damn thing about the simplest architecture. He continued to insist on the same as Shiri: they say,Scissors' proposal is part of our architecture and any attempts to go in a different direction in development will screw everything up. We got into an argument that we decided to go to Brad.
Brad was the founder of our firm. Don't believe the newspapers: Not all tech startup founders are greedy people and sociopaths. However, in the advertising business everyone is just like that, and Brad was no exception. He was an unpleasant and unprincipled son of a bitch. But he made a good impression on investors and knew how to program, which set him apart from many bosses. He got a little screwed up when the entire development team rushed into his office, but he listened to us.
David began to explain what was the matter, but literally messed everything up. I couldn't believe he was lying to keep a good face in front of Brad, and intervened. He asked not to interrupt, but Blake said that if he had not misinterpreted the situation, no one would have shut him up. Then everything went downhill. Somewhere in the middle of it all, Brad finally figured out what we were arguing about, and interrupted us: "This is the stupidest decision I've ever heard of." He added that the decision is contrary to plans, programming and business practices. Miserable losers David and Shiri began to accuse Blake and me of badly influencing Brad because we were his favorites from the beginning. According to David, we got good projects one after the other, while he and Shiri worked like slaves in galleys. Brad called him a jerk and urged him to go to work. But he didn't go.
That part of the story ended at 8 p.m. Brad fired David and Shiri for a combination of overwhelming incompetence, equally overwhelming lack of command, and general inadequacy. He said that in vain he gave a chance to David and Shiri: they say, from the very beginning it was clear that they lacked qualifications, and at every opportunity they repaid him for kindness with laziness and sabotage. In addition, in his opinion, they hurt the company and, possibly, worked for competitors. David and Shiri also called him a tyrant, and the whole company was called a murky office that uses vulnerable workers to personal gain Brad. They also accused Blake and me both of being with Brad.
It was already eight in the evening. For five hours we had a fight with each other in Brad's office. At 8:01 am, after David and Shiri flew out of the office, the three of us looked at each other and thought: Fuck, the generator of controversial topics really works.
I would like to repeat it once more: during these five hours, none of us even thought of it. We've been too busy arguing over the Scissors approval. We did not have the opportunity to look from the outside and realize that the whole dispute was due to a statement that was generated as the most controversial one. But at 8:01 am, when we won the argument, we stopped and thought: well, your mother.
That evening we were already too tired and went home, but the next day all three (Brad and the remaining two programmers) gathered to talk. We discussed what we got. Blake gave it a name: Shiri Scissors. In one of the ancient languages, the word "scissors" is cognate with the word "schism" (schism). Scissors create schism, make a split in the fabric. And that's exactly what we did. It's time to change your business: to move away from advertising and start producing superweapons. At least call the Pentagon and tell them that we have a program that makes people hate each other. What other ethics? We worked in online advertising; we would sell our grandmothers to Somali pirates if it would add clicks to us. The train left long ago.
It's not easy to call the Pentagon and tell them about your superweapon. Even in Silicon Valley, they just won't take your word for it. But Brad asked his friends for a favor, and about a week after David and Shiri were fired, Colonel DARPA was in our meeting room, who was interested in what kind of kipish we had here.
Suddenly we have a problem! We couldn't show Colonel the Scissors that led to David and Shiri being fired. He is not from our company, not from the advertising business, he will be bored. We didn't want to generate a new controversial thesis for the Pentagon. Even Brad might have guessed that getting the US Army involved in a civil war is not the best solution for clicks. In the end, we came up with the following. We decided to explain to the Colonel what Reddit is and ask him which community we should destroy for example.
He thought for a moment and replied, "Mozambique."
We clearly underestimated the cultural divide. When we asked the Colonel about the target for Scissors, we expected to hear something like "board lovers" or "My Little Pony fans". But DARPA colonels think differently about the world, so he chose Mozambique. I started explaining that Reddit doesn't work that way, that I need a subreddit with this topic, but Brad interrupted me. “Mozambique has a subreddit,” he said.
It seemed that gears were turning in Brad's eyes. On one of them it was written "this dude is already skeptical, if we give up the slack, he will just leave." Another gear was calculating how many hits Mozambique was bringing. As in the Bible: mene, tekel, uparsin. “Perhaps,” he said, “this subreddit fits. We take Mozambique. "
The colonel left us a business card and left. Blake and I launched Shiri Scissors on the Mozambique subreddit. Yes, ethics, all the cases, but as I said: business in online advertising, let's go. The only condescension that we allowed ourselves was that we took the tenth most powerful topic - we didn't want to destroy everything in the bud, just demonstrate. At the heart of our topic was the accusation of the prime minister of disrespect for Islam, expressed in a special way - I will not, again, give details. In the absence of anything better, we PM this to the admins of the Mozambique subreddit asking what they think about it. I don't remember what story we came up with, something about a political science student from the United States who studies the culture of Mozambique. They say, we will be grateful if you ask your friends what would happen if the prime minister did this, thank you, we are waiting for an answer.
We spent most of the week on the Mozambique project. And then again the news. David and Shiri sued us for illegal dismissal and racial discrimination. Brad, Blake and I were white. Shiri was Indian and David was Jewish. In court, of course, they would laugh at this - where are anti-Semites in Silicon Valley startups - but on paper there was no reason to fire either David or Shiri. They worked well, received positive assessments on attestations, and the company had no financial problems - a couple of weeks before the incident, we even posted an advertisement about the search for new developers.
David and Shiri knew why they were fired, but it didn't matter to them. The hatred of the company, caused by the conflict over the Scissors claim, was so blindsided that they would tell any lie to destroy the entire company. We are trapped. It was impossible to tell the court about the Shiri Scissors, because we tried to sell them as a secret weapon to the Pentagon, besides, the recognition of sabotage against Mozambique would be black PR for us. The court demanded documents about what happened in our company before and after the dismissal. Those who worked directly for the Pentagon, perhaps, could ask them for a letter that our work is classified, but the Pentagon did not believe us yet. The colonel made fun of us, no more. There was nothing we could do.
I still don’t know how we would have dealt with this because Brad went to David’s house to punch his face in. Maybe this will seem wild to someone, but you need to understand that David always annoyed his colleagues, and during a dispute in Brad's office he said that if anyone deserved physical violence, then he. The lawsuit was the final straw. I will not judge Brad, who cleared the Augean stables after David, while paying him decent money, especially when it ended with David's betrayal. However, it was over for our company. Brad was arrested, there was no one else to pay for the office. Neither Blake nor I knew how to work with a business, we were simple developers. We quit, Brad went to jail, and so Name Hidden Online Ad Company, Inc.
I would like to point out that we got off lightly yet. We were just incredibly lucky for no particular reason. If Shiri and I began to argue about any statement around US policy, we would destroy the entire country. If Shiri Scissors had been developed by Google, it would have ruined Google. If the Scissors thesis concerned not a piece of our software, but the IT industry or business as a whole, then the entire economy could be destroyed.
And so, we destroyed only our company and, possibly, a couple of closest competitors. If you look at online advertising posts around Fall 2017, you'll find a lot of weird stuff. The story about the CEO [ www.latimes.com/local/lanow/la-jaffer-arrest-20171022-story.html] an advertising company arrested for murder, seduction of minors, a fight with a police officer, plus 3-4 more points, which then turn out to be false charges related to mental disorder [ www.thewrap.com/tech-child-rape-vungle-zain-jaffer ] Is just the tip of the iceberg. I cannot clearly explain how exactly the Scissors thesis got out of our company and why it did not spread further, but I am sure that if I dig too deep, black helicopters will circle over my house. And that's all I want to say about this.
Personally, I left the advertising industry. I started working for a bigger company that used machine learning for voice recognition and tried to forget about the story. Although I still get angry when I think about the architecture proposed by the Scissors thesis. Once I saw a woman who looked like Shiri in a cafe and went up to her to express what I think. But it wasn't her, so I didn't end up in jail with Brad. I checked the news from Mozambique every day, and it was quiet for several months, and then it stopped.... I really don't know if it's our fault. There are many conflicts going on in Africa, and if you watch something, something will definitely happen. The Colonel never tried to contact me, and I don't think he took us seriously. Maybe he didn't even check the news from Mozambique, or maybe he did, thinking it was a coincidence. Maybe he even called us at work, but after he heard that the phone was no longer being serviced, he decided that the game was not worth the candle. Time passed, the conflict did not flare up, and I hoped that the topic with Shiri Scissors in my life was closed.
Brett Kavanaugh then tried to go to the Supreme Court [ www.axios.com/brett-kavanaugh-timeline-allegations-vote-412d33d6-e5dd-43eb-9322-fd2a3867be9b.html]. The situation gave me an attack of deja vu. In the week that he appeared before the court, I understood why.
Shiri told me that when she ran the Scissors throughout the archive, she received several conflicting scenarios of US policy. She showed me two or three of them as an example. One of them described the situation extremely accurately: "A Republican trying to go to the Supreme Court is charged with a sex crime that he committed as a teenager."
It scared me into a cold sweat. Did someone get hold of the Scissors and start using them against the US? Perhaps the Pentagon colonel paid more attention to the project than he intended? But why does the Pentagon need strife in America? Maybe some enemies got hold of the technology? I subscribe to the New York Times: Putin was apparently the first to come to mind. But where does Putin get the Shiri Scissors? Could I confuse something? I couldn't get rid of these thoughts in any way. I didn't have a list of theses from Shiri, but I had most of the Scissors source code to make the build in a few sleepless nights. Then I bought some of the algorithms from AWS and set them on the reddit comment archive. It took three days and a five-figure sum of money, but I rebuilt the list that Shiri should have had. I remembered correctly - it was Kavanaugh.
And it was also Colin Kapernik.
You've probably heard of him. American football player who refused to stand up during the anthem [ https://en.wikipedia.org/wiki/US_national_anthem_protests_(2016%E2%80%93present) ]. Why was I surprised, because I already knew that the Scissors predicted another split? Because Kapernik started protesting in 2016. And we did the first build of the Scissors only in 2017. Putin couldn't get them from us. Someone got ahead of us.
Of the scissors' hundreds of most controversial theses, Cavanaugh was in 58th place, and Kaepernik was in 42nd. 86th place was occupied by the mosque at Ground Zero [ www.theawl.com/2015/10/the-sad-true-story-of-the-ground-zero-mosque ]. In 89, there was a pastry chef who refused to bake a cake for a same-sex couple's wedding [en.wikipedia.org/wiki/Masterpiece_Cakeshop_v._Colorado_Civil_Rights_Commission ]. It's not perfect, but number 99 is very similar to the 2000 case of Elian Gonzalez [ adst.org/2015/09/little-boy-lost-the-case-of-elian-gonzalez ]. Five cases out of a hundred. Could this be an accident? After all, the country is big, a lot is happening, and if something similar to the Scissors thesis appears, then, of course, it is blown up to the national level. But some of the events were too accurate. If this were a coincidence, then I would expect that there will be many more approximately suitable topics than exact ones. But there were only two examples of topics. It looked more like Scissors' theses were tailored to the perfect result.
The first perfect match was a mosque at Ground Zero in 2009. Could Putin have an analogue of Scissors in 2009? I’m sure not. This may sound strange to an outsider: why shouldn't some government be eight years ahead of a small advertising company? The answer is simple: machine learning is developing much faster. Russia could never have kept its eight-year advantage in machine learning a secret. Even the Pentagon could not hide a program that "ran away" eight years ahead. The NSA is thirty years ahead of the crypto industry, and everyone knows it.
But who then generated Scissors' abstract in 2009? I have no idea. And you know what? I do not care.
If you just read any of the Scissor's theses out of context, it will be completely harmless. It just looks like banal truth or obvious nonsense. It will not activate until you try to discuss it with someone. At first you will think that your interlocutors are idiots. Then they will call you an idiot and you start to defend yourself. Crescit eundo [lat. "Growing on the go"]. You will start to notice all the little inconsistencies when people lie to you, to themselves, to their audience every time they open their mouth to defend their idiotic point of view. Then you will notice that all their lies are clinging to one another, that they are pulling up all of their beliefs to defend the Scissors thesis. Over time, when even this does not work, they will begin to incite others to hate you so that no one will ever hear your arguments, no matter how true they are. Finally,the Scissors' thesis is no longer important to them. They go too deep in their hatred for you, building their very existence on hatred and the desire to prevail over you. You have to prove that they are wrong: not because you care about the Scissors thesis, but because otherwise they will do everything to poison the people around you, and make it impossible to talk about your right and your point of view on existence. You know that it is. Your brain is constantly picking up arguments to defend you, repeating arguments. why their attacks are cruel and unjust, and one question revolves in my head: how to crush them? How can you convince those around you not to listen to them, so that they do not attract these people and do not use their weaknesses against you? How can you resist their contrived arguments,before they mess up the brains of good people so much that they will never be dissuaded? How can you ensure your safety?
Shiri once read me two or three theses of the Scissors. She did not say whether she agreed with them or not, and I did not say whether I agree with them or not. And in this case they did no harm.
I never hear voices in my head like crazy people do. But sometimes I talk to myself. Sometimes I use both roles in a conversation. Sometimes I imagine that one of them is another person. A few years ago, I was at odds with a girl, and sometimes the voice in my head is the voice of my ex. I know what is going on in her head, and I always guess what she would say about this or that occasion.Therefore, sometimes I have a conversation with her, despite the fact that she is not there and we have hardly communicated since the moment of separation. It may be strange, but if so, then well, consider me strange.
That was enough. For some reason, Scissor's thesis at number three worked. No other, just this one. A purely hypothetical conversation with an imaginary version of my ex in my head about Scissors' third thesis took me over. Shiri's scissors only work with other people. Other people work as a trigger (and I use this word intentionally as in trigger warning). After the trigger has fired, you no longer need other people. You just need to know that they exist.
I was confident that I would write this story overnight. But it took me two weeks before Halloween - a lovely evening for a ghost story, right? I either booted or smoked pot in order to somehow calm down and think about anything except the third thesis of the Scissors. Although no, not really, I definitely tried not to think about the first two theses, because if I think about it, then I can think about other people's objections, and then I'm finished. I was going to call my ex three times to find out where she is, and if I got through and she answered me, then I don't know what I would do to her. But it's not just her. Half of the people disagree with me about thesis number 3. I don't know who they are. And I never thought about it. I'm pretty sure about that. I can't imagine that I know these people. My friends are too decent people. But I can't be sureit is not so. That's why I drink.
Perhaps I should talk about the alliance against the mysterious manipulators who launch the Scissor theses at us. I want to talk about nurturing radical compassion and kindness as the only defense against such attacks. I would like to give you an inspiring Obama-style speech about the bonds that bind us, stronger than the forces that sow discord. But I can not.
Remember what we did with Mozambique? How, because of some remnants of ethics, did we release a very weak Scissors thesis on them? So, just to make them feel bad, but not destroy their entire country with one click? This is exactly what those who release them on us do. Weak theses. Enough for the rage. Not enough for the end of the world.
But I read the entire list. And then, like an idiot, everyone thought about it. I thought enough about thesis # 3 for the trigger to work. Even starting to discuss it is in itself so disgusting, so hateful and disgusting that Idi Amin [ en.wikipedia.org/wiki/Idi_Amin ] would have blushed with shame at the very thought. And if the Scissors is right, then half of you will happily support this thesis.
For those who have not heard a single strong Scissors thesis and did not fall under it, it is easy to say "do not give in to manipulation" or even "we need to work out a plan to counter and maintain peace, despite the Scissors theses." But how do you know that you are right? How do you know that there is no fact unknown to you, knowing about which, you would agree that it is easier to destroy the whole world and start everything from scratch, like sewer mutants, than to allow people who support such horror to continue to smoke the sky with their presence ? How do you know that you do not behave like a schoolgirl who claims that it is not good to use foul language, although in her life there was nothing worse than a candy that fell into the mud? If she was kidnapped and tortured, would she change her mind? And if she cannot describe the torture to her friends, but can only say “something terrible happened to me”,and they will still talk about the inadmissibility of bad words, on whose side will you find yourself? And why do you think I'm “spoiled” when I tell you that after the Scissors thesis, compassion, kindness and unity can go to hell with leaps and bounds? The pitiful remnants of old morality prevent me from announcing the entire list and letting you kill each other. I am held back by the traces of my past attitude to life and my thoughts from months ago. So hear:I am held back by the traces of my past attitude to life and my thoughts from months ago. So hear:I am held back by the traces of my past attitude to life and my thoughts from months ago. So hear:
Delete Facebook. Delete Twitter. Throw out the phone, unsubscribe from the newspapers. Tell your family and friends not to discuss politics and social issues with you. And if they give slack - stop communicating.
Then stock up on canned food. Stock up on water. Learn to shoot accurately. If you can build a bunker, do it.
Because one day those who give us the Scissors theses will feed us one of those worse.