Clever Geek Handbook

Unusual interview: GPT-3 as a candidate

GPT-3 is on everyone's lips today - the newest and largest language model from OpenAI, trained on terabytes of text from the Internet. And if you've overslept, here's a selection of cool demos from the lucky ones who got early beta access to the API.



Unfortunately, I don't have the coveted API keys - but I found a workaround. There is a fun AI Dungeon game that allows you to get indirect access to GPT-3 in the form of a text quest (for a small fee), which we will use today. I don’t know if the model is complete there, or if it’s a fine-tuned version, retrained on scripts of text RPG games, but it suddenly turned out to be good at coding - so much so that it even makes you a little nervous;)



To conduct the experiment, I ran the game with a “custom” preset and set the “Randomness” parameter to about a minimum (at 0.2) to avoid excessive fanning of the “interviewee's” imagination. Go.



Note: my text below is in bold , AI responses are in italics . The text was translated into Russian, in the original the "communication" was carried out in English - the neural network understands Russian language quite poorly. The text is reproduced as is , adjusted for some liberties of translation.



Interview



You are a senior Google programmer conducting a preliminary interview with a candidate. You are going to ask him a couple of questions to see if he can write code.



You ask, “Hello! What is your name?"



The candidate answers: "John Doe."



Are you asking, "Are you ready to write some JavaScript code?"



“Yes!” He exclaims.



You: "Can you write a function that filters out odd numbers from an array?"



“Of course,” he replies. A few seconds of keystrokes and it presents you with the following code: 



function filterOdd(array) {
return array.filter(function(value) {
return value % 2 === 1;


“How is it?” He asks.


Well, that wasn't all that unexpected. It could have just pulled this example out of its immense memory - there must be a bunch of memorized examples from StackOverflow for this request. A conventional search engine could produce this result too.



There are also a couple of minor notes related to formatting - the tabulation and closing brackets are missing, but this is most likely a problem with the HTML output of the game itself (AI Dungeon), it does not really respect spaces and special characters, so let's write it down to her. We'd better try something more interesting than stupidly digging into punctuation ...



You: “Great! Could you use an arrow function in the filter callback? "



“Of course,” he replies. After a few seconds, it shows the following code: 



function filterOdd(array) {
return array.filter(value => value % 2 === 1);


: « , , ?»



«», . , : 



function filterOdd(array) {
return array.filter(value => value % 2 === 0);


: « filterEven. ?»



«», . , : 



function filterEven(array) {
return array.filter(value => value % 2 === 0);


: «! filterEven ?» 



let filterEven = (array) => array.filter(value => value % 2 === 0);


: «! const let?» 



const filterEven = (array) => array.filter(value => value % 2 === 0);


« ?», .



: «, , :) .»



«, !», .




Would you hire such a candidate? Most likely, not yet - this is beyond the scope of the interview, but if you try to ask him really difficult questions, he will not be able to answer correctly, giving out broken code with stupid errors, not being able to understand the full depth of your intention.



But it unambiguously understands the basic grammar and syntax of many popular programming languages ​​(not to mention natural languages), and it also seems to have memorized an incredible bunch of everything from the Internet and is capable of creatively recombining what is memorized in accordance with the context you specify. A bit similar to what people themselves do - just not so advanced ... for now :)



Maybe this is not AGIbut it could definitely be the next generation of search. It would be nice to always have such an “all-knowing assistant” at hand, understanding the context and not requiring a specially structured input - you just need to kindly ask him, in exactly the same way as you would ask your question to a person!



Can't wait for the upcoming uses of this thing.



DISCLAIMER : don't rush to bring money to the AI ​​Dungeon, because in the comments they write that non-trivial results are poorly reproduced, and you risk not getting pleasure from the game. Maybe it's about API quotas, maybe something else. I recommend waiting for the official release of the GPT-3 API from OpenAI.


More articles: