From the creator of the Tarsnap crypto service for backup

In a recent discussion on Hacker News, a commenter asked the question:

So what do we think of Tarsnap? The author is clearly a genius who spends time on backups instead of solving millennium problems. I say this with the greatest respect. Is the temptation of entrepreneurship a trap?

At first, I wanted to answer in the thread itself, but thought the topic deserves a deep answer that will be read by more people than in the middle of a HN discussion with more than a hundred comments.



First, let's deal with the philosophical side of the question: yes, this is my life, and yes, I can use it - or spend it - as I like. But there is nothing wrong with asking how best to spend your time. This is doubly true when it comes not only to my personal choice, but also to the broader question: is our society really structured in a way that encourages people to be of less value than they could?



Although I slightly dislike the premise of this question - in particular, the statement that I was "wasting [my] time on backups."



On the one hand, it's true: Tarsnap has been my job since 2006. I do some counseling — not so often lately — but financially, it was Tarsnap who paid all the bills (including the purchase of the house I’m moving into next week). On the other hand, my work on Tarsnap has seriously expanded into related areas.



In 2009, many Tarsnap users asked for a passphrase key protection feature, but I found the completely disastrous state of the art for password-based key generation. Then I came up with the scrypt tool- and in the process opened up a whole new field of cryptography. Of course, I did this to improve the security of Tarsnap; but it's not entirely fair to say that I was "wasting my time working with backups."



In 2011, wanting to securely connect daemons on different hosts and not being satisfied with the existing TLS-based options, I wrote spiped . Although not widely adopted in general, I still consider it a significant contribution to computer security - like scrypt, I created it to meet the needs of Tarsnap, but it would be a stretch to place such a versatile open source tool in the narrow definition of “working with backups ".



Around the same time I started working on kivaloo, a high-performance key-value data store. Perhaps this is the least used of all my programs, I do not know who else is using it at the present time besides me (although this possibility is not excluded for an open source program) - but I think that this is one of the best examples of my code, and in it may find more use in the future than Tarsnap itself.



Since 2006, and especially since Amazon launched the M3-enabled EC2 family of HVM instances in 2012, I have been building and maintaining the FreeBSD / EC2 platform . While I don't have exact statistics on its usage, last year's poll showed that 44% of people running FreeBSD in the cloud use Amazon EC2 ; therefore - despite the fact that there are currently only 22 peoplesponsoring my efforts - clearly my work here has been productive. Again, my primary intention was to get FreeBSD running on EC2 for Tarsnap, but this work is unlikely to be fully categorized as "working with backups".



Of course, the question is not whether I have done anything useful, but whether I have spent those years with the maximum benefit. Based on the reference to the Millennium Challenges , I believe that the person was referring to an alternative in the form of a research career. Indeed, if life had turned out differently, then between my student studies in number theory under the direction of the late Peter Borwijn and my doctoral studies at Oxford, I could seriously think about the Birch-Swinnerton-Dyer hypothesis.(BSD, one of the Millennium Challenges - approx. Lane), and this BSD is very different from the one I'm currently involved with!



So why didn't I choose an academic career? There are many reasons for this, and the launch of Tarsnap is certainly one of them, but most of the reasons boil down to this: "University science is a lousy place to do innovative research." In 2005, I prepared the first article on using shared caches in multithreaded processors as a side channel for crypto attacks, and in 2006 I hoped to continue this work. After earning my PhD from Oxford University and returning home to Canada, I became eligible for a Postdoctoral Fellowship from the National Council of Science and Engineering Research Canada, so I applied and ... was not approved. My supervisor warned of the risk of research that is “too innovative” for a young scientist: the committees don't know what to do with you, they don't see you have any reputation to rely on. Indeed, I ran into this problem: the reviewers in the journal of cryptology did not understand why they were sent an article on processor design, while the reviewers in the journal of computer hardware did not understand why they were sent an article on cryptography. From both my own experience and the advice I received, it became clear to me that if I want to succeed in academia, I need to publish additional articles every year - at least until I get a position at the university.



In many ways, starting my own company gave me the freedom that scientists strive for. Of course I have clients to help, servers to manage (not that they need special management) and business accounting, but professors also have classes to teach, students to supervise, and committees to attend. When it comes to research, I can follow my interests, ignore the whims of grant agencies, recruitment and promotion committees: I can do projects like scrypt, which is now widely known but languished in obscurity for several years after. as I posted it. And in the same way, I can do a job like kivaloo that has been essentially ignored for almost ten years,without any signs of a change in the future.



Is it possible a world in which I would now be a scientist and work on the solution of the Birch-Swinnerton-Dyer hypothesis? Sure. Perhaps the world's most talented students receive a kind of "mini-genius grants" upon graduation. If I had received a five-year grant of $ 62,500 a year with the sole condition of “doing research,” I would almost certainly continue to work in academia and - despite more interesting but longer-term questions - would publish enough publications to secure a permanent research position. ... But this is not how grant agencies work; they provide grants for one to two years, with the expectation that successful studies will later apply for additional funding.



In short, academic institutions are systematically promoting precisely the kind of short-term optimization that, oddly enough, is often blamed on the private sector. So no, entrepreneurship is not a trap. This is the only way to avoid the trap right now.