Toptal.com Application Process

A couple months back a friend pointed me at Toptal.com. I believe it was a slight since I had, rather arrogantly, said to him once I considered myself one of the top developers in the Denver area. It looked like a good way to challenge myself as a developer, so I applied. I wasn’t expecting to get any work from it, it was really just another interview.

Right away I had a good impression of the process. After applying I was notified that I would go through multiple steps of the screening process. Just the fact that it was going to take many weeks (and months) to go through made me think this was a solid process. At least it was better than a “one and done” interview.

After the initial discussion I was told I had to take a technical test. Right away I knew I wouldn’t do well. These are always algorithm tests (which I’ve spoken out against before as a sole way to judge the ability of a developer). Yup, three algorithms I had to solve. Complexity in both time and memory were concerns as well as extreme boundary cases.

But, I was ok knowing I wouldn’t perform well on these because, surely, it wouldn’t be the only way to judge my abilities. I wrote code and got the basic test cases to pass. While learning more about negabinary (involved in the second test) I stumbled across a solution (in Python) to the problem.

Before I explain any more… I was told that I should approach this process and answer all tests and questions as if I was asked to do so by a paying client. A completely reasonable and understandable request. So, I did exactly what I would have done if I was being paid to solve this problem: I took the solution I found, copied and pasted it into the answer box, ran tests (which passed) and moved on. I had hoped to come back to the problem but time was running out and a working solution seemed better than none. I had even tried to attribute it back to the original source but apparently I copied/pasted the wrong link as my next-stage technical interviewer specifically asked for the source.

I finished the third test and submitted my response. Now I was to wait 2 weeks until I could discuss my answers with a technical screener, we’ll call him “Steve”. I don’t like waiting, especially when there is some explaining to do, but so be it.

The first question “Steve” asked was why I used Python for my second answer. I explained that I have written some Python in the past, was familiar with the language, but then explained exactly how I got to the answer I did and explained everything. I was honest and forthcoming. I wasn’t getting any sort of positive response from “Steve”.

He then told me the next step would be a live-coding test. Ok, good, here we go, I get to work a real-world code, not just some algorithm test.

Nope, another algorithm test! I was asked to write unit tests to verify about 10 test cases in finding Circular Primes. I was told I’d have 20 minutes to complete 2 tests. This being the first. So, I set off to work as “Steve” watched me go via Skype screensharing. I had gotten the tests written and some initial code written in just a couple minutes and was feeling good. I suddenly realized I didn’t know how to check if a number was prime, so I asked and got permission to go out on the internet and find a solution… 45 seconds later I had a method for checking if a number was prime and continued.

Tests weren’t passing so I started stepping through code to see what part of the process I had overlooked when suddenly I hear his voice come over my headset… “you’re out of time.”. “OK”, I thought. I got pretty far, maybe I’ll do better on the second test.

“Since you didn’t pass this first test I can’t give you the second one and I’m failing your application. You can reapply in 2 months”.

I was shocked. I sat there staring at my screen not completely sure I understood. But, I stayed professional. I asked a few questions, thanked him for his time, and wished him a good day.

Now, I can’t help but wonder if my fate was sealed before I started. I’m sure they had reviewed my answers and realized I had copied/pasted a solution. My explanation didn’t elicit any response, positive or negative, from “Steve”. It was if he was just listening and dictating my response. In fact, I’ve spoken to quite a few technical screeners over the last few years and rarely have I had as shallow of a conversation about my skillset as I did with this gentleman. The questions always start the same… “what language do you use?”, “What JS frameworks do you know?” … “Do you TDD?”. Each time I’ve been asked these questions I give some pretty thorough and long-winded answers. Usually that involves more questions, specific technical questions regarding solutions I’ve used, and more conversations. If you know me, you know that I won’t stop talking about things I’m interested in, even to a fault. I’m sure I’ve bored the hell out of people explaining the significance of Extremis to the Iron Man storylines for the last 10 years.

I recall a specific question I was asked: “what libraries do you use?”. I started mentioning a few of the big ones… Durandal, KO, Angular, ASP.NET MVC Web Api. I wasn’t going to list every single library I use, that would take 20 minutes and a review of my nuget packages. There was no digging into specifics, just very boilerplate questions from someone not familiar with what I do. I’ve had this type of interview before from 1st stage HR screeners and this wasn’t impressing me. I got no impression that he knew or cared about my skillset and my abilities.

So, I have to imagine that if I hadn’t submitted any answer instead of copying/pasting that Python code, I probably would have made it further. More so, though, if they’re willing to dismiss a developer based on his/her ability to answer algorithm questions then this isn’t an organization I want to be a part of.

Simply put: over the last 8 years of being a full-time developer, and 12 years in this industry, I have never once had a business come to me and ask me to find all the circular primes between 1 and N. Not once. I understand the value in seeing if a person is able to work through a complex problem, but that’s not what “Steve” or these scores reflect. They only show the solutions ability to catch edge cases and stay efficient. In fact, highly optimized code is often very detrimental to the types of applications I’m writing. Code readability is directly and negatively affected (in most cases) by highly optimized code. And code readability has always been, in practice, more important than highly efficiency in the work I do.

When I wrote the original Stackoverflow question years ago it was out of frustration. I had gotten out of an interview after being asked an algorithm question, one I felt I didn’t do well. Ironically enough, they came back the next day and offered me the job. However, years later I do understand the importance of this type of question. But, simply having a highly efficient answer is only one aspect of how good a developer is. I know a few developers who would probably do really well on this type of question but I wouldn’t hire to help me get a customer’s application developed.

In the end I’ve accomplished what I wanted to with this process… I found out where I stand. I’m highly dubious of their “top 3%” claim, both before and after having gone through the process. I’m disappointed that I didn’t perform better on the tests, but it’s not a skillset that I see as being highly valuable in my career so I’m not going to worry about it.

This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to Toptal.com Application Process

  1. Farrukh Qazi says:

    LoL !!! I must say I had fun reading your story 🙂 and I totally agree with you. It’s been what 10 years in programming for me professionally but never once I had to go back use these algorithms in any application. Yes, never used them doesn’t mean I never cared about performance or memory restrictions but there are tools for that. You build and optimize, I believe its impractical to build something that is highly efficient else we wouldn’t have all those processes for refactoring.

    Anyhow, I feel good having someone resonating my thoughts over these test type screening. I think if one day I will get into a position to start my own company, I would definitely hire people on temporary basis and ask them to work for a day or two. I do care if they copy code or build it themselves or ask their friend to write it for me, all I would care is their approach to go about the solution that I asked for. That’s what I think is needed in the programming jobs as well, how good of a solution you can come up with for a given problem.

    Good read again !!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s