Several months ago I tried their hiring process for a position not listed here.
Hopefully this can help guide anyone interested in applying. You can have an idea of what to expect.
Basic timeline:
Day 0 - Saw advertisement on Who is Hiring thread
Day 2 - Applied via an application form that asked
me for some personal information and to complete
two tasks. I had to send a POST request containing
some specified information and write a bash script
to process some data.
Day 12 - Saw HN frontpage ad for same position and decided
to send the same application materials again.
Day 16 - Response from a human who explained they were working
on other roles for the last 15 days and now they
were finally ready to look at applications for the
role I applied to. Asked if I would take a 60 minute
HackerRank assessment. I agreed to try it.
Day 17 - HackerRank assessment available for 2 weeks. Message
explaining a 2-3 week timeline from that point until
possible offer. Steps listed included:
1) HR assessment
2) phone call
3) take home project
4) 2 technical phone calls
5) background & reference check, meet team
6) offer
Day 25 - I completed the HackerRank assessment.
Day 31 - Response that the position had been filled but they
were interested in speaking with me for future openings.
Overall it seems pretty typical. I was a bit surprised by how many screening tasks they had. Two just to submit the application, an online timed test, and a take home project seems like quite a lot. No further contact since then, although I haven't been following closely enough to see if they ever posted the same or similar positions again.
Thanks for commenting. Sounds like you applied for our DevOps role, which we had a flood of really great candidates for and hired 2 people in that timeframe quickly.
We have been refining & trying to streamline our interview process, but our #1 priority is to make sure we find a really strong fit (both technically and culturally) since our team is still small and we're looking for all-stars :)
This one has left me with a particularly bad impression since they specifically say they take referrals from HN very seriously.
I don't recall being asked if I found out about them from Hacker News (I did), but they did ask me to provide links to LinkedIn, Twitter, StackOverflow, and GitLab/GitHub profiles.
Here's the grand sum of response so far, I think I've waited as long as is reasonable to share. It's been well over a month. No other contact.
Thanks for your interest in Kaggle. This is
an automated message from Kaggle's applicant
tracking system to confirm that we have
received your application and will review
it in the next seven days.
Don't promise anything about 'seven days' if you aren't going to at least respond with an automated email saying you threw the application out after a quick glance.
I'm sorry you had a bad experience. Would you mind emailing me at a@kaggle.com? I'd like to look up your profile to see what happened. I'm in charge of recruiting so it's my fault if your application slipped through the net.
I just noticed that the URL was missing from this month's link and last month's link. (This month I just copied last month's post.) Again, Glad that this was pointed out.
I may have used too harsh a tone in my earlier message, so I apologize if you felt attacked.
If you are tracking HN referrals with a special URL, you wouldn't have known I came from HN. I found your site and the Careers link from a Hacker News post that was not a hiring advertisement. Maybe you could allow people to self-identify their affiliation instead.
Unfortunately, I see no benefit I might gain by reaching out to you directly. I have posted a negative review of your company's hiring process in a public forum. If I was interested in applying for your current or future openings, that probably wouldn't help my case.
I'm happy to have helped identify any issues with your process, and again apologize for the tone in my previous post.
You're right: we will miss people with this way of tracking referral source. I'm actually less concerned about this issue than when I first posted on HN. We now only post on HN, Stack Exchange and Github and we've found that the quality of the candidate pool is much higher than when we posted more broadly. I view HN, Stack Exchange and Github as equally credibly sources, so distiguishing HN from other sources is less important for us now than it was a few months ago.
Re harsh feedback, I understand the perspective, but think it's actually not such a big issue for us for two reasons:
1. as a culture, we strive to give each other direct feedback, so your comments are in keeping with a culture that we aspire to
2. on a more light-hearted note, FAR more harsh things have been said to us on the Kaggle forums. Helps us develop a tough skin :).
This interaction seems to have gone well on both sides, leaving me feeling particularly warm. As a member of the peanut gallery with no stake in the outcome of this interaction, I'm nonetheless glad this was sorted out!
A few days after that post, I got an email from a human who had actually read and reviewed my application materials. Very refreshing response and left a great impression. Did not progress further in their process at this time.
I can see how responding individually to a large volume of applications would be very tough to do in a reasonable timeframe.
Total time from my first submission to a human response was 15 days. If that represents a peak period, maybe it's a good upper bound on what to expect. I wonder what the lower bound is.
Hello! We try to respond to candidates as quickly as possible depending on the where the pipeline is at with any give role. If you would like an update on the status of your application please feel free to reply to People Operations, and we will respond to let you know exactly where your application stands.
This one seemed interesting so I gave it a bit of effort. The ad is well written and pulls the right levers. Well done on that front.
I'm a bit disappointed in the challenges page. The copy on the site tries to emphasize how difficult they are and that anyone who passes the challenges would be considered a serious candidate. While they were fun challenges, they're way too easy.
I completed multiple challenges then sent along my information. As soon as they saw a resume that didn't include much experience in the security domain, I was basically ignored.
Recommendations for Praetorian:
If you want the challenges to mean anything, make them more significant so that completion would actually be a strong positive signal for the candidate rather than a BS-detector. If someone passes your challenges, you should be begging them to work for you, not the other way around.
Cut back a bit on the elitist rhetoric, it is a bit of a turn-off. I think it will attract more mediocrity and scare away some great talent.
Final word: the challenges are fun for anyone and if you like that kind of thing, you will enjoy beating them.
I am bummed that you got the impression that you felt ignored for not having security expertise. I am not certain when you applied but I know our focus lately has been on senior engineers so that likely contributed to our response. Regardless, if we didn't do a good job relaying that information to you that is not good and I personally apologize (it was likely my fault as I handle a good chunk of the initial responses to applicants).
Thanks for the feedback though. It is appreciated.
Thank you very much for sending in an application. I do apologise for the delay in responding to you. We are receiving a large volume of applications at the moment.
If you can send an email to jobs@gitlab.com with your name and the position that you have applied for I can take a look and find out what the status is.
Thanks again for taking the time to contact us at GitLab!
For me, the paid work day was 8 hours in a Slack chat asking a lot of questions of 1-2 employees. My task was to get familiar with their existing systems by querying these employees. I was asked to prepare a report about what I found.
The guys seemed busy so for a lot of the day I was on my own, waiting for more information.
It was intense, but by the end of the day I had a pretty good sense of the internals. I think it was a good test overall.
It sounds like it worked like we want, kind of! The work days should be more interactive than what you experienced, I think. The really good ones seem to be super busy for the first 4 hours, with 3-4 existing employees interacting. The latter half is frequently quiet if people are on the right track.
I can guess which position you were going through the process for, and it's something that we've been talking about a lot.
One thing we've noticed is that even when it's not perfect, the process we have helps a lot. You had an imperfect work day, but got a good idea of what the actual work is like. I'd rather you have a perfect workday though.
I tried this process because it sounded pretty fun and unique.
One warning I have for anyone interested is that there are some big waiting times involved. I'll share my timeline. Some of the lags were on my end, but I was assured I had as much time as I needed.
- Day 0-6 - First contact
- Day 7 - Assigned work sample
- Day 21 - Submitted work sample
- Day 62 - Work sample results returned
- Day 72 - Paid work day
- Day 79 - Work day report completed
- Day 85 - Final decision
Overall there were 21 days where they were waiting on me and 47 days where I was waiting on them. This seemed to me like a lot of waiting but maybe it is normal.
I probably worked 15 hours on the work sample, 8 hours on the work day, and 8 more hours putting together the report after the work day.
Oof, bummer. Your experience is slower than average. We ran into a grading crunch when we launched our Enterprise product. When you submit your work to us, your submission is anonymous. Three engineers grade each submission.
Platform Engineers will experience the greatest wait periods. The other roles tend to be quicker.
Expected time-frame:
* Proceed from application to work sample: 24 hours.
* Complete work sample: Up to the candidate. We build them to take 2-6 hours.
* Grade work sample: 1-4 weeks.
* Schedule work day: 1 week.
* Receive decision: 2-5 days.
Thanks for taking our time to try the process out, it is appreciated. The method is constantly under improvement.
It's hard to see how you can have much success with this method. Current recruiting strategies involve getting from recruiter telephone call to final decision within 2 weeks, or 1 if possible. The faster you move on a candidate the more likely they end up working for you. At my current employer initial phone conversation, telephone interview and on site were all within 7 days.
While I'd never argue its a "good" thing to have a slower pipeline, in the past when I had pipelines with timing similar to this companies, it didn't have extreme negative impacts on the success of the pipeline.
I don't know why that was, but my suspicion was that our highest qualified leads were those coming from candidates that were "happy enough" at their current positions to not be frantically searching.
As long as we seemed like a good place to work, and we did a good job of communicating why back logs were happening, we didn't lose many of the candidates we were most excited about.
my experience is that I am mostly happy where I am, and I am not in market when I am Happy, but when I am not happy, I want to get out asap and wait times as high as a month is a real turn off
It seems you truly care about feedback and the quality of your process. FWIW: I'm glad you are making these timelines transparent so people can decide whether the position is a good fit or not for them. As someone who is in the field for over a decade, it strikes me that this timeline is really long and you run the risk of getting people who couldn't get a better offer from another top employer faster. In addition, I personally have stopped engaging any hiring process that has a take home exam. My view is that my degrees and track record are sufficient evidence of my competence. Also, my time is really valuable .. so engaging with companies who are forced to waste their engineer's time as well as mine seems fair. Just one person's opinion :)
Agreed. That's kind of ridiculous. I don't understand this whole "we are not interested in speaking with you as a real human being but 'here', here is some homework for you to do' Seems really silly.
It has its tradeoffs. The point is, if they're going to go that route the assignment should be short and sweet, and the response should be quick. Especially since we can pretty much tell, like, right away whether we like someone else's code or not, now can't we?
Basic, common-sense considerations which a lot of places don't seem to appreciate, unfortunately.
And to not respond at all is just a gratuitous insult.
He's understating what he's done a bit. We're vastly more consistent with candidate feedback than we used to be, largely due to tooling + process changes that grayfox has put in place. It's improving, we'd like to turn evals around in <1 week for everyone.
Apologies,but this process feels really lame. I know that many companies do it, (like atlassian, for example) - but it feels as though one is interviewing for a slavery post. Let me explain:
You hire people. People have lives. But this type of process just illustrates that you're a Corp and not really having the best interest of the people you're hiring. The reason I state this is because corporations don't give a fuck about anyone who under performs and they shall fire anyone at will. You're not a family, you're (the corp) not going to put in nearly as much effort as the candidate did to get the job, in order to keep said candidate.
You make them take many weeks to apply and get the job, but even for trivial reasons (we lost that customer/contract or we don't have the funding) you'll kick said candidate to the curb without a second thought.
It's a one sided position in the favor of the corp, and for that reason I would never choose to work at any company with this method for hiring - and especially a company that has this example of how long that fucking process takes.
The point is, you think you're looking for "the best fit" but you're actually alienating people who would be a good fit.
Of course they're a 'corp', and of course they're looking after their own interests over yours. Looking after your interests is your own job, no one else's.
The employer-employee relationship is partly a fight. The best you can hope is that it be a gentleman's one, with no hits under the belt, rather than an all-out brawl or backstabbing.
grayfox is showing you cards that the average employer would hold back and would only release at gunpoint: salary range, internal details of hiring process, how long the process takes... seems this is gentleman's territory for now.
The long process selects for people who are happy with their jobs, which is the situation of most experienced and skilled people. I have seen worse.
Edit: I'm realising this is more aggresive than I like. I'll leave it as is, with the addition that I once thought like you, and this here is the state of mind I've come to have now that I've been on both sides of the recruiting fence.
I sympathize with you and also dislike long evaluation times. I also have kids and a life outside of work. Personally I think 6 hours is too long to design a work sample around but it's not that bad, I've done work samples (early in my career) that took 3 days and that I would've billed out for $3k when I became a consultant later.
In grayfox's timeline, things really seem to hang up around the time it takes to evaluate the sample, asking for 1-4 weeks. This is another point in favor of minimalist work samples (smaller sample = less time required to review). IMO work samples should be a simple problem where the candidate demonstrates his basic awareness of the core skills needed to perform the daily work and the bulk of the hiring process should occur in a discussion between parties.
Anyway, I'd suggest that they tighten up this process by a) minimizing their work samples and b) making it priority to review work samples as they come in, which means adding more resources for this purpose if necessary. It shouldn't take more than a week to hear back at any step of the process.
Minimalist work sample tests don't work. The work sample test is the test. There is no other criteria involved in whether to hire someone other than their performance on the work sample test. Otherwise it's not a work sample test; it's an intro.
This is a step forward for our industry. For those who don't like it, there are plenty of jobs where you can do the traditional whiteboard hazing.
I've had a lot of success hiring with minimalist work samples.
>Otherwise it's not a work sample test; it's an intro.
Well, like I said, I think the work sample should only be needed to show basic awareness and that the rest of the process should be based upon the course of a discussion around the candidate's relevant experience, the company's needs and intended role for the candidate, etc., so perhaps it's not wrong to call the code sample an "intro". I referred to it as a "litmus test", meaning it's a simple pass/fail; they are either able to throw something together in 1-2 hours showing that they have basic awareness and competence dealing with the problem space, or they're not.
Overall fit is much more important than something like "candidate Y has better indentation habits". Human cycles are thousands of times more valuable than CPU cycles. It's better to choose the good fit candidate whose coding habits can be trained up over the course of his employment than the unstable abrasive candidate whose code ran 1.5x faster than anyone else's.
>whiteboard hazing
Heh, I don't suggest this either.
The frightening reality about hiring is that it can't really be reduced down to a formula. There are subjective judgments that have to be made (in both directions, meaning that you shouldn't be absolutist about things that are correctable in their code sample) if you're going to get good hires and a cohesive team. Questions like "Is this person an active, curious learner?", "Is this person able to field fair critiques of his work product professionally, reasonably, and humbly?", and "Is this person's personality going to mesh OK with the rest of the team within a professional work-day setting?" are all much more important than a raw benchmark of their code sample.
I know that a lot of people don't like that subjectivity, especially when they're on the wrong end of a subjective judgment, but I don't think it's a good idea to discard evaluation on those metrics. We just have to hope hiring managers are using reasonable subjective criteria, and if they're not at the company we want to work for, we gotta move on. Fortunately for us, there are plenty of fish in the sea looking to employ programmers right now.
Our sample projects are meant to tell us that a person is technically capable of doing the individual work for a given job. The work day that comes after answers "can they work well with us?"
The criteria for both project/workday is predefined ahead of time, and the part we spend the most time on. Sample projects have ~40 criteria, nothing quite so minute as "indentation habits" but we do give points for "idiomatic use of golang" on the go work.
What we found when we were putting processes in place is that we just shifted our bias. As an exampe: if a recruiting process judges abrasiveness during a traditional set of interviews, it's probably biased. It's better to put people into a work scenario and judge their actual work and how they interact with other people.
A self-contained project that gives the candidate room to demonstrate a basic grasp on core skills. It should take no more than an hour or 2 hours at most if the candidate chooses to stay within the constraints of the project (in my experience, candidates will often throw in a few extra features since we're not monopolizing their time with a demanding list of requirements, which provides really awesome insight into the candidate). In most cases, if the candidate is capable of a simple project like that, they'd be equally capable of a larger project, and there's no use wasting anyone's time on a larger thing (unless you're focusing on the wrong stuff, like specific knowledge of one particular library).
For my purposes, I like variations on a project that asks them to implement a very basic listing call from a public API. I let them use any language they want. This shows that they can put together a project in some language, look up API and/or library documentation, provision an API key, reference external API docs and code a function that reads against it. I understand that Compose covers a different space and would need to tailor a different minimalist project.
For me, if they're capable of this minimalist code task, it demonstrates basic competency and filters out almost all of the guys that aren't worth wasting time on. It doesn't demand too much of their time and allows them maximum freedom, which allows us to see a lot of information not only about their organizational skills but also about their code ideals. It doesn't penalize a great asset for not having run across a specific language, platform, library, algorithm, or data structure in his/her past; those things can be learned quickly by good candidates. It doesn't depend on knowledge of trivia or number of times they've seen the problem in past interviews/tests. It's not overly academic and doesn't depend on how long it's been since they reviewed their compsci textbook.
The rest of the information needed to make a hiring decision is derived from an extensive discussion on their background in the field, their attitude and goals, and their immediately relevant experience.
Agreed with (b), we've gotten lots more consistent and turnaround on all the evals has improved substantially (largely because of the person you're all responding to).
Hopefully this can help guide anyone interested in applying. You can have an idea of what to expect.
Basic timeline:
Overall it seems pretty typical. I was a bit surprised by how many screening tasks they had. Two just to submit the application, an online timed test, and a take home project seems like quite a lot. No further contact since then, although I haven't been following closely enough to see if they ever posted the same or similar positions again.