> I'm unclear what employers are deriving from this exercise at this point
It filters out the complete frauds.
Yes, there are people who know all the right things to say in a job interview, but cannot code at all. If you hire one of them, it takes a bit to find out they cannot code, and a bit longer to fire them, so you're out $$$$ paying their salaries for nothing.
For example, a recruiter I know will ask a tech candidate "what is 20% of 20,000?" A significant percentage cannot answer the question. Some even cry. It's shocking.
A friend of mine was looking at getting a FAANG job. He was worried about the leetcode tests. I suggested he spend a month going through the leetcode books studying them - that the return on his time investment doing that will be one of the best ROIs he's ever done. He did, and got the job. (Although the LC was just a first gate one had to go through to get to the real job interview.)
Personally, I have no idea how I'd do on an LC test without prep. But I don't have a problem with studying it to get a top job.
> it takes a bit to find out they cannot code, and a bit longer to fire them, so you're out $$$$ paying their salaries for nothing
Not just that. They block the hiring process. Maybe there was just that one open position on your team, and that bad 3 month hire postponed a good hire by 6 months. It's also very incomfortable to fire and start again.
My interviews contain questions that are basic, and any new team mate should know. I keep asking them because 90% of the candidates actually can't answer them well. You should know what a "dot product" is if you want to work as a ML engineer, that kind of stuff. Or be able to open a text file and count word frequencies.
True words. It's not 'imposter syndrome' if one comes across as an imposter (though it might have been just be a bad day).
I can appreciate the gating process because it's a real drag to get hired and then spend more time than necessary not doing my work but trying to help coworkers catch up on very basic and fundamental skills in order to be able to collaborate with them.
> For example, a recruiter I know will ask a tech candidate "what is 20% of 20,000?" A significant percentage cannot answer the question. Some even cry. It's shocking.
During our interview, we ask candidates to design a history system. The key is to realize that our database is only 8Gb, and storing a year of updates is only 160Gb, so $2pm in AWS. Once there, a simple DB table suits, no need to set up Amazon S3.
So we ask them for the multiplication. First we give them the data, and if they don’t do the calculation, we nudge them, then we ask them, then we write the multipliers for them, then we take the calculator out and write the result for them.
Those are Masters degrees. They can’t even do calculations, let alone getting it right, because 500MB x 200 days a year = a few petabytes apparently. And after that, they’re exhausted, it’s impossible to ask them the rest of the questions like “So is it worth worrying about a Rube Goldberg machine when you’re in for $2 of AWS costs?”.
Honestly, after ~13y in this, I only know a handful of items from the famous latency table, notably those I've used a lot (mutexes cost 25ns, is only 1/4th of main memory reference tho, then disk seek is x100K of that). I'm "DevOps" (and titular variants thereof) in most of my resume.
That said, I'm not above your bar as each cloud provider have their own pricing model. I'm not even above your bar for AWS---which I've used the past six years---just for the sheer diversity of their offerings, not to mention regional variations. I know how EC2 servers are priced relative to each other but when we include ECS, DynamoDB, Lambda, etc., I'm gonna need a cheat sheet.
Out of curiosity what do you find wrong in the s3 solution? This is a data volume that rdbms can manage, but are you looking specifically for the engineer to say something like “I’ll maintain a Write Ahead Log in the database”
It’s just that it’s easier to pull if it’s in the same db as everything else. The S3 solution is perfectly ok and 10x cheaper (.20$ pm instead of 2$), but the difficulty is, now you need to do a REST call to S3, and if you want the history per user or per day or per object, then you need to design all sorts of metadata to be able to fetch the right objects.
Aye - I don’t tend to have a problem honestly with “classic” lc tests. I tend to take issue with uncommon/nuanced Dynamic Programming questions and clever uses of binary search. Both of these categories tend to rely on the engineer knowing the “trick” of the particular question.
Perhaps I'm naive, but I suspect that hiring based on a candidates ability to produce optimal solutions to LC Hards will bias strongly towards candidates who can't code, but are very good at interviewing.
I get that perhaps you were using the proverbial "I" in your last sentence as an example of something everyone should be willing to do in/for their career, but does someone of your stature even need a real interview, much less a LC test, to obtain a top job? It's kind of funny imagining a young tech bro interviewing you, tbh.
I generally do not use clever algorithms in my code. I just use straightforward ones. Rarely, I might need a better one and go looking for it (like a better hash algorithm). I rarely use a data structure more complicated than an array, list, binary tree, hash, or single inheritance.
What I have, though, is decades of experience with what works and what doesn't work. (My favorite whipping boy is macros. Macros look like they are great productivity boosters. It takes about 10 years to realize that macros are a never-ending source of confusion, they just confuse and obfuscate every code base that uses them. I could go on about this! ...)
I have become pretty good at writing modules that minimize dependencies, and pretty good at the user interface design of a language.
But still, if the job wanted a leetcode test, I'd take it, no problem. I'd study up first, though.
If a young tech bro was interviewing me, I'd suggest he show me his best code, and I'd do a review of it :-) The point of that would not be to humilate him, but to demonstrate the value I can bring to improving code quality.
If I was being interviewed for a job writing a faster divide routine (the ones I wrote were shift-subtract, slower but bulletproof), a better random number generator, a cryptographically secure hash function, a tighter compression algorithm, a faster sort, I'm not the right guy for that.
P.S. I was once asked to review the code of a famous programmer I won't name. I was shocked to discover that the large codebase had 3 different implementations of bubblesort in it. I replaced them all with a call to qsort(). He asked me how I managed to speed it up :-/
My favorite blind spot is how our definitions of quality change over time. I knew someone who had a mature codebase which a new developer made substantially faster by removing his old optimizations. He’d measured very real improvements back when he made that hand-rolled assembly code on early Pentium generations but by the time we revisited it less than a decade later the combination of compiler and processor improvements meant that the C reference implementation was always faster. (I was assisting with the port to PowerPC and at first we thought that it was just XLC being especially good there, but then we tested it on x86 with GCC and found the same result)
Beyond the obvious lesson about experience and sunk costs it was also a great lesson about how much time you assume you have for maintenance: when he’d first written that code as a grad student he’d been obsessed with performance since that was a bottleneck for getting his papers out but as his career progressed he spent time on other things, and since it wasn’t broken he hadn’t really revisited it because he “knew” where it was slow. Over time the computer costs eventually outweighed that original savings.
I have done probably 500 to 1000 tech screens for big tech companies (as the interviewer) and this is completely true.
I have interviewed many people employed in tech as programmers for their entire career and they can't code. I don't meen leetcode, I mean they get confused trying to write brute force substring search. The nested for loop seems to be too complex for them to keep in their head all at once.
I have had numerous people cry. Again this was a screen, not leetcode. I'm asking them to check if text has mismatched parens, or find a substring. Things you do for homework in your 2nd programming class freshman year. Things every competent programmer can do while chitchatting about the job.
I would estimate that more than 10% of screens are like this, again these are employed people in the industry for years, sometimes tens of years.
Edit: I understand that people can get flustered, I understand that some people have trouble under pressure or while being observed. That's why I pointed out multiple times that the problems I gave are extremely, EXTREMELY easy. I'm basically asking them to write down their name and they sit and look at the pen like they've never seen one before. If you can't write a 5 line function to find whether a substring occurs in a larger string when given 45 minutes, your choice of programming language, and as many attempts as you need to debug and try again you simply will not be able to do any useful work as a programmer. If you don't know that to match parens you need to use a stack (or at least that it's one way to do it) you either have a very poor memory or no training in computer science at all - either of which is frankly disqualifying. Anyone borderline competent would invent a stack when presented with this problem if they hadn't already been told this fact a dozen times during their education or even light reading about algorithms.
I always try to give everyone I’ve interviewed the benefit of the doubt. You never know what’s going on in their lives, and even if they fail a trivial question it doesn’t mean they are faking the ability to code.
I joined Facebook back in 2018. Didn’t study at all for the interview and passed somehow. Then I probably conducted 200-300 interviews in my time there, so I became quite familiar with the questions. My performance ratings were all exceeds or greatly exceeds. I voluntarily left on my own after four years to join a unicorn startup. I didn’t prep for that interview either but passed it too. Well, the startup failed and many people went back to Meta. So I actually prepared quite a bit this time and scheduled a mock interview with them. The mock interviewer said I did great and not to change a thing. When it came time for the real screening interview... I failed the matching parentheses question.
I generally try not to make excuses. Almost every interview I’ve failed has clearly been my own fault. But in this particular one the interviewer kept interrupting me every two seconds and I absolutely could not think. I had done matching parentheses many times before in practice, but the constant interrupting rattled me to the point where I totally lost focus and bombed it. Not a great experience.
So yeah, I’d just recommend giving people the benefit of the doubt. Everyone has difficult moments occasionally, but it doesn’t mean they’re stupid or can’t code.
I run coding interviews at BIGCO. Half of the candidate success relies on the skills of the interviewer. A bad interviewer can bomb the best candidates.
Something I have changed my stance on a bit is automated coding interviews. I used to be adamantly against a company giving candidates automated code tests, but I see now that it takes the interviewer out of the equation.
Pre-screening, depending on how it’s done, could eliminate good candidates.
There have been times I’ve received answers in interview that weren’t the written answers, but I looked it up afterward and tested it out… and they were right. I learned something news and tweaked the answer reference as a result. If those questions were in the pre-screening instead of asked directly by me, it would have filtered out good people.
I remember fighting to get access to the pre-screen data to see what the answers were and find if there were any other cases like this, where the non-technical pre-screener was filtering out potentially good candidates, because we couldn’t give them exhaustive answers to questions being asked.
>but I see now that it takes the interviewer out of the equation.
Well yes, that's why I'm against it. A one way "interview" is an audition, not an interview. There's nothing worse than wasting 2, 5, 10+ hours on something that ends up with a template rejection letter.
That's great for the interviewer, but devastating for the interviewee. They can't even get feedback on how to improve.
Nobody is going to give an interviewee feedback, even if they are interviewed by a human. There is too much legal risk to open the company up to discrimination lawsuits.
There is also nothing worse than wasting 2, 5, 10+ hours on an in person interview to just have the interviewer flunk you our be unfair to you.
I still believe that personal interviews are important, I'm just raising the fact that a large portion of an interviewee's success is based upon their interviewer.
Yes, and actions have consequences I'm not going to audition if I have not at least talked to a human first. The game industry in particular does this to abuse Artists and Designers with spec work, so I feel especially strong about the power dynamic here.
>There is also nothing worse than wasting 2, 5, 10+ hours on an in person interview to just have the interviewer flunk you our be unfair to you
The interview stage inflation is definitely a problem, but speaking with actual people still has benefits. You get an idea of their culture and you can still network even in such a situation. It's not guaranteed but you get a much higher chance to get advice and feedback on or off the record if you're polite. People are flexible, some standardized exam may never even reach a human.
>'m just raising the fact that a large portion of an interviewee's success is based upon their interviewer.
Indeed. I'm just stating a viewpoint where an interview needs to be personal. An audition in this software space is about as impersonal as you can get.
As an interviewer at Google, we arent given an exact list of questions to ask or what to evaluate (there are broad categories).
It is really entirely up to each interviewer how the interview goes and they are usually scheduled between 2 other meetings so often the interviewer is distracted.
Yeah, rejected after previously working there. Was also going for a different role than I previously had. To their credit, I respect that they don't just hand out free passes back in.
Welcome to the wonderful world of coding under pressure, which almost never occurs in the real world. It's a non-issue when you're young and don't feel the pressure. But when you have grey hairs in your beard and know you're already walking in with two strikes, all of a sudden the fog of war kicks in.
> I have interviewed many people employed in tech as programmers for their entire career and they can't code.
Think about this contradictory statement for a while. Can it actually be true? Or is there something else going on?
If the interviewee has nothing but sub-1yr stints on their resume, perpetually getting fired before vesting at any company, then yes, it's very possible they actually can't code and just fake it at every interview.
But everyone else... if they have spent years at tech companies writing production code then obviously they know how to code. They might be great or maybe mediocre, but guaranteed they at least know how to code.
So, if your interviewing technique is concluding something that is obviously impossible, then start by considering how to improve the interview technique.
I’m an electrical engineer who does circuit design. I’ve interviewed many electrical engineers over the years and the situation of applicants having “years” of experience while simultaneously not knowing how to design a single circuit is real. In our field it’s usually because although the person’s title is engineer, in practice, they don’t do any engineering. There’s just a ton of peripheral work (basically paperwork related to operations and compliance), which is very important, but is not design.
My guess is computer science has a similar issue.
Lots of people with programming in thier job title but they don’t actually program. And based on ltbarcly3’s empirical measurement, “lots” is above 10%. ;)
10% of applicants that make it to a phone screen. I estimate that the number is much lower than 10% overall because incompetent people with good looking resumes tend to do a lot more interviews than good candidates.
"claim" kills? Isn't part of the HR screening about some litmus test to make sure they aren't very obviously lying?
And yes, this is part of why the obsession with FAANG on resume is very overrated. Very few companies require the skillsete FAANG needs. Some of thst FAANG culture is orthogonal to what medium/small sized companies require.
I don't think you read the thread. They did have those jobs, and presumably they were on teams that did vaguely what is on their resume. No HR doesn't investigate candidates or do background checks before you interview them, that would be very expensive and silly.
I feel like you kind of vaguely are aware of these topics but have never actually had a job or something because you seem completely unfamiliar with the basics of how hiring is done in the industry. Are you from Eastern Europe maybe?
Quite the opposite (and sadly, still. American). I'm just so frustrated how I'm 10 years into this career and the interview process feels more random than ever. I can apply to 100 job and interview for 10 that all fall through. Then I can just go to a bar and trip into an opportunity I wasn't fully expecting. What does that say about the interview process?
>No HR doesn't investigate candidates or do background checks before you interview
An HR screen isn't a background check. It's "can you talk about your roles and provlems solved like you actually did it. A good HR screen should make sure they aren't blatantly lying
>that would be very expensive and silly.
Let's both not pretend the interview proces is in any way optimized for any metric. You have often non-tech roles create a description for a tech role (leading to famous blunders like "have and jave script is the same") . You have an increasing amount of rounds of interviews to go through for a job that may not exist or may already be reserved. And more and more of the parts are being outsourced, leading to power quality candidates. All that before throwing a reckless reliance of AI on everything.
The most optimal hiring is to focus on high quality hires brought in as fast as possible. Or not to hire if you don't need to hire. But we're not really running on sensible business practices these days.
You keep saying 'we'. The companies I have worked for have on average had very streamlined interview processes, with the exception of very large (>2000 engineer) companies. I don't know how to fix hiring for companies that big. I'm sure you have a lot of 'ideas' that are really just pointing out the problems. We all see the problems, but fixing them means either empowering individual managers (many of which would just hire their cronies instantly if allowed to) or inventing new ways to do HR which is obviously nontrivial, especially with incumbent HR morons fighting you.
An HR screen will sometimes remove frauds (not usually because they can't really tell, it removes uncharasmatic frauds only), but it also has a very high probability of removing anyone on the autism spectrum - no matter how qualified. I really don't want people on the autism spectrum removed from my hiring funnel for software engineer. If you ever looked over the shoulder of a recruiter or HR when they do a first pass on resumes you will be horrified at how many of the best candidates they pass over because they have no idea what they are reading, and how many very poor candidates they pass along for very stupid reasons like having 'Yale' for their college (despite it being for History and despite it being the extension school, true story, and this person ended up getting hired despite negative interview feedback and then fired for incompetence a few months later).
> An HR screen isn't a background check. It's "can you talk about your roles and provlems solved like you actually did it. A good HR screen should make sure they aren't blatantly lying
Correct. While not perfect, if you have good HR hiring team they will do a decent job at feeling out the people before they get to you.
> I can apply to 100 job and interview for 10 that all fall through.
Part of the difficulty is that (despite any myths around engineering shortage) there are so many qualified people for every role that it is overwhelming.
I just opened a new job req last week, I have over 1100 resumes in the queue already. And this is a pretty specialized technical role in a specialized department, not a generic "javascript software engineer" role, either.
Obviously I can't read all of them, which makes me sad because someone took the time to send their resume and I feel like I should give them the respect of reading it, but there are simply not enough hours in a week. While HR does the screening, I also go and do a random sampling of the resumes and everyone who has applied seems at least moderately qualified. But of course I can only talk to about 1% of them at best.
I know a lot of people who can tweak code that already exists. However, if they are sitting in front of an empty text editor and given a goal, they don’t know how to break the problem down and build a solution with the tools the language gives them.
In a large environment, someone may rarely need to start from nothing, so the interview format throws them.
That said, I think being able to break down a problem to solve it with code is a really important skill. Without it, the person will always have to lean on others to fill that skill gap.
> I know a lot of people who can tweak code that already exists. However, if they are sitting in front of an empty text editor and given a goal, they don’t know how to break the problem down and build a solution with the tools the language gives them.
But being able to break things down and come up with a solution is not necessarily something that needs to be done quickly, in the time you have for an interview. Quite often I've been faced with a new problem and done absolutely nothing visible for ages. Literally just reading around the problem, asking questions, and sketching in my mind without writing a single line of code.
This is often faster and better than starting immediately.
There's another extreme, too: people who can code up an app in a matter of days, but can never learn to navigate and successfully maintain a legacy code base (even their own!).
And there's no easy objective way to screen for this in a job interview. So they pretend it doesn't exist and never ask questions like: "Suppose you're designing a green-field app that you think will grow over time like X, Y, and Z. How would you design and organize your code so that the app stays maintainable and flexible over time?"
I could talk for hours on this subject with concrete examples if anyone ever asked.
I think another problem is that there are so few engineers/architects who really "get it" on this subject. I can only think of a few ex-coworkers with whom I could have the kind of in-depth conversation about app design and organization that I'm picturing.
I've never worked in big tech, always for startups or non-tech corps with a few rock star devs and a lot of decent devs. So maybe it's different at a FAANG. But in my head I'm picturing a bunch of algo-geniuses whose code turns into a big mess over time when requirements take a right-turn and break all their beautiful abstractions. I've worked on a few apps like that and it's not fun.
Nothing on interviews is objective to begin with, so we can discount that.
Your template sounds like a fine way to ask such a question. I think the issue is managers or someone above simply don't want to invest in proper interview questions and instead just do that FAANG does. Even though very few companies need such core algorithmic knowledge but need people who can properly navigate legacy code. FAANGs will actually make sure new hires learn the code base, unlike many companies thst want you to "hit the ground running".
I think this is why people job hop. They never have to support their unmaintainable greenfield app. Without the pain of those mistakes, they never learn to make the next one better either.
The goal when hiring isn't to find someone who is barely skilled enough to just slightly come out as better to have on staff than not. In fact these are often the worst hires as they slow everything down and create work for you to find tasks they are capable of, but it seems unfair to fire them because they aren't totally useless all the time. You hire the best candidate overall, and you are very far from describing that.
2. While I said the goal was to hire the best candidate, this was as compared to someone who is just barely better than not filling the position. Hiring a top 5 candidate is also very good. The argument in GP was (paraphrasing) "I know people that aren't good programmers but they can manage to do little changes to existing code therefore you shouldn't give interview problems these dummies can't solve."
Too right. I had an idiot manager who hired the next person with a pulse - we got some money-motivated guy that complained about everything and didn't listen to what he actually needed to do, and my manager nearly destroyed the team by pretending that he was fine.
I thought that too until we had a guy start two years ago that interviewed well and answered questions like he knew what he was talking about but then when he started, one of his first questions was how to figure what endpoint was being called by a website (as obvious as looking at your browser's Network tab, or browsing the website's source to find the endpoint). Not long after that he had to clear a couple more values as part of a cache invalidation procedure and was utterly bamboozled by the if (!cached) { clearCache(); } else { cache(data); } logic. That was an exceptionally draining few months until he was sacked.
What is really crazy about this though is that sometimes it really is the interview setting, as people unused to interviews can, at times, emotionally collapse. I have seen people who are actually good programmers get wrecked by simple questions because of their inability to handle stress, and how the lack of interview practice turns a simple exercise into a hellscape.
It's not as if testing for performance under stress is useless: Tough on call rotations happen, and you might need someone that does well under pressure at 3 am in the morning. But the picture you get on a screening isn't as clear as it appears.
Not to imply that you're wrong, but I've always been bad at LC interviews, but surprisingly (for myself) passable when called upon to troubleshoot and code up a hot fix in the middle of the night. Maybe those are not entirely similar types of pressures.
There is a big difference between solving a critical bug, which is your job, vs. passing an arbitrary leetcode test from a disinterested 22 year old where you might lose your house if you get it wrong. You have 15 minutes.
It’s also not the same kind of stress. I’ve interviewed many a candidate who were sweating and/or shaking at some point of the interview (with heavy reassurance and me trying to to steer the conversation towards and area where they feel stronger), but they can often end up being calm and reliable alert responders nonetheless. So far I’ve seen very little if any correlation between being able to handle interview stress and on-call stress.
Leetcode performance isn’t going to be representative of 3am incident response though. If that matters, you’re probably better off asking a classic “tell me about a time you responded to a page in the middle of the night” type question.
> Tough on call rotations happen, and you might need someone that does well under pressure at 3 am in the morning.
On a stop-the-world production incident at 3am I know that codebase like the back of my hand, and my job very likely doesn’t depend on whether I solve it in the next 30m. There’s barely anything stressful about it.
On an interview, with my future on the line, and presented with an unfamiliar problem?
You are still not testing if they can code. But whether under the scrutiny of another person, they can code. Maybe this is important to you, maybe peer programming is important to you for example. I remember a young dev who couldn’t pass such tests, as he would get too nervous, yet he had written the core tech for many shipped products. The type of guy you would just stick in dark room. With age this type of stuff settles down.
“Scrutiny of another” who’s a peer or even a client is super different from an interview.
The latter has more in common with an open mic night, the prospect of which the vast majority of people are terrified and would break down if they attempted it.
My first attempts at public speaking were a frozen horror. But I kept doing it, and it kept getting easier and easier.
I recommend that when there's an opportunity to get up in front of a mike, take it, and get comfortable with it. It's a skill that will serve you well.
I agree with both you and the parent to some degree. I have experienced interviews that went badly because they were in an IDE I wasn't familiar with, or in Leetcode (never used it). So I'm distracted by how to use the tool rather than how to code the solution. But I can also see how stuff like finding a substring is easy - most languages have a function for it. But at that point, you're testing different things - problem-solving vs rote memorization of a provided function.
I feel like a good middle is to allow Google for documentation searches, not solution searches. Without searching (or IDE with radix completion), I'd probably fail the test for not knowing the syntax off the top of my head.
Not everyone uses it (or their language equivalent) frequently depending on the needs of their project. If I'm really using it frequently, then I would probably be looking at higher accuracy/resiliency options like parsing the input and comparing to a map, or regexes.
As someone who does these types of interviews on both ends, something like this is why I like to start with an outrageously simple question to break the ice.
Surprisingly, I have an 80% fail rate on the first question usually, which is just “find the second largest number in an array of numbers” for a 5 YOE role.
I just saw that this comment was fading from downvotes.
I think HN is just overwhelmed with people who somehow got hired into technical jobs but can't do them. Anyone who thinks that 'finding second largest number in an array' or 'finding if a string is a substring of another string' are hard, or that the pressures of an interview are a valid excuse for being unable to do it are simply not intelligent enough to be successful in this field and they are casting about for some excuse to protect their ego. These are problems that a bright 12 year old with no programming experience could do for fun, and these folks are purporting to be educated, experienced, professional programmers.
As much as people like to say 'muh anxiety', I've yet to meet someone who can't do very basic coding in an interview but also they are capable of basic coding in any other context. I would suggest that this sort of person is so rare that you will probably never meet one over the course of a normal career.
Elsewhere in this thread there is literally an example of someone who, by their own description, worked at a FAANG company for some years getting good performance reviews, has enough interview experience that even if they couldn't generally code their way out of a paper bag they were surely good at interview-question coding, ... and, under pressure from a pushy interviewer, completely failed to do a straightforward coding task.
I guess they could just by lying "to protect their ego" but in that case why post their comment at all?
It really is the case that some people sometimes completely go to pieces when stressed, and mocking them by saying "muh anxiety" doesn't stop that being true.
I will say that, as the interviewer asking the question, I do recognize that people have anxiety, and I would prefer to nudge them out of a spiral, because a bad interview is generally not pleasant for anybody involved.
To some degree the interview is not about getting the question right but also how you respond to questions about your answers. I would rather have someone who started with a totally incorrect answer and then reason back and forth with them til they got on the right path, than someone who came up with the right answer and then clammed up and refused to explain how they got there. (I once had a candidate tell me, verbatim, "I don't think analyzing performance of an algorithm is part of an engineer's job.")
I would prefer if there were a more humane, less stressful, and scalable way to do an interview. The problem is that as a profession we lack the continued examination/licensing of other fields like engineering or medicine, so we don't have a good barometer of skills otherwise. And I often have to sift through massive numbers of people clearly overshooting their shot to get a good position.
The solution to the problem in question is about 9 lines of very simple code. In a 45 minute interview, that is 5 minutes per line available. It's about 1 character every 40 seconds. I think anyone can get flustered and mess up. It's possible for someone, on a bad day, lack of sleep, hung over, experiencing an anxiety attack to mess this up, but it's really really a stretch. This is a VERY EASY task, and if someone has done it before there's nothing sneaky or hard to do here. The version where you match different kinds of braces is very slightly more complicated but actually less lines of code.
Yes maybe someone can't come up with this in an interview. My point is 99.99% of the time when someone fails this they are lying about their work history or they are so stupid they can't hope to do the job. 00.01% of the time they could do this easily and the stars aligned to give them a rare bad day. The best way to deal with that is to just assume everyone who fails this can't do the work, and the 00.01% person just interviews somewhere else and doesn't have another bad day. I don't even know what the alternative to that is, just hire everyone who claims to be qualified and hope for the best? Give everyone who demonstrates very very strong incompetence signals 2 or 3 more tries?
def matched(str):
count = 0
for c in str:
if c == "(":
count += 1
elif c == ")":
count -= 1
if count < 0:
return False
return count == 0
I completely agree that this is a very simple problem; I'm not at all arguing that it isn't. What I am arguing is that this doesn't make it super-unlikely that an otherwise competent person screws it up. If it's really a 1-in-10,000 event, isn't it rather surprising that we've got someone right here in this thread to whom it happened? Someone, moreover, not just barely competent but able to do well with a demanding employer, and who has a lot of exposure to this sort of interview question?
I think this is good evidence that, in fact, it's not vanishingly unlikely that someone genuinely competent flubs what should be an embarrassingly straightforward coding task in an interview.
That doesn't, unfortunately, mean I have a good suggestion for a better way of telling who is and who isn't capable of writing code. Maybe this is the best we can do. If it really is just a matter of "sometimes, at random, people have bad days"[1] then provided it's fairly rare this issue doesn't matter too much. I'm more worried about the possibility that it's more "a smallish fraction of otherwise-good people often have this sort of bad day", because then that smallish fraction of people may be getting completely overlooked, which is bad for them because it may take them ages to get a job and bad for employers because it effectively reduces the pool of good people to hire.
[1] In this particular case, the "bad day" seems to have included an obnoxious interviewer, which it's reasonable to hope wouldn't be repeated at another company's interview.
But, again, pointing out a problem unfortunately doesn't guarantee having a good answer. But, equally, not having a good answer doesn't mean the problem isn't real. It would be very nice to believe that all the people failing interviews because they fail to solve an easy coding problem are undeserving incompetents who should be bombing out of the interviews, but it looks to me like that ain't so.
That's fine. Lets say 5% of the time a world class programmer would get this wrong, but 100% of people who can't program get this wrong.
Lets say we interview 100 (N) people for a role. (100 is a lot but pick a number yourself and follow along)
The probability that a random interviewee is a top programmer is lets say 1/50 (T). So in our pool of 100 we have on average 2. Give everyone this problem, and if they get it right they move to round 2 which is 4 more interviews. Lets say 20 (P) people pass to round #2.
We are paying for a total of 100 + 80 = 180 interviews (N + 4P). To give everyone who failed the first round another try would require another 80 (N - P) interviews. So for almost 50% more cost (N-P)/(N + 4P) we gain (T * N * 5%) = 0.1 additional top programmer making it to round 2 on average. Just interviewing another 80 people (same cost) would get us (T * 80 * 95%) == 1.52 additional top programmers to round #2 on average. There's basically no reasonable % you can pick for how often people screw up where it makes sense to do anything but just ignore the possibility.
Sure, but now you're making a different claim from the one you made before.
Before, you were saying: anyone who says "muh anxiety" is just trying to cover for the fact that they're incompetent.
I suggested that we've got pretty good evidence that interview stress really truly can, and not only-vanishingly-occasionally, make competent people fail to do simple interview-type coding tasks.
So now you're advancing a different claim: even if some competent people appear incompetent in some interviews because of the stress of interviewing, you should make interview decisions as if those people are just plain incompetent, because most people who fail to do simple interview-type coding tasks really are incompetent and identifying the few who were just stressed out is difficult.
That could very well be true! But it's not what I was arguing against before.
... But I'll argue against it just a bit, even though I mean it when I say it could well be true.
1. There may be much less inefficient ways of catching the competent-but-stressed candidates than "just interview everyone twice"; completely incompetent people probably don't interview exactly the same as competent-but-stressed ones. (I don't know for sure because, of course, when you interview someone and they don't perform well you don't generally get to tell which category they were in. But I bet it would be possible to find out.)
2. If the situation is that some people are particularly susceptible to interview stress (but that this doesn't make them bad at actual programming jobs unless they're in an extra-stressful environment), the "eh, just reject them" strategy might be good for companies that are hiring but punishingly bad for the people affected who just can't get a job because they are bad at interviews. If there were a good way to identify people who are good but fall apart readily under interview-stress, that might be a big deal for those people. (Which someone on the hiring side might not care about, of course; but it's sad when some group of people gets systematically screwed over.)
I repeat, as I have said before, that I don't have a solution to this problem nor even good reason to believe that there is one. I'm just pushing back against the blithe assertion that there isn't a real problem here.
As an interviewer, my company allows the person to pick the language of their choice. At that point, you could just write it in psuedocode if it came down to it, and if you actually knew what you were talking about it doesn’t matter all that much. It’s not like I know the in and outs of every single language people have answered the question in.
In an era where linters and autocomplete and IDEs have existed for decades it’s silly to fail an interview based on tiny error checks.
> 'finding if a string is a substring of another string'
Unless you mean using a "substring" function or regexes... coming up with the naïve O(n^2) algorithm is indeed easy, but finding a fast algorithm like KMP is, I would say, non-trivial?
For the "second-largest number in array" it's rather easy to come up with an O(n) algorithm, so I would consider the two cases to be different.
It's rather funny, but I wonder if you've ever been a Game Master? One of the most common problems running a game, is making the puzzles simple enough. You have to use kindergarten or back of the cereal box level puzzles or the players will fail.
I played with engineers, it's a game, and there is nothing riding on the outcome and we often failed stupid simple puzzles.
Interviews have a lot riding on them and yes stress is terrible. Stress eats working memory for breakfast, NASA learned that the hard way.
At that point I have to ask if your screening is really working. There's hundreds, thousands of potential SWEs that can be productive, but you keep picking people who struggle with CS101 problems?
You sure it's not nepotism, or an inability to make sure the resume fits the person? Or any other number of non-merit based criteria from HR?
You just reminded me I said it on a programming interview for Java one time and my friend was giving them the Fizzbuzz test.
And, and having to explain everything, well, this person had never heard of it. And, and for the more, didn't know what modulo was, like, how do you, how do you say you know Java?
I suck at programming but I could bang out 100% of the things mentioned in this thread. In like four languages, including FORTRAN.
Typically speaking, when you encounter a problem in excess I believe you should start to assume that perhaps the problem is on your side.
For context, back when I was interviewing I encountered many interviewers who process involved simple questions with simple solutions but with extra gaslighting on top. 'Are you SURE that's the right solution?' while I'm in the middle of throwing out a correct answer. It's actually very easy to guide someone away from a correct solution into the wrong one by simply making them second guess themselves under pressure.
It was something that I started to be extra cognizant about when I was the one interviewing people, that it's really quite easy to throw someone off track.
The other replies to this comment aren't wrong but I feel they fail to take into account that (especially with startups): having someone watch you code during an interview is one of the the least stressful experiences the person will face with the company - once the job begins. Companies can't take on the risk of dealing with a bunch of passive-aggressive bullcrap once the real code reviews begin after the hire. Most tech job reqs contain the words "Excellent written and verbal communication skills" on purpose.
Imagine a person with a pilot's license refusing to fly tandem for an airline interview.
> Imagine a person with a pilot's license refusing to fly tandem for an airline interview.
That’s not a great comparison since a pilot has already passed substantially harder tests to get that license and a flight is exactly what the job is. If you have a candidate with a pilot’s license you can assume at least a baseline level of capability which you can’t assume for a software engineer. That has pros and cons but it definitely means interviewing is a noisier process.
The other problem, however, is deeper: the job of flying a plane is exactly what’s tested to get a pilot’s license but what many places do for developer interviews is wildly unlike the actual job so it’s more like interviewing pilots based on trivia questions about the number of rivets on a B-52 and how well they can solve 3-D puzzles, and then being surprised when there isn’t much correlation with real world performance. For example, only at the most toxic companies will the interview be one of the least stressful parts because the rest of the job is a team effort. What makes the interview challenges stressful is doing it without your normal tools while someone else is looking for reasons to fail you, but in a normal job your coworkers are trying to help you succeed because even if you’re not friends you are all better off when your company succeeds. At a startup, trying to ding someone for trivia challenges is like hitting the iceberg to prove that the navigator made a mistake.
Go on youtube and watch NTSB / FAA investigation reports. If you think that a pilot's license demonstrates that someone is a competent pilot you are very very mistaken, and there are scores of counter examples.
Airlines very strictly test pilots and insist on check rides regardless of what official qualifications pilots show up with. They do this because they are professional organizations with decades of experience and it is necessary for safety. Giving someone the benefit of the doubt because they manage to meet the absolute minimum standard that makes operating an airplane legal is insane.
Yes, that’s what I was referring to with “baseline level of capability”. It doesn’t mean everyone is the same, or that they’re all super pilots, but anyone with a commercial license has at least 250 hours doing the primary job function and that makes it’s reasonable to ask them to do a test flight.
In contrast, we don’t have anything like real certifications for developers and many of the interviewing questions are very unlike the actual job. Hiring would be easier if we did have something closer to what a pilot’s license conveys, but that’d also slow the field down since programming has changed a lot more over the last 50 years than flying.
Dude, I have no idea what a stack is because I never studied computer science. It’s never held me back except during interviews with people that expected me to understand computer science fundamentals.
If someone asks me what the time complexity of something or the other (I literally had to go look up the name), I completely freeze up.
Whether you know the term doesn’t matter when you understand the basic concept that for inside for == bad.
Anyhow, it seems to be working out pretty well for me, so while I’m also annoyed by incompetent people the idea that it’s due to lacking CS fundamentals sounds bizarre to me.
It filters out the complete frauds.
Yes, there are people who know all the right things to say in a job interview, but cannot code at all. If you hire one of them, it takes a bit to find out they cannot code, and a bit longer to fire them, so you're out $$$$ paying their salaries for nothing.
For example, a recruiter I know will ask a tech candidate "what is 20% of 20,000?" A significant percentage cannot answer the question. Some even cry. It's shocking.
A friend of mine was looking at getting a FAANG job. He was worried about the leetcode tests. I suggested he spend a month going through the leetcode books studying them - that the return on his time investment doing that will be one of the best ROIs he's ever done. He did, and got the job. (Although the LC was just a first gate one had to go through to get to the real job interview.)
Personally, I have no idea how I'd do on an LC test without prep. But I don't have a problem with studying it to get a top job.