In the interviews I do, I tell the candidate that:

1. There is no time pressure. Work at a normal pace, as if you were working here. This is not a speed test. I don't expect you to finish. I mainly want to know how you think.

2. You should look things up. Behave the same way you would when coding at home. Use Google, Stack Overflow, documentation, etc.

This probably works better for the interviews I give because the problem is not implementing an existing algorithm. It's a realistic task, something that we've actually built on-the-job.

The resulting cognitive dissonance adds more pressure to the situation instead of relieving it.

That's hardly the case. Hiring for programming is different from hiring for a cashier. The programming world, at least, if you hiring for good programmers, would rarely have a queue of candidates lined-up. You get tons of resumes daily, sift through BS, find someone good, get to know him more and after some rounds, you hire him if he is worth his salt.

Rarely you would hear the phrase, "We hired the other candidate because he was skin-of-teeth better than you." Judging from the lack of engineers, a company would be very lucky to have more than one really good candidate for a position.

there was never anything close to anybody getting "cut" in favor of another interviewee that was fractionally better than they were. we fully intended to make offers to everyone that we thought was good.

I think you are being a bit negative. Why not believe the interviewer? It seems pointless to trick candidates by suggesting they are safe to look things up, then secretly judging them worse than other candidates. I would judge more harshly someone who didn't ask if they could look something up, or assumed they couldn't. Particularly so if they had been told they could but then proceeded to struggle and not look something up or ask the interviewer. I'd rather someone I may work with have a relaxed attitude and inherent curiosity rather than a paralyzing fear of failure.

Also viewing it as fundamentally a competition is focusing a lot on the negative perspective from your side. An interview is about finding a good match between a company and an employee. It can be just as much about you finding somewhere you want to work as it can about them comparing you to other candidates.

There may not even be other candidates, and they could be trying to make you comfortable and give you an idea of what it would be like to work together. Maybe suggesting that they do understand the cognitive dissonance and they do understand that interviews are stressful and hence no need to add additional pressure.

Not at all.

I have one hour to decide if I want to work with a candidate.

How quickly someone works? Sure, that is important.

How good is their sense of humor? Can they joke around? Can they be involved in a technical deep dive on a problem? How about coming up with test cases?

I have 60 minutes and one programming problem. I've hired people who've taken from 5 minutes to 40.

People have very different working styles. Since interview as a process can't be tailored to all of them, it is rationally refined to suit for the majorities. For the minorities left, I have to say one has to adjust, trying to go through the 5-hour drills.

I, unfortunately, is one of the minorities. Given a hard question, one has to think first. I am the opposite of thinking aloud. In real work, I either stare at walls, or look at the vast void far away, silently. I might have a piece of paper, drawing odd shapes and graphs, because I am primarily a visual thinker. In the interview, it would definitely look odd. It could be awkward as well if I completely fall into silence, ignoring the interviewers.

So I would face to the white board, wrote a few lines of pseudo-code, uttered some words, while trying to get my inner self into my real work routines. Sometimes it works relatively well, sometimes it failed completely, especially the interviewers kept talking that requires my interaction. In many cases, I understand the interviewers were trying to help, giving my hints, while the effect was the opposite of their intents.

However, if your approach is so radically different that I can't see how you got there, you may not be the ideal candidate.

Don't worry about being different in your approach. It is, however, very important that you are able to keep someone on the same page.

Edit: autocorrect

At my next interview, I'll bring a bed, ideas come when I lay in and stare at the ceiling.

Problem solving is similar in that your current environment constrains your thinking. Getting up and changing your environment changes the constraints on your thought processes. The bigger the change in environment, the more constraints removed and the greater the opportunity for creativity.

We've found this works really well. There are unit tests, so its always a promising sign if they go for them first. Depending on what they pick up, change or ask questions about, lets us a lot about the candidate.

Having control over your nerves when s* hits the fan is a valuable quality, which we should all strive for.

Most of these interviews are tests to see your approach to solving a problem. The best practice is to come up with a strategy before you hit the first key, ask as many questions as you can and find as many corner cases as you can. All the sane interviewers will appreciate that even if you don't end up with the best solution. And please treat all interviews as a learning experience, you will be much better off.

The adversarial and extreme time boxed nature of a coding interview is something that is truly outside the day to day experience of the vast majority of programmers.

>Having control over your nerves when s* hits the fan is a valuable quality, which we should all strive for.

That may be true, but think about that for a minute. Go back to college and think of how many people in the class were comfortable going up to the board and solving problems in front of everyone.

When I was taking Automata, we could get an extra point on our final grade by going to the board and correctly working out a problem that we hadn't seen before. Only myself and 2 other people ever did it--in a class of 60.

Do you think that your company is solving problems so hard and paying so well that you can can only consider 3 out of every 60 developers (or whatever the real ratio is) who are otherwise qualified who have also mastered control of their nerves far beyond what is required 99% of the time on the job? Maybe if you're Google, for the rest of us we need to find a better solution.

By the way, even though I was able to solve problems at the board, I hated every minute of it, and I refuse to work for companies that require this kind of interview.

So you've never taken an exam in your entire life?

I don't mean to be snarky. I just don't understand this extreme disdain for coding interviews, even when they're very forgiving and flexible. Yes, exams aren't fun, but they're necessary.

Even if you can look up whatever you want, use whatever language you want, take as long as you want, do it right on a real desktop (not a whiteboard), and if you're asked a reasonable, real-world problem, not an algorithmic brainteaser, you'll still have people say it's a horrible process and it's unfair. And that's already way more forgiving than any exam I ever took in school.

I mean, are programmers expected to not be able to do anything at all in an interview setting? Like you can't be expected to produce any code of any kind, as if you don't know how to program at all? I think it is a bad sign if someone who is a talented programmer utterly buckles under a little bit of pressure. To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

So what, if I hire you and at some point need you to hotfix something in, is that unfair? Maybe I do know how to do it myself but don't have time because I'm busy with something else. That situation doesn't seem all that different from the interview, except it would actually be higher pressure because there's a real problem, not a fake one.

You missed the context where we're talking about a professional setting.

>And that's already way more forgiving than any exam I ever took in school.

You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

>To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

And you'd be wrong. I've worked with very talented programmers who are absolutely terrible at interviewing. The problem is you're biased by your experience. You are probably good at this kind of interview and you are probably a good programmer, so you assume that any good programmer must be a good interviewer.

I know for a fact that these 2 processes are orthogonal. Books like cracking the coding interview exist because it is possible to practice and game the system. I've met just as many people who are good at interview questions, but are terrible software engineers, as I have people who are bad at interviews and are excellent software engineers.

The fact is that our current hiring processes are broken. Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this. Newspaper writers work on deadlines, but they don't have to write an article on the spot while someone watches them type.

Studies have constantly shown that work sample tests are really the only form of interview that shows real predictive power (that and general intelligence tests). The closer you can make the work sample to real work, the better. Finding the nth item of some arbitrary sequence in 15 minutes while someone watches your every move is not close enough to real work to have much predictive power.

>So what, if I hire you and at some point need you to hotfix something in, is that unfair?

If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there. I'm also probably very familiar with the domain, and it's not some artificial problem that I may have never seen before. But more than that, why don't you see the difference between a hotfix and an artificial situation where someone who already knows the answer is standing there watching your every keystroke, and judging your value as a programmer?

There are better ways to interview programmers. Here is one:

Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Repeat this process if necessary.

Get everyone together and talk about past projects the candidate has worked on. Have the interviewers evaluate the candidate, and make your decision.

You know whether the candidate can code, and you've removed the adversarial nature and unnecessary stress from the interview process. Most importantly, you got to see them work in a much less artificial environment that's closer to what they'll be doing day to day.

>You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Yes. Is this not common? I've had a number of teachers who, as a graded form of evaluation, would ask us to present a topic to the class, and/or ask questions about it. This was extremely common on high school, and happened once or twice in the more advanced courses in my college. I also had to defend my thesis to graduate, which was basically a more lengthy version of this.

I live in Argentina. Maybe it's a cultural thing.

> would ask us to present a topic to the class, and/or ask questions about it.

This is common in the US as well, but presenting on a prepared topic and solving novel problems are 2 completely different things.

That being said, a professor student relationship isn't analogous to an employer employee relationship. Hopefully an employer wants to make the interview process as enjoyable as possible to attract as many qualified applicants as possible.

I didn't miss it. I just don't see why it's relevant.

Why does your ability to handle that situation differ so greatly depending on whether it's an academic or professional setting?

> You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Well, when we give technical interviews we let the candidate sit at a desktop and we don't particularly pay much attention to what they're doing. We definitely don't just stare at them. I know that at some popular companies like Google they may use a whiteboard and they might (but might not) pay more attention as you write your code. I don't really advocate that.

I know that's not always the case, though; I interviewed at Palantir once, for instance, and one of the interviewers ignored me completely until I was ready to present my solution.

That said, in school there were in fact a few occasions when I had effectively an oral exam. There also were occasions when I needed to do some work on a chalkboard in front of several people (if not the entire class). There were some occasions when I even needed to deliver a 45 minute highly technical presentation to the class and that doubled as an exam.

> Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this.

Well, my roommate's a chemical engineer at a major oil company. For his current job, he had to give a presentation to the entire interviewing team about his technical work at previous internships. It's not an exactly isomorphic scenario, but I'm sure they paid a great deal of attention to every single thing he said.

Lawyers have to take the bar exam, which I imagine is easily just as hard as an interview at Google. It might trigger social anxiety less, but that won't last long -- by the nature of the profession, you'll be in front of the court eventually.

Doctors have to take similar exams, equal in difficulty (if not greater), plus they are observed extremely rigorously during labs and clinical work.

> If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there.

It's not that regular of an occurrence, but it would be a sort of big deal if you were totally unable to act in that sort of situation. There would be a limit in what sorts of projects you would be given.

"Life and death" is an exaggeration, though. It's not any more life and death than a 15 minute quiz.

> Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Are you suggesting that you can perform well when you have someone helping you, but you can't when you don't? Or is it just an issue of social anxiety?

I didn't say it did. But just because something happens in a university doesn't make it the optimal way to conduct an interview.

>I don't really advocate that.

That goes a long way towards alleviating the problems that most people have with this type of interview.

> For his current job, he had to give a presentation to the entire interviewing team about his technical work at previous internships.

Talking about past experiences is completely different from solving problems under pressure. Almost all professions require this, and I'm at a loss as to why it's not good enough for software.

>There were some occasions when I even needed to deliver a 45 minute highly technical presentation to the class and that doubled as an exam.

Again not really the same thing at all. Delivering a prepared presentation is an entirely different issue. And if public speaking like this is required for the job, I see no problem in including it in an interview.

> by the nature of the profession, you'll be in front of the court eventually.

Yes, you said it, that kind of thing is an essential part of the job for lawyers, not so much for software engineers.

>plus they are observed extremely rigorously during labs and clinical work.

Yes this is true, but this happens when they are just starting out. No one is going to take a surgeon with 10 years of experience and force him to do an operation for an interview.

>It's not that regular of an occurrence, but it would be a sort of big deal if you were totally unable to act in that sort of situation. There would be a limit in what sorts of projects you would be given.

Again, I have never once encountered a situation where I had 15 minutes to solve a complex problem that had never seen before.

>Are you suggesting that you can perform well when you have someone helping you, but you can't when you don't? Or is it just an issue of social anxiety?

I'm not suggesting that. I do fine in technical interviews the same as I did extremely well on exams in school. However, even though I often made the highest grade on an exam, I hated every minute of it, and I feel the same way about technical interviews.

Here's the bottom line. I do fine in technical interviews, but they cause me more stress than they're worth to me. You may be fine with this kind of interview, but from the comments on hacker news a very large percentage of software developers are not. Many people hate them, and their predictive value is dubious.

The interview format I outlined is objectively more similar to the day to day work of an average developer. Studies have show that work sample tests are the best way to judge a potential employee. Why not strive for a better interview process?

It really comes down to this. If you can eliminate a section of the interview that many people dislike, which isn't critical to the performance of the job, and it will increase the tests predictive value, why wouldn't you do it? Do you want to hire people who are good at interviews or do you want to hire people who are good employees? If company A depends on an interview process for which 50% of otherwise qualified interviews will perform well on, and company B has an interview process for which 60% of otherwise qualified candidates will perform well on, which company will perform better in the long run?

Current technical interviewing techniques are demonstrably terrible. Their predictive power is terrible. The only argument is whether there is a better way to do it. I think there is.

I honestly think you're sort of dodging the question here, maybe not intentionally.

> I'm at a loss as to why it's not good enough for software.

Probably because you can say anything you want to impress people, which means that as an interviewer you can't really trust anything they say and must probe them very deeply about any work they've claimed to have done. At that point it's already a sort of technical interview.

We actually use a mix of this. We don't require a formal presentation but one of the interviews involves having the candidate discuss past work in detail.

We definitely would not feel comfortable eliminating coding interviews on the basis of that one interview alone. In the past, we've hired some people on the basis of things like this, ignoring poor results on the technical screen, and those have been our worst hires.

> No one is going to take a surgeon with 10 years of experience and force him to do an operation for an interview.

Well, a surgeon's literal every move is being observed during every single operation. And to get to that point, the surgeon was basically subjected to excruciatingly rigorous daily oral examinations for 5-7 years during residency.

> they're predictive power is terrible.

Honestly, we haven't had many issues with it, so I don't know that I could agree it's terrible.

Actually, and I'm not exaggerating here, every single one of our worst hires has come from giving somebody the benefit of the doubt when they didn't do well in the technical interviews.

---

At the end of the day, I agree with you wholeheartedly that we need to find a way to improve the interview process. I just don't think the current process is that terrible, especially when you get away from large, lumbering places like Google and look at smaller, more forward-thinking companies. Google has absolutely zero ability to adjust to specific candidates.

I understand your frustration with being experienced and still being subjected to technical interviews on BFS or whatever. It's annoying and sometimes even insulting, and in the worst case it demands that you go spend an inordinate amount of free time brushing up on interview material that never actually shows up in the real world. So I'm very much with you on the inadequacies of the current procedure; I just don't think it needs a drastic alteration.

I think replacing whiteboards with actual computers, algorithmic brainteasers with real-world problems, and extreme timeboxes with more open-ended formats are the main changes that are needed. Just let someone sit at a computer and write a program to do something fairly common. And don't stare them down while they do it. I really, honestly do not think this is that unfair or crazy.

It seems a lot simpler than and just as effective as doing a pair programming event where you have to make sure that both people have not seen the particular task before, which basically means the interview will have to be completely customized for each candidate. And that's not necessarily a good thing -- it means it's pretty hard now to compare two different candidates.

If it works for you, then it works for you.

The one thing I know for sure is that peer reviewed studies over the last 50 years show that work sample tests are the absolute most predictive tests you can perform.

If what you're doing is as close to a real work sample as you can reasonably get, then you're probably doing better than 95% of companies. And from your description of your process it sounds like you are.

>It seems a lot simpler than and just as effective as doing a pair programming event where you have to make sure that both people have not seen the particular task before, which basically means the interview will have to be completely customized for each candidate.

The pair programming isn't necessarily essential in my opinion. The essential part is eliminating the artificial adversarial and timeboxed nature of the traditional whiteboard interview.

>it means it's pretty hard now to compare two different candidates.

That's actually a pretty key insight. The practice that many companies have of allowing interviewing engineers to use any problems they want is completely opposed to this.

If you aren't judging whether they get a correct working solution (i.e. passes some unit tests after running it through the compiler/interpreter), and actively engaging them on their thought process while they attempt to find a solution, it can be a useful tool (esp. in early filtering with really trivial problems). Interpreting the results, and picking the coding problem, are not easy tasks.

That said, I've certainly found myself in the position described by the story, and all I can say is that I'm glad I've never actually needed to find a job in my career (new opportunities have always come from people who already know me and my track record). You may (though I wouldn't bet on it) get fewer bad "hire" decisions by doing a code interview like the one in the article, but you also get bad "no-hire" decisions. I'd love to see some empirical data on the subject from someone who knows how to design an experiment.

For instance all my designer friends show off their portfolios, then maybe they get a take home work sample. My writer friends submit samples of their work. And my engineer friends discuss past projects they've worked on.

But, an interview technique can be useful for a company or not. Asking candidates to write code on the spot isn't inherently useful or not useful. Assessing a candidates typos and compiler errors smugly is probably not very useful. But assessing their approach to understanding expectations, understanding problems, and solving problems is useful. If a candidate just assumes the expectations without asking in the interview, they are likely to do the same on the job.

> But I wanted to appear smart, so I didn’t express any confusion.

And I don't get it. When I ask questions, if anything, I get told that I seem smarter for it. Asking questions is a sign that you're paying attention and actively trying to understand things. I don't think I've ever had anyone question my intelligence for asking questions for clarification.

(I'm not trying to criticize the OP in particular for this. I've heard it from a lot of places, and wanted to point out the fallacy.)

This goes a long way when you're talking to clients or partners too. You often need to ask questions that will appear stupid to them, but that you don't know the answer to. They normally take it that you actually care about the job/problem, and often you'll gain some important insight that you otherwise wouldn't know.

If I did get a snide response to a clarifying question in an interview, it would be an almost certain signal that I should run away from this company. Their rudeness is actually a favor, potentially sparing me months or years of unpleasantness in their company.

In fact some interviews are intended to flush out behavior like this. Give a candidate an ambiguous problem statement, and see if they ask the necessary clarifying questions.

This meta-problem (ambiguous problems/unclear goals) happens all the time in the real world and bringing clarity to situations like this is a good skill to have.

I'm asked to code a simple matrix in java. I ask about the fill factor, and get told quite low. The constructor takes the dimension and it's assumed square. I ask the interviewer if a csc matrix is ok, and he kind of says he's not sure what a csc matrix is. I tell him it's column sparse, and proceed to quickly write out, on a whiteboard, a simple csc matrix -- basically, an array of dimension columns of ArrayList of tuple <index, value>. The constructor is pretty much

   ArrayList<Tuple>[] columns;
   void SparseCSCMatrix(int dim){
      columns = new ArrayList<Tuple>[ dim ];
   }       

We then go at it about whether I should have asked him to clarify, and I basically decide I don't have to play along with stupid gotcha questions so I told him words have meanings and a "matrix" with only 10 non-empty columns is not a matrix. And I don't have time for wankers.

At least I learned what they were like fast...

In my experience, "matrix" is a mathematical concept, and is independent of the implementation. As such, an extremely sparse matrix, such as your alleged interviewer described, is absolutely a matrix. And for such a matrix (density 10^-9), such a dictionary-style approach seems fairly reasonable.

Next, it seems to me that "CSC matrix" is a particular encoding that works well for sparse matrices, that is not related to the code that you wrote. It contains no index/value tuples, and it consists of three top-level arrays (that, in the general case, are of different lengths).

https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_spars...

http://netlib.org/linalg/html_templates/node92.html#SECTION0...

http://docs.scipy.org/doc/scipy-0.15.1/reference/generated/s...

Actually, it's not obvious to me how to do arithmetic on it (without converting to a denser format), though the scipy docs say it's fairly efficient, so I'm having a little think about that. Interesting.

Anyway, sounds like you're glad you didn't get that job, and I'd imagine that they're glad they didn't get you, so everybody wins.

   [1, 3] = 3
   [11, 3] = 17
   [9936, 3] = 3

What makes me shake my head is you don't understand what csc matrices are but didn't let that stop you from telling me I don't understand them. HN in a nutshell.

(Or even worse, there's also "I don't like your questions because my idea won't stand up to close scrutiny." I get that every once in a while too.)

- Pick a real world task (e.g. text munging).

- Suggest in advance that they bring a laptop with their preferred work environment. Allow any reasonable language if you're willing to train them on your stack.

- Write the problem down ahead of time. Have someone else at your company read it.

- Do the problem yourself. Ballpark your estimate for a great candidate at 3x the time it took you.

- Tell them there is no penalty for using stack overflow over the language docs.

- After you give them the problem, give them a few minutes in peace (leave the room) while they think it over. Talk about their approach out loud before they begin working.

- Ask them to spend 5-10 minutes re-factoring their code once it's working.

I'd further expand this one:

- Write the problem down ahead of time. Have someone else at your company read it.

You also need to have someone else (not you) at the company complete the problem. Ideally have a "model employee" do it. If they are unable to "pass" your interview question (under near-interview conditions) then maybe the question is a bad indicator.

Keep in mind that asking a co-worker to complete it is much lower stress/easier for them than a candidate, as they likely have near nothing on the line (maybe reputation). So if they find it hard, an interviewee will likely find it near impossible.

Even if you aren't intentionally trying to do the opposite, which unfortunately many interview techniques, like whiteboard coding effectively do, conducting interviews where a candidate is comfortable and at their best is ridiculously hard.

The only difficulty I can see with letting the candidate use any language they like is some languages are inherently easier and faster than other languages to solve certain problem classes. This may make it hard to compare candidates.

Regarding the interview, there's this trick I learned that presenters do when they come on the stage. Instead of focusing on the audience and their talk immediately, they first drink a glass of water slowly. It lets you set your own pace, and deal with the first high of adrenaline a little.

In this case, they asked you a question and you immediately fired up a repl and started typing away as if it's a race. But if no one is holding a stop watch it's not really a race. If you're going to have to do this kind of interview more often, perhaps practice on your generic problem solving strategy.

If it were me, I'd probably ask for a piece of paper, and then write down the components of problem on the highest level, then drill down the details of each component. At each level of detail check with the interviewer if the requirements of the component match their intentions. When you're confident you've got the algorithm down on paper, then you can start on working it out in the REPL.

Also, I'm not a LISPy guy so no idea how you usually do this, but those first three mistakes you made were horrible. Obviously you were nervous, but these kinds of mistakes don't happen if you have a good IDE and disciplined code style. You can't miss matching brackets if every opening bracket starts a new line and a new indentation level. You can always clean up the new lines when your algorithm is done.

Practice and preparation are the best countermeasure to stress.

Not that I agree with this interview style at all by the way. You are most likely an excellent developer, and if they don't decide on hiring you they probably are making a big mistake. If I was hiring Clojure people I'd be surprised I'd get someone that would work a REPL like that without internet reference at all.

Live programming interview tests are rubbish, because:

a) They produce a huge rate of false negatives. People who do not "interview well" flounder in a programming task with someone peering over the shoulder.

b) They can only test "algorithmic" tasks (for lack of a better definition). Most programming is not algorithmic, it's higher level: glueing already written libraries, with very light data manipulation or validation.

Live programming interviews should not be the sole quality of judging someone, but if we've first given them a coding exercise to do on their own time and they passed that, then it's useful to have them come in and do some coding to show that they can walk through their code and add additions/fix bugs to it without problem.

Which on that same note, I'm not sure why you would say they can only test "algorithmic" when we can do like I just described and have them for instance code a new feature in an existing code project of theirs.

That said, I agree with everyone else that they shouldn't be rushed, should allow the interviewee any and all resources they care to look up, and should shoot for "real-world" exercises over made-up scenarios.

It's rude beyond insult to do this during a job interview, whoever decided that was ok to do is basically sending you a very large warning message that even hiring is a job that does not require full attention.

The studies that get brought out every time this comes up are very clear. The best predictor of future success is a work sample test. Solving a bitesize problem in 20 minutes while someone watches over your shoulder is not a work sample test--it's about as far away from a work sample as you can get and still call it programming.

At the very least why not let the candidate work on the problem on their own for 30 minutes and then check back in.

I've said it before, the best interview process I've ever heard of is this:

Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Repeat this process if necessary.

Get everyone together and talk about past projects the candidate has worked on. Have the interviewers evaluate the candidate, and make your decision.

You know whether the candidate can code, and you've removed the adversarial nature and unnecessary stress from the interview process. Most importantly, you got to see them work in a much less artificial environment that's closer to what they'll be doing day to day.

I remember once being so nervous that I completely blanked. I'd been asked to reverse a linked list. A very simple task, one I'd even practiced a few times in various languages prior to my interview and should have been able to breeze through.

But I froze. My hands were visibly shaking as the interviewer stared at my back... I shut my eyes tight, trying to think through the problem (I knew this was a simple task, and I was frustrated that I somehow couldn't recall the solution). At first I tried to write out a recursive solution to the problem, but my anxiety-riddled brain just couldn't cope with recursion that morning. So I erased the board and started over trying to find an iterative solution. That went even worse.

"Um, I need to repeat this block of code, I think, maybe. There must be some way, some kind of construct that causes a block of code to be executed several times, almost as if one were looping through the code." AGH! Think!

But I was coming up empty, I searched my memory, ready to seize on the first seemingly helpful thing I could find. And so my mind wandered back to a time when I was about 8 years old, learning to program for the first time on my Apple II. I had just discovered the magic 'GOTO' command, which was the perfect tool to cause uncomplimentary messages about my older sister's hygiene to scroll down the green mono display for all eternity (or until someone hit Ctrl-C).

So... I completed my solution on the whiteboard using a GOTO. The interviewer put his face in his hands and began laughing. Eventually, he stopped and said "Um, ever heard of a for loop?"

"Well.... yes."

I would have probably walked out or at least snarkily asked them if these types of algorithms are what's done day to day. The answer more than likely being no.

Are you paying candidates for this time? Unless I really wanted the job, I doubt I'd be doing full-day tasks as part of the interview process.

When I was interviewing about 4 years ago for a company, their (third and) final round of technical tasks took around three hours and was invoiced afterwards.

Good for you but FYI, brain teasers have been banned in most companies in Silicon Valley for a solid decade. The fact that they are no indicator of the quality of an engineer candidate is a well known fact.

I think you mean "specific, but highly visible companies" instead of "most companies." Having recently gone through a round of interviews, I can tell you the brain teaser is alive and well at quite a few hiring companies in SF and SV. I think people know they don't work, but hiring is difficult and most people are lazy, especially at companies where the interviewers would rather be coding.

I would also like to reiterate the cargo-culting part of my criticism and expand my rant a bit to point some fingers.

One of the worst things that happened is the pernicious belief (mostly popularized by Jeff Atwood in his blog[1]) based on an non-peer-reviewed, and later retracted study[2] that implied that programming ability is a genetic trait, instead of just a skill and a craft like any other. The same blogger is also responsible for popularizing the idea that most candidates for engineering jobs are charlatans trying their best to fool you into hiring them.

To be clear, I'm not implying that this one blogger is responsible for all the ills of the industry recruiting practices. But I don't think I'm exaggerating when I say that the interview part of the process for engineering hires in the industry is far more adversarial than it needs to be and those with influential soapboxes who've brought us here have to accept some responsibility.

The state today is, that far from the adversarial process being a necessary evil, it's actually a point of pride for certain companies (some of which I've worked for in the past). One can only hope that these attitudes change over time like those about the brain-teasers.

[1] Here's Jeff's blog

http://blog.codinghorror.com/separating-programming-sheep-fr... published in 2006

[2] Here's the surprisingly detailed and personal retraction from the author of the original study... published last year.

http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-...

I've heard countless people in the industry repeat this belief of a programming gene to me over the last few years. They may all not remember where they originally heard this but those who do invariably point to Jeff's blog rather than the original study itself. I can only hope that Jeff uses his platform to beat the drum of this retraction as loudly as he did for the original mistaken study.

For that specific problem, I'd probably be satisfied if they wrote a function that took a number like 341 and returned [3, 4, 1], and then mumbled something about a set or a hash (or a list even) for storing the already seen numbers.

Fortunately, they didn't stop there and I was able to do harder tasks better later on ;)

I reacted pretty much in the same way on my driving exam. Both that interview and that exam weren't telling you anything about how I actually program or how I actually drive - even under extreme conditions. Being judged is just a completely different experience that makes me behave in completely different way.

Took me about 2 minutes to do. Whereas in the interview I wrote entirely the wrong thing initially, then went "oh wait, that makes no sense".

I was really nervous, and since it was a phone interview, I tried to speak my thought process, which didn't quite work out due to me constantly stammering. So I just took to typing all the time so that my interviewer wouldn't think I was taking too long coming up with a language or copying answers. I started making silly mistakes like using `int sum=0` in python. I took the input as a number, converted it to a string, and then further converted it back to a number, and did mathematical operations on it, trying to solve the problem. Every time I thought I did it, it would get stuck in another case.

Ultimately, the interviewer deleted 50 lines out of my 60-line code, and just added a print statement on a variable I was already using for storing things temporarily, which turned out to be the correct answer.

Needless to say, I blew the interview. I haven't done an interview since, and I'm gonna be up for it very soon. I'm not sure if I'd do better this time, but I'd be ready with a glass of water, and try to be as relaxed as possible.

This is an incorrect assumption, at least when I interview people. A live coding interview has (almost) nothing to do with the "quality" of your code, and doesn't necessarily even demand complete correctness. It's about observing your process, and trying to simulate what it would be like to work with you:

Do you approach a problem with caution until you're sure you understand it, or jump in head first and start writing code you hope is correct? Can you verbalize your thought process and communicate it to others? How good are you at reasoning about the runtime behavior and correctness of your solution? Do you consider edge cases and naturally want to test them? Things like that are won't show up in a take-home coding test, and tell me a huge amount about what kind of programmer you are, and whether I want to hire you.

See: http://www.yacoset.com/Home/signs-that-you-re-a-bad-programm... - there's a lot of truth to this article, and many of these flaws will be glaringly obvious in a live coding session.

And yes, of course some people interview poorly or are just having a bad day. That's unfortunate, but I'm not about to give up a powerful interview tool just because it can have false negatives. Until I find a divining rod that will point out the best programmer in a room, asking a candidate to write some code is the fastest way to learn what kind of programmer they actually are. (And also immediately weed out people who can't actually write real code.)

I frequently hear people say things like this. Now maybe you do think this way and I'm not saying you don't, but I think that makes you exceptional. In about 80% of interviews I've done where I successfully completed the problem, I got an offer; and in 100% of interviews where I did not complete the problem, I did not get an offer. That seems to indicate pretty strongly that solving the problem is by far the most important thing for most interviewers.

But coding at an interview under stress and time pressure has nothing to do with a work process in a normal company, unless you also force your employees to work under stress and strict timelines with you sitting behind their shoulder and watching every keystroke...

I'm a designer, not a programmer, but I've found that I interview best when I get explicit guidance from the interviewer. "I want to hear more about x, don't worry about y. What I want to learn about you is z." It makes the interview feel like a collaborative process (especially helpful when interviewing people you will work directly with, as you're pre-selling them on the team dynamic they'll experience!). It allows me to provide the information the interviewer needs most, in order to effectively evaluate my abilities and fit. It also clarifies for the applicant what things are actually worth stressing out about.

(EDIT: this was actually a joke...)

Bad situation, but I don't draw any conclusion from it. There are lots of inherently hard things in life that get messed up if you are nervous, insecure and can't communicate. That isn't necessarily an argument against them.

Writing Clojure functions under pressure is like gardening at 55 MPH. You're trying to make something still and beautiful and overdoing it (or missing entirely) as you drive by.

"Number theory" is a bit exaggerated. This is a warm up question to see if the candidate is able to decompose number 530 into digits five, three and zero. If you can't do this in a few minutes, you're probably out of the running from the get go.

No interviewer will ever ding you for writing code that doesn't quite compile yet, because it's just part of the process of writing code in an STL.

However, with a DTL, every time you run your program and it crashes, the interviewer mentally takes away points. Which is not really fair since it's how DTL's work, but it's a reality: a runtime crash is much more damning than a program that doesn't compile.

Stepping back, I think the simple fact that the company this person went to interview with recently switched from Ruby to Clojure would have already been a huge red flag to me.

E.g. when he calls (happy 97) it prints 88. However, that's 9^2 + 7, when in fact the answer is 9^2 + 7^2 = 130.

1) Agree with the interviewer before starting to code on the algorithm and problem. I.e. You could say I need function X that would be used like such, and function Y that would loop through such and such. Even better is drawing a high-level example on the board. That way, the interviewer knows that you understood the problem and can be systematic about it. (If you don't have a whiteboard, just write in comment at the top, whatever)

2) Once you start coding, start small.. especially in the REPL. I.e. instead of starting with the whole function, start with the string, then separate it in digits, then sum it up, then work from there. That way, if something doesn't compile, it's perfectly clear what step it was and once you have your function, you're 100% sure it works. TDD style.

    next = (n) -> "#{n}".split('').reduce ((p,c) -> p + c*c), 0
        
    happy = (n, set = []) ->
        return false if n in set
        return true  if n is 1
        happy next(n), set.concat(n)

    console.log(n, happy n) for n in [1..100]

EDIT: removed some bitterness. :)

Starting with a programming problem that takes hours is simply being lazy from the company's part. It's easy to hand out and review but a lot of work for the candidate.

personally I really need human feedback rather then 8 hours at my desk guessing and searching for what's being looked for

it's one of those situations where I see the value for the interviewer but as the applicant it's completely sucks especially when you have 4 sample projects to write at home while juggling phone interviews and you know having a life/job

Matasano (founded by tptacek) does a good job at this: http://matasano.com/careers/

Give the candidate a research paper in PDF, then ask them to implement the algorithm described there in code.

You get the same feedback on whether they can actually code, and what kind of code they write, but it takes the pressure off having to think of what kind of algorithm to use, what the big O complexity of it is, and a thousand other factors that will probably never relate to what their day job will consist of.

Consider that the interviewer has now seen that this person will choose to, without clear understanding of either the goals of the task or the task itself, work alone as though he must solve the problem in isolation in order to prove himself.

The most important thing that a candidate needs to be able to do on the job is discuss expectations; failing to do this in the interview is more egregious than any compiler error. Another important thing is to understand the problems they are working on; failing to even discuss the problem is more egregious than any compiler error. Another is collaboration and humility; you can demonstrate these skills in a coding interview as well.

This interview demonstrated many more important indicators of a poor fit than typing ones. Just because you pose a coding problem doesn't mean you're unable to assess candidates on the things that matter.

P.s. You are not a bad programmer... Pressure can mess your mind. My secret to finding great developers that everyone else misses is by making them feel calm (giving them an easy problem and giving them encouragement so interview stress becomes less a factor, best way to find good developers who just are bad at interviews)

One was a very simple task, setting up a server as a customer would at an ISP using WebMin type tool. Really simple stuff. Guess what. I couldn't figure it out in time. Maybe because I had been consciously staying away with gui stuff to stay sharp with cli. Or maybe the proprietary tool the ISP was using was very unintuitive. Maybe the suit and tie I put on for the first time in 2 years got to me. Either way, had I NOT been in an interview setting, I would've figured it out. But with this interview, I couldn't figure it out.

In the 2nd company, I was giving an online test using https://trueability.com/, to complete from home. Set amount of time to complete some test scenarios. Far more difficult stuff than setting up a small web server at an ISP. And I would say it was a far more realistic test of my skills.

IMHO, interview should really only focus on compatibility, not actual tech skills. If your company is really spending time on tech skill test DURING interview (especially onsite), you are doing it wrong. just my 2 cents.

Also, the way I understand it, you can keep a personal profile on TrueAbility and point recruiters to it for them to review your tech skill level.

     (defn int-to-digits [n] 
       (map #(Character/getNumericValue %) (str n)))

    (defn next-happy [n]
      (reduce + 
        (map #(* % %) (int-to-digits n))))

    (defn happy? [n] 
      (loop [n n seen #{}] 
        (if (contains? seen n)
          (== n 1)
          (recur (next-happy n) (conj seen n)))))

I couldn't agree more. Especially when you're under pressure, bite off smaller pieces. Not only is it possibly better practice in general, but in a time of diminished cognitive capacity (such as when much of your bandwidth is being eaten up wondering how you're coming off) they will be easier to get right, and small successes will help ease your stress as much as they help solve the problem.

    happy = (n) ->
      sum = 0
      while (sum += Math.pow n%10, 2 ; n = 0|n/10) then
      sum
    
    happyness = (n, s=[]) ->
      (n is 1) or (n not in s) and (happyness (happy n), [ s..., n ])
    
    console.log n for n in [1..100] when happyness n

    function happy(n) {
      for (var sum = 0; n; sum += Math.pow(n%10,2), n = 0|n/10);
      return sum
    }

    function happyness(n, s){
      s || (s=[])
      return n==1 || !~s.indexOf(n) && happyness(happy(n), s.concat(n))
    }

    function h(n,s){for(s=0;n;s+=Math.pow(n%10,2),n=0|n/10);return s}
    function H(n,S){return n==1||!~(S=S||[]).indexOf(n)&&H(h(n),S.concat(n))}

1) Decompose a number into digits. Less experienced candidates will likely convert the number into a string and then the characters back into numbers while more experienced candidates will tend towards a small loop using div and mod.

2) Use some kind of library to insert and test membership in a set data-structure. Less experienced candidates may not be able to do this without stackoverflow, big red flag.

3) Some basic flow control wrapping it all together. Less experienced candidates may fail here similar to fizzbuzz.

The interviewer likely thinks these things are a good proxy for the quality of a candidate. For example, the author would have failed #1 even without miscommunication/stress/time constraints.

So it seems here is a an instance of where life was not fair... in my favor. But now I'm in a position to hire other people. So I don't want to judge them in the same way I was. I want to find out who's got the chops completely apart from who can dance on cue.

(defn digits [num] (let [ x (map str (str num)) s (map read-string x)] (apply + (map #(* % %) s))))

I do think the interviewer should have pointed you in the right direction or offer some sort of hint to gain an understanding of how comfortable you are with the language rather then just having you fumble around but depending on the role they were hiring for these questions and this practice don't seem bad to me

Typing in an editor (probably with Send-to-repl functionality) would have caught a lot of errors. But yes - doing it with a proper IDE or even a statically typed language would really had helped to focus in the problem and not on getting-it-to-run.

You could also precalculate or cache the answer for 1 through 100 to save rerun time.

But maybe a little bit more math would reveal much better optimizations.

I can write white board code with ease as those are almost always two way conversations

Skype n_goles

    def happy(n: Int) = n.toString.map(_.asDigit).map(Math.pow(_, 2)).sum.toInt

    def find_happiness(n: Int, seen: List[Int]): Boolean = {
        if (happy(n) == 1) true
        else if (seen.contains(happy(n))) false
        else find_happiness(happy(n), seen :+ happy(n))
    }

Of course, some of the issues the author faced can't be solved by Clojure's compilation step. When his code fails at runtime in the repl, it compiles successfully.

While interviewing, I tell the candidate the problem and then leave the room. He is alone and comfortable (I hope). I check back after 30 min. If not done I still give him more time.

The cost of screwing up there is small and yet its excellent exercise in coding in front of an audience.

As far as this interview question is "Do you %?", I like it as a test. For the rest, I feel sorry for the author. Good luck next time :)

in the interview i fluked a question about js scope because the whiteboard handwrite of the interviewer made it hard to see which variable was i and which was j... but after he explained why i got wrong and i realized the letter mistake i was too stressed and dumbfounded to make a case, so i just left.

6 mo later, my current employer is still independent while that company is now owned by Microsoft...

Although some function decomposition of the problem would have been nice to see!

The idea is that we switch back and forth. One of us writes a failing test, then the candidate writes the production code. The candidate then writes the next failing test and one of us writes the production code and the next failing test. If the candidate gets stuck, we usually try to talk through the situation and if they are really stuck, one of us will offer to take over.

The candidate is allowed to look things up on the internet and even ask questions. Often we will suggest things. For instance, after the candidate writes some production code, we might say, "That works well. But how about refactoring it using inject," for example.

I often use a pomodoro timer set to 20 minutes. We'll do as much as we can in the first set. After that first set we'll ask the candidate how they think it is going. We'll offer some advice about technique and such. If it was an absolute disaster (happened a couple of times), we'll just leave it there and thank the person for their time. Otherwise we'll ask if they are up for another round. Usually the first set is pretty bad because the person may never have done real TDD before, or may find the ping-pong awkward at first, or may be very nervous. Talking about how they are feeling at the break usually helps and the second set is much, much better.

After the second set we'll ask the candidate how they think they did. I think this is important because if the candidate says, "It went really well" and it was actually awful, then we have a pretty good idea of their skill level. If they say, "I wasn't really happy with my performance, but I enjoyed the style of programming", then we will have to take into consideration a variety of different factors to see if we believe that the person can do better. If the person says, "I really didn't like working that way", then we can rule them out as a candidate regardless of their skill level (also happened a few times unfortunately).

The feedback we've gotten from candidates on the process seems to indicate that they really enjoy this interview style. I think it removes a lot of the spotlight from the person and allows them to relax a bit. Especially having us come in and help out in sticky situations allows the candidate to get over stupid blind spots (like the converting between string and integer in the article). So far, I think it has helped us spot potential in a few people that I'm sure we would have overlooked using a more standard, "Here, solve this problem" approach.

On the downside, there is still considerable room for improvement. Language lawyers (know every detail about the programming language but are still poor programmers) can often look very, very good in this kind of test. Similarly, inexperienced but very talented people can often be frustratingly mediocre because the unfamiliar method of development can hamper their style. You need to be very aware of the amount that the person improves during the interview.

One final comment about this approach. We have occasionally been in a situation where we said, "I'm not sure how good this person is. They seem like their skills are fine, but whether they will be really excellent, I don't know. However, it was really fun programming with them." In those cases we have hired the person and never regretted it.

As I understand it, you are taking the sequence

n -> (sum of the squares of the digits of n)

First, the sequence never diverges to infinity. For example, 9999999999 -> 9*81.

Second, not all numbers converge to 1. There is a stable cycle that does not include one.

Consider

4 -> 16 -> 37 -> 56 -> 61 -> 37 -> ...

So, any positive integer either

1. Converges to 1

2. Converges to the stable cycle I just mentioned above.

So the interviewer should be fired for not understanding the question he was asking.

'The next sentence of English is where the problem started. I _thought_ he said:

“The sequence will either go to one or to infinity.”

but he actually said:

“The sequence will either go to one or infinite_ly_.”' [My emphasis]

(Firing's also a bit harsh for getting an interview question wrong)

That's also a horrible question for illustrating functional programming style. It would be much more efficient to have a simple loop caching the array n->f(n) and then finding the ones that converge to 1.