Since i'm a contractor i get interviewed a lot, i sometimes come up against these brainteasers which i usually dont mind. Sometimes they're good questions like how would i solve problem X using X language, because thats relevant, i think its a good question.
However, i hate when i get questions like "how many golf balls can you get in a double decker bus" and "whats your most embarrassing moment", both of these were questions i was actually asked in an interview. My answer to the first was "I dont know and i'm not going to try and work it out because 1) i dont have enough information and 2) its not relevant to this position. I'd happily answer hypothetical questions about situations that could plausibly come up during this position, but working out number of golf balls in a bus doesnt show you how i would solve the problem of the database running slow or validation checks not working correctly." For the second one i simply answered "I have never been embarrassed in a professional setting as i mainly just do my job and i'm not embarrassed to say 'I dont know' or to ask for help. The only times i've been embarrassed were in my personal life and those are stories not appropriate for an interview."
Needless to say, these were questions asked by the CEO of a startup i thought was absolutely amazing, i had sat through a tech interview with the CTO before the CEO came in and felt i did well with the tech questions. Once the CEO came in and asked me these questions, i wondered how an awesome startup was achieving this success behind the leadership of total and utter moron. I decided not to work there as i lost all respect for the CEO.
I used to take this a step further. The split second one of the silly questions popped up, I'd stand up and very politely end the interview. Reactions were priceless. Interestingly, I'd often get invited back, or on a couple of occasion offered the job after a follow up phone call, where I give the sort of responses you have given.
The problem I have is that these types of question suggest to me that the company doesn't really know what it wants to hire, and there for is not one I would want to work for.
The problem most people have is that they really need the job and feel compelled to play along, which I can only assume these interviewers take as positive feed back.
There could be a chance that they are simply testing you to see how you respond to silly questions and how much of a stuck up asshole someone can be (not that you are one).
I'd imagine it's a good way to spot well adjusted human beings.
This, of course, assumes (positive) things about the company and the interview process, which may not be true for 95% of the sample as far as I know.
> "how many golf balls can you get in a double decker bus"...
> 1) i don't have enough information and 2) its not relevant to this position.
Whilst I agree somewhat with the sentiment, I don't think that's at all an unreasonable question.
1. You do very clearly have enough information to make a decent informed guess.
2. It is a relevant question.
It's relevant because it assesses your general sense of numeracy. Whilst that won't show how quickly you'd be able to deal with a database issue, it does give an indication of how you might go about making broad judgement calls in the absence of any hard-and-fast numbers. For example: Is it worth looking into this database issue, or is it likely to be insignificant compared to the network overhead?
But if that's the information you're after, why ask for golf balls and busses? These are real world quantities with decidedly different qualities than whatever appears in standard computing scenarios. Describing how you'd go about troubleshooting a three layer architecture with inordinate network load on the database might be a better question then?
I actually think that's a decent question - in many cases you have to estimate something based on incomplete information in a domain that you aren't an expert in and maybe all you are after is an answer to the nearest order of magnitude.
In my experience some people simply won't answer those questions - not because they can't but because they aren't happy working that way. Of course, in some situations you absolutely do need people who will always work things through from first principles and attempt to give as precise an estimate as they can, but sometimes you don't.
really what sort of bus? Routemaster ,Bendy Bus, one of those new Boris busses, a US school Bus. Also how does one easily estimate the free space inside a bus. How many pasendgers are there with or with out a driver or conducter can I remove the seats and other furnature inside.
Of course one can trivialy estimate the number if one assumes that bus is a cuboid and that a golfball is an incompressble sphere - I would would probly have to look up the theorems on Google as its a long time since I used this sort of thing in anger - though bak in the day I did correct a bridge design for the 4th largest consulting engineers when the Engineer had use 2d rather than 3d :-)
It could also be a culture question. I worked on a trading floor and we'd make "markets" on when lunch would arrive, the cost of an airline ticket, the number of times I would injure myself in a week, etc. While there are more precise ways to ascertain a potential hire's arithmetic and estimation skills, this is more fun (especially since the interviewee's answer would usually kick off a round of bets being placed by the team on what they thought the answer was - this had the added benefit of giving us a relevant distribution to test the candidate's answer against).
It also drew out people who may not be a fit for our desk's culture of camaraderie where answers to banter along the lines of "that's personal" or "that's not relevant to my job description" would be awkward at best.
Additionally, I have heard from consulting buddies (the heathens!) that it's can good way to see how well a candidate can bullshit on little to no relevant data.
In any case, I don't think it's empirically valid to conclude someone a moron for asking brain teasers in interviews. I never asked them whereas many people much smarter than me did.
The purpose of this type of interview question is, per my argument, to get an idea of cultural fit and estimation skills. Like any measure it picks up some signal and a lot of noise, e.g. bullshitting skills. The trick is to combine measures such that the noise largely cancels out while preserving the signal.
If this were the only type of interview question used the points you brought up would be a legitimate concern. Given that it was, in my case, married in a much greater proportion with concrete questions I maintain that it generated unique, useful information.
Traders generally have, prior to entering a trade, the opportunity to think through everything thoroughly and ask for help on the bumps. There are also times when the cost of the time for analysis is so great that a gut, if rough, call is needed, e.g. holding hard drive manufacturer equity as news of Bangkok flooding breaks. Being able to ball-park figures, after issuing a disclaimer about the uncertainty of one's estimates, is also a generally useful analytic skill.
It should also be noted that part of recognising the priority of things to be analysed is the degree by which they deviate from expectations - generating many of those expectations is a form of intuitive estimation.
But, as I said, it's mostly to estimate whether this is a person we'd like to spend 10-16 hours a day near. Constant righteous indignation would probably get annoying as fast as serial bullshitting.
I think "how many golf balls can you get in a double decker bus" is a fair question for assessing whether someone can apply basic maths to a to a real life (i.e. not perfectly defined) problem. If that's a skill the job needs it's a perfectly reasonable question.
I also think it's a fair question, but more about logical reasoning and estimating than basic maths. As an engineer, you could be asked to estimate the number of load balancers needed for X traffic with Y latency - similar problem.
In my experience of those kinds of problems the catch is finding realistic values for X and Y - often nobody is going to hand them to you on a plate and you must work out a way of estimating them - that is the real skill.
But Fermi problems are quite relevant in computing work. Here is a question I was asked recently:
"How many users do you think will ever concurrently use a certain subset of our website?"
Actually, the question was phrased as "how would you architect Feature X?" This naturally led to "do we need a distributed approach?" And of course, the answer to that depends on the answers to the Fermi question and "how many users can a single box handle".
I've never been asked a to solve a Fermi problem in an interview, but I would most likely require that the interviewer give me the data I need. In the golf ball example, I would ask for diameter of the golf ball, dimensions of a bus, etc. Otherwise they're just asking me to prove that I have seen a golf ball up close, that I've been riding on double-decker buses and that I'm decent at measuring physical objects with my eyes.
Or you could guesstimate a diameter of a golf ball and then tell them how much the answer could change as a function of your error, rather than demanding perfect information in an imperfect world.
The goal is not to get a precise answer, the goal is to see how you think.
Sorry, I didn't mean to imply I wanted precise information. Their guesstimate is going to be better than mine, which is why I would ask them for that info. I'm just not comfortable with pulling data out of my lower dorsal region. And I guess that's the part of how I think.
Another reason I like Fermi problems, is that the kind of people who I want work with (read: curious-minded tend-to-be-geeky programmers) tend to love them, and solve them for fun. I can't count the number of times we sat over lunch, arguing weird questions like how many planes it would take to move a certain river, or how many people it would take to eat a rhinocerous.
If someone is put off by these kinds of questions, I would question whether they fit with the kind of culture I think a group of smart people have.
> "how many golf balls can you get in a double decker bus"
> My answer to the first was "I dont know and i'm not going to try and work it out because 1) i dont have enough information and 2) its not relevant to this position.
The genesis of these kind of questions is http://en.wikipedia.org/wiki/Fermi_problem (checkout the classic "how many piano tuners"). Fermi approximation's purpose it to estimate without concrete data. In the end, if the numbers aren't right, you just need to replace the numbers, provided your method was correct.
I am not contending its relevance(largely irrelevant). I am just pointing out the origin.
The CEO doesn't know the answer to how many golf balls so he wasn't looking for the end result, he was looking for where you start when problem solving. Many people have no idea how to start solving a problem they haven't seen solved before. The question is valid and you sir are grumpy. But I do give you kudos of standing up for your opinions, that is a trait I'd like in someone I'm hiring.
> working out number of golf balls in a bus doesnt show you how i would solve the problem of the database running slow
What about estimating how many I/O operations in a second the DB supports? You'd be surprised how many candidates out there lack basic arithmetic abilities.
I have never heard of anyone being asked "what's your most embarrassing moment?" What a bizarre question. Clearly, my most embarrassing moment, whatever it may be, is far too embarrassing for me to go around telling random strangers about.
The only utility I can see in that question is to assume that it's a question you're expected to dodge, and that the deftness with which you deflect the crazy question is a measure of your interpersonal skills. To launch into a story about this one time at band camp is presumably the worst possible answer.
Still, I don't think it's good even for that question.
The correct answer is a funny anecdote that is only mildly embarrassing at best. Bonus points if you can make yourself look resourceful and/or humble at the same time.
A few years ago I was interviewed by Microsoft in Salt Lake City, now closed. The job was a creative type job, a game designer. A fellow there said he was a tough interviewer. He ONLY asked me brain teaser type questions. His first question concerned a radio design 20 years from now. I drew what I thought it might look like and he seemed unimpressed with my answer.
He then asked me this: "You and your family of 9 are on one side of the river, there's a flash flood coming. There is a boat but it only holds 3 people. What do you do?"
I said: "I get in the bow, row to the other side, and wave 'good luck' to my family. See, I don't get along with my family very well."
He got a little agitated, and told me that in this case I love my family and want to save them. He asked me to try again.
I said: "I flip the boat over to make it more buoyant, put the kids on top and the adults hang on the sides."
He became angry: "There are piranhas in the river so you can't tough the water. Try again."
I said: "I lay the oars of the boat so that more people can ride in the boat..." He interrupted me by saying: "Just answer the question."
I said: "I have given you 3 creative and interesting answers. Since you are NOT trained in psychology you don't have any way of understanding my answers you feel frustrated."
He said: "This interview is over"
I said: "OK, but you know nothing about me".
I have refused to answer these types of questions since then.
This is one of the biggest problems with these trick questions - there really isn't a correct answer, just the answer that the interviewer wants to hear.
True! I gave him 3 creative answers and the job required me to be creative. I felt I had shown that with my answers.
In hindsight, the office was shutdown and I would have been there only a few months. The scary thought is that he's out there somewhere continuing with this interview technique.
I've had a few of these in my time, and I think, to an extent, they're legit, and I'll explain why.
I was asked once - "how many gas pumps are in Raleigh?". I asked back "do you mean pumps for the general public, or do we need to include commercial/industrial pumps too?" and "do you actually mean Raleigh proper or the metro Raleigh area? if so, how do we define that? All of the county?"
Next question was "go to the whiteboard and design a house". I took out the marker, started to draw, then turned and asked "who will be living there? Is this a single family or duplex? mobile home? stick built or modular?" and a couple other questions.
I was told I was the first people to ask questions before drawing.
I take the point of the brainteaser-type questions to be "how do you react to issues that come up where you don't know a lot of stuff?". For some positions, this is less important, but if you're a customer-facing position - even in development, you may interact with customers or other business units - how do you react when you get odd-ball "left field"-type questions? Do you keep your cool? Do you just react and say "that's stupid!"? Do you walk out in protest? Do you probe for more detailed information before making a pronouncement?
More than ever, I think 'brainteaser' questions are far more about personality-judging than intellect or even raw "problem solving" ability.
Yeah, I agree with this. I've been asked a similar problem to the gas pump one and a brain teaser one as well. The goal for them is to be able to get a better feel for how you deal with the problems, not necessarily the solutions that you come up and how accurate they are.
Being a dick to an interviewer (as basically suggested in comments above) is great if you like being unemployed.
Additionally, though (and I think you're saying this), it's not just how you come up with a solution from a technical/procedural standpoint. It's more - how do you react in situations that are outside of your comfortzone? And as you said, being rude to an interviewer is a sure way to stay outside their company.
Ahhh, I failed (flailed) one of these once! I spent minutes trying to figure out "You have a business card in Japanese, how do you figure out what it says?"
I asked if it had the phone number, they said yes, and I said the numbers are the same as in English... They said "okay, what next?" ... I was stuck. Next? Well, I suppose I could take a photo of it and ask for someone to translate it online... Or find someone in real life to translate it for me...
After a couple of minutes, they were ready to move on, disappointed, when I said "Well, short of asking for help, or Googling the phone number, I have no idea off hand how to find it."
"Oh, well, searching the number, that's what we thought you were going for initially."
It was. I'd thought they meant "next" as in "other than that method". Having the number was so evidently "problem solved" for me, that I didn't explicitly say "search out online". ... I suddenly felt like I was in grade school again, and my teacher had failed me for not showing the work in math class. I had the right answer, but would never get credit. I went on to flop the next question or two, as I vaguely recall.
The interviewer cannot guess what's in your brain. Part of being a good communicator is being able to actually spell out your thoughts when necessary.
Besides, if you had said "then I search the number online" then it's plausible the interviewer would ask what would you do if you didn't find anything relevant right away. For instance, if you search for my cell phone number, you get 0 results connected with me.
> The interviewer cannot guess what's in your brain. Part of being a good communicator is being able to actually spell out your thoughts when necessary
The flip side is when the interviewer doesn't communicate well. Or asks a specific question, but means something slightly different.
I've dealt with that before. Assumed they meant what they said, when the reality was, they were being more precise than intended. My fault for assuming I guess. I now try to clarify the intent, and make clear the precision.
I'm guessing many people's first response to this situation in real life would be to call out to coworkers - hey can anyone read this?, yet most of us would stick to the solo problem solving mechanics in our answer - search the # online, ocr + babelfish, call the number cold, whatever.
I understand fermi tests are intended to show broad problem solving abilities, but imho, these tests are for lazy people. Talk to a candidate for 20 minutes. If you can't tell if he/she is suited for the role, or at least for a follow up interview, one of you is not qualified.
Slightly pedantic, but this question was not a Fermi problem - Fermi problems are numerical approximation problems, and this one is more behavioral.
That said, I don't think they're lazy questions necessarily. However, I'm a physicist, enjoy Fermi problems (actually compulsively do the calculations when I hear a new one - a million golf balls!), and do think that order of magnitude approximations and quick unit changes are important skills to have developed.
I do think it's very lazy when they come out of a book or are riddles, because there are so many short approximations we can do at any moment. How much will the electric bill be for this office when it's August? or I just made a pot of coffee and forgot to place it back on the burner - when will it be too cold to enjoy? No accurate answer is needed or cared about, but the ability to predict important factors and come up with a list of contributing components is important.
Once an interviewer asked me, how would you design a telephone for blind people and is there something already on our phones today for it. My answer was short, i said "Braille". He turned the landline phone towards me and asked to take a look. The middle keys sometimes have this bevel shape(just like a keyboard). Since i'd already said braille, i thought he meant something else now. I said i don't know.
He's like 'notice the small bevel at the bottom of the center key". Apparently he didn't know what it was called and that it was named after the inventor. Well, technically it's not proper Braille, but i would have taken that as a correct answer.
Is it Braille? Isn't it just designating the center key in a number pad (the '5'), from which it is evident to the user what the key around are? There is a slight bump on my '5' key, but one bump is certainly not Braille for '5'. Braille is like Morse code, it refers to a specific 'code', not just "any bumps meant to be felt for".
>Well, technically it's not proper Braille, but i would have taken that as a correct answer.
I just wanted to highlight that even though the candidate/me was aware of the basic concept. The answer just wasn't the exact thing the interviewer was looking for, hence it was wrong according to him.
I can't tell what your interviewer knew or intended, but the point of the question it seems to me is the "one non-Braille bump on the '5'" solution is actually better and simpler than using actual Braille. So answering Braille actually does seem wrong to me, and insisting it's not wrong seems like missing the point. You had generalized Braille to mean any bumps, so maybe your interviewer knew what Braille was, and that this was not Braille, and thought the point of the question was specifically that it was not Braille.
Edited for clarity, sorry. I didn't explicitly say "okay, then I call the number," or "search it online". As far as they were concerned I stopped at "okay. I found a phone number... Then I take a picture of it."
It boils down to me not being perfectly clear, I guess. No hard feelings or anything, of course. Like I said, I threw myself off my game anyway.
From the article:
"I'm going to ask you to solve one of my problems for every problem you ask me. Remember, an interview is a two-way process, I'm trying to determine if I want to work with you too."
This is the best part really.. Instead of "just say no" remember it's a dialog. If the brainteaser is interesting or you can quickly respond to how you would approach solving it then don't be an jerk about it. And feel free to have a dialog ask something back and see how they approach problem solving etc. Make it fun and interesting for all involved. If you're going to work with these people you might as well start right now in the interview!
I never take an interview with a potential client or employer without having a very clear set of my own questions for them and getting those answered.
I often ask off the wall questions or talk about a deep problem we solved recently and ask for the candidate for ideas on how they would approach the problem. I always share the answer we came up with. The reason for the question is not a perfect answer but to see how a candidate approaches potentially solving the question.
Yeah, and this is how the boss's nephew gets hired into a management position he's unqualified for, poisons the culture, destroys morale, and causes a mass-exodus of quality people.
I wouldn't even hire my best friend without him interviewing with 4 people (individually) who would be working with or close to him. I also wouldn't take part in the interview myself, to ensure that the treatment was 100% fair and merit based.
A personal recommendation gets you in the door to the interview faster, but you still have to prove yourself, and if you're really that good you'd want us to prove ourselves as well before signing on.
That's probably also how Yukihiro Matsumoto, the creator of Ruby, got hired by Heroku. Sometimes an interview really isn't necessary and can only do harm.
I realize (presume?) that what you are saying is meant to be a bit of hyperbole, but most of us are not (yet!) so well-connected that a decision-maker would offer a job without this kind of screening process.
Here is something that I always wanted to do, but haven't had the nerve. At the end of the interview, when you are asked "Do you have any questions?", I'd like to say: "Yes. There are 5 sailors marooned on a desert island, and they collect all the coconuts (.....). So what is the number of coconuts?"
I'm not fond of brain teasers either, but small, self-contained questions have value. Often if they're simple enough you can use them in many interviews and establish a base line (if the candidate did a, b or c she will usually work out, if she took route d, that betrays a weakness in X etc).
This is not a replacement for evaluation of a candidate on their own merits, and having a good in-depth technical conversation. But I like having something to ground my evaluations in as well and make them even slightly comparable.
For reference, I do the same thing when I get interviewed; I have a few questions I ask any prospective company so I have some comparative way to evaluate them. Never bad to try to have some metrics in your process.
The most important point in this post is this: "an interview is a two-way process".
I am constantly surprised how many people come out of interviews without asking detailed questions about the company, job, culture and work atmosphere.
You have to be more sure that that they will be a good fit for you than you have to be for them. You will be one of many: they will be everything to you, probably for years.
This is so completely true. The company is being assessed as much as the candidate. Probably anyone who has interviewed candidates has lost one that they really wanted to hire simply because the company 'failed the interview'. I know I have.
Worthwhile candidates tend to have criteria and the self esteem required to ask questions right back at you.
I'm going to take the contrarian point of view here: partially for the sport of it, and partially because I genuinely think there's value in puzzles and brainteasers -- though I do think the value is limited, circumstantial, and merely one datum out of many in an interview.
These questions are about having a logical process, regardless of where it gets you, and spelling that process out in a sequential fashion. It's more important to communicate "I do X, and check for Y. Then I do Y, and check for Z" than it is to arrive at a reasonably accurate Z.
The added component of pressure (i.e., you've only got a few minutes, in the uncomfortable setting of an interview, in which to answer the question) helps (or allegedly helps) detect your ability to remain calm and logical under tight deadlines.
Basically, the question is asking "Under pressure, and faced with a seemingly insurmountable problem, will this candidate give up, or will he try to address obstacles in a rational, collected, and systemized fashion?" They're essentially variants on the Kobayashi-Maru test of Star Trek fame.
Furthermore, I don't agree with the apparent consensus that oddball "puzzle" questions would be better replaced with more technically relevant brainteasers. That's not the point. You have technical questions to test your technical literacy. These questions are trying to test your character. (Brainteasers are often non-technical precisely because the interviewer doesn't want you to be able to fall back on existing knowledge as an escape hatch from the question). Some brainteaser/puzzle questions are much more inane than others, but the exercise itself isn't entirely worthless.
I have never asked a puzzle question in an interview. If I were compelled to do so, I would never hire someone purely on the basis of his or her performance on one. It's one of the least important variables in a hiring process. But I'd consider with some skepticism anyone who outright refuses to answer a brainteaser, or who gets totally flummoxed by one, or who gives up without at least attempting to work out an approach.
Ultimately, no single type of interview question is flawless. That's why you have a wide variety in your arsenal. You've got technical questions, case questions, puzzle questions, whiteboard questions, and even the oft-derided "Tell me about a time when..." questions. In isolation, none of these types is sufficient. In combination, they test different aspects of a candidate's thought process and preparation.
Brainteasers and problems are fun and I love doing them. I just think it's not appropriate for an interview, especially when you catch an interviewer presenting you with questions that are simply taken from known 'problem banks'. I could have easily pretended to work through them like I've never seen them before but I think it's a lot more truthful to call them out and say 'I know all your questions'.
"especially when you catch an interviewer presenting you with questions that are simply taken from known 'problem banks'"
Sure, but then, almost all interview questions can be similarly gamed. If an interviewer asks me to "walk him through my resume," or to "Tell him about a time when I did X," does he really expect that I haven't thoroughly prepared and rehearsed those answers? And if I truly haven't, then that's saying quite a bit about how seriously I take the process.
I'm not suggesting that brainteasers are as valid as other question types. There are plenty of question types that better get to the heart of the candidate's aptitude, experience, skills and capabilities, domain knowledge, and so forth. But everything can be rehearsed for. I don't think that's a valid categorical critique of the puzzle type.
Better arguments against puzzle questions, IMO, are: 1) They might select for good bullshitters, as opposed to good thinkers; 2) The opportunity cost of spending time on a puzzle question is the time that could be spent on a more relevant question; 3) Some people just aren't good at (or interested in) puzzles, but that doesn't mean they're not awesome on the job -- ergo, passing on a candidate because he flunked a puzzle is much sillier than passing on a candidate because he flunked a technical exercise. (Conversely, loving a candidate because he kicked a puzzle's ass is even sillier).
Good points. That is probably why people are often not successful on their first interviews. It takes a few to get into "interview mode" where you have good answers for common questions about your CV.
I was quite dismayed when I called them out and they didn't come back with new questions or made up some on the spot. They simply decided to drop that part of the interview and move to the next section.
I've seen too many good people (friends and students of mine) miss out on jobs simply because they failed these type of problems.
"I've seen too many good people (friends and students of mine) miss out on jobs simply because they failed these type of problems."
With this I would absolutely concur. I have more than a few friends who are smart, and who are rock-solid performers, but who just don't have a knack for brainteasers. I would hope than an interviewer is able to suss this out in an interview, i.e., make the determination that a person is great in every aspect, would probably be a great fit, but just happens to suck at puzzle questions. Sadly, I'm sure it's more often the rule than the exception that an interviewer will reflexively pass on someone who flubs a puzzle.
And I agree with you that puzzle questions require flexibility not only on the part of the interviewee, but also on the part of the interviewer. If you're just tossing out questions based on a book of canned puzzles, don't be surprised when you get canned responses. The exercise becomes entirely meaningless when that happens, and there's really no use in proceeding with it if it does.
The main problem I have, is when I've seen the answer to a question but don't remember it 100%. Your brain generally goes into recall mode, and rather than actually solving the problem I sometimes give an inaccurate answer and it can become hard to kick your brain back into thinking the answer through. Sometimes telling the interviewer you've heard the question before is enough to make him move on, other times they want to hear your answer anyway.
yes, one of my friends, who works aggressively on brain teasers, got accepted to a job offer due to ability to solve such questions. however he told me that he knows thousands of variations of such problems. When asked he just pretends as if he is thinking to solve them, but he said most interviewers indeed are at a beginner level for such questions.
The review article "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings"
sums up, current to 1998, much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but only about at the 0.5 level, standing alone). One is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general cognitive ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both operate at about 0.5 level in validation studies), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court
held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrating that the test is related to successful performance on the job. Companies outside the United States are regulated by different laws.
I'm not too familiar with the US justice system, but I read the Wikipedia page the last time you brought up that specific case. Reading (a little) between the lines, Duke Power seems to have set out to discriminate against black employees. That seems very different from using an IQ test to select the best employees, especially for a job that actually is highly IQ-dependent.
It takes one bad apple to ruin it for the bunch, as they say.
That said, I lived and worked for a few years in a country where IQ tests were the norm for hiring people. When hiring for my own team (C++ devs), I did not use them as I found they were a very poor filter with both high false positives and high false negatives in terms of indicating whether or not you should hire a particular person.
Intent is often irrelevant. If something is shown to have disparate impact on a protected class then it is likely to be ruled discriminatory, and therefore illegal, regardless of intent.
You could argue -- probably with varying degrees of success, but still -- that the puzzle question is an attempt at an IQ question. Certainly, there are vaguely similar questions of reasoning ability, numeracy, and pattern recognition on actual IQ tests.
Anyone who's taken one of the commonly accepted IQ tests will recall that it's full of sequential pattern matching, word-unscrambling, and so forth. And I realize that most here have probably not taken the LSAT for law school, but that test is essentially a string of interview-style brainteasers. (The LSAT was initially derived from IQ tests, though it should be said that this topic is not without considerable historical controversy).
The last time I hired I asked two puzzle-type questions during the interviews: a math (statistics) puzzle and a programming puzzle.
The math puzzle had a definite correct answer that can be arrived at by anyone remotely qualified for the position, but which causes even statistics undergrads to scratch their heads for a minute. The real purpose of this is to see if someone is able, willing, and perhaps even eager to actually think.
The programming test does not have a single correct answer. As far as I know there's always a trade-off depending on the importance of competing priorities. I tried to lead them into discussing the design differences to handle the different scenarios. It's very interesting to observe the thought process. Some people think of one fairly workable design and then they are done. Period. Other people realize that a good design in one scenario is suboptimal in another and try a new design. Some go farther and look for an 80% solution to both.
You can call this screening to some degree. It's also about discovery, to be part of the larger picture of how this prospect would work with our team.
You have 50 red marbles, 50 blue marbles, and 2 buckets. You must put all the marbles into the buckets but you may distribute them any way you like. I randomly pick a bucket, and then randomly select a marble from that bucket. How, if possible at all, can you maximize the probability of me picking a red marble?
My intuitive answer would be to put one red marble in one bucket, and the other marbles in the other bucket, making it 100% likely to get red if you picked bucket 1 and almost 50% likely if you picked bucket 2, for a combined probability of just under 75%.
The expression I would want to maximize would be (r/(r+b))+((50-r)/((50-r)+(50-b))), where r and b are integers between 0 and 50 inclusive (I've forgotten the calculus required for this).
Most people somehow get stuck assuming they must distribute the marbles evenly (always 50 marbles in each bucket), even though it's neither stated nor implied. It's interesting to see how long it takes people to challenge their own assumptions.
I expect I would get stuck wondering if I'd be out on my ear if I answered that you could put all the marbles in one bucket, take them back out, and then put one red marble in each bucket.
TLDR: Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrated that the test is related to successful performance on the job
Thanks, tokenadult. This is an interesting result, and may give pause to some of the readers here.
I doubt it's actually true, in that brainteasers aren't scored on a pass/fail basis but are used as tools to see "how you think".
Still, don't give Al Sharpton any ideas, or brainteasers may be the new frontier for anti-discrimination lawsuits.
Perhaps the end-game is a system where job interviews are illegal; instead, you tell the Department of Labor your needs and the Department of Labor (in fourteen to eighteen weeks) sends you your new employee, chosen according to race, gender, sexual preference and [if at all possible] qualifications.
Suppose you flip a coin on 100 candidates, one of whom is qualified. You'll get about 50 "hire" answers, and the odds are 50% that one of those hires is actually qualified.
The article is referring only to employees that were hired and were also given the GMA, so qualification was not an issue.
However, using a coin flip rather than a GMA score on a prospective employee would imply a score of either 0 (missed 'em all) or 100 (perfect), and thus would likely have 0 correlation with the actual employee performance, rather than .51.
So your conclusion is right even if your analysis is not.
A contrarian p.o.v: They are going to lose out on exceptional cases where the candidate is a great engineer but is not able to devise a strategy towards a solution. However, there is a huge correlation between a good engineer and the ability to arrive at the solution (keep in mind they look at how you arrive at the solution too)
Personally, I'd make the candidate solve a decent intermediate level problem using a computer (and Google too). Give him/her the closest approximation of the typical job environment.
The few times I've been asked it, mentioning it by name results in them swiftly moving on ... the one time it didn't, the person didn't recognise the name and insisted I'd provided the wrong answer - which was a huge time saver for me in terms of turning them down.
It's entirely possible they were trying to force me to defend the position I'd taken. It probably annoyed them that I took their pet interview question (that they expected me to struggle with) and shot it down in front of them. Not that I'm claiming any special skills - probability is a hobby of mine, this problem was a gateway drug into it and I work at the very low/noddy end of quantitative finance - so a certain familiarity with things like these is to be expected. I was more surprised the guy didn't recognise the name when I said it.
I'd go as far as saying that this question makes your interviewing process to be more "cheat-friendly" - the chances of an interviewee not knowing of the problem and solving it on the spot are a lot lower than those of him having heard of it and pretending to solve it on the spot.
The first time I heard of it I was surprised about how counter-intuitive it seemed, but it didn't baffle me completely because I have a strong information theory background. From an information-theoretical point of view, it's not completely counter-intuitive that new information can change probabilities a posteriori, particularly when there is mutual interdependence.
To be fair most explanations of the Monty Hall problem gloss over the most important fact, that Hall knows the correct answer. When I first learned about it I had a hard time comprehending it until I very carefully reread the description and realized this key fact.
I can't remember where I read it (or I'd provide a link) but the real "a-ha" moment for me came when I read "... so what if there's a million doors instead of 3? Does that make a difference" (it doesn't to the argument but it does to most people if you frame it in that way allegedly) in an explanation.
Unless you're young and have little to no experience, that attitude will and should fly. In that case the company should be just as much trying to convince you as you are convincing them.
> Unless you're young and have little to no experience, that attitude will and should fly
I can only see you coming across as an unpleasant person if you just start asking the interviewer how many golf balls fit into a bus right after they ask you why man hole covers are that size. And then there are companies like google (and the many who blindly copy what they know about google's process) and the few available details about their interview process all point to the fact that these brain teasers are really important to them for various reasons, so that attitude or just not answering these questions at all or asking them in return will very likely rule you out there - which is ok, of course, if you do not really care much about working for them or not. But in most companies, especially those using these dreaded brain teasers, I doubt this will "fly" and in most startups they will just talk shop with you anyway.
And while these brain teasers might or might not tell them something about you, you asking the interviewer a brain teaser is most certainly not going to tell you anything about the company other than, well, they have a pretty default and quaint approach to interviews. So what remains is definitely a bad after-taste and if that is what you are going for then good for you. But don't forget, most HR drones are pitiable people who have to make impossible decisions about people they don't know based on all sorts of nonsense and lies and then they are responsible when the new hire turns out really bad... so, no wonder they rely on superstition and obscure magic to tell them the future they can't possibly know.
Most importantly: If you are really that good and they all want you, you shouldn't even have to deal with HR to begin with; you should be aiming for that. So the whole thing is a completely moot point and the OP comes across as quite arrogant.
"I can only see you coming across as an unpleasant person if you just start asking the interviewer how many golf balls fit into a bus right after they ask you why man hole covers are that size."
Why is one any more unpleasant than the other? An interview is a two way street.
If the company values brainteaser solving enough that brainteasers are asked on interviews, why should someone considering working there just assume that everyone there is capable of doing them? If my job were really similar to solving brainteasers to the point where they are being asked on interviews to judge my employability, I don't want to work there if my potential coworkers suck at brainteasers, because then I'll just end up with an unfair workload picking up the slack for them.
And if the company doesn't really value brainteaser solving ability, having that pointed out to them by turning the tables is perhaps a very useful lesson for them whether they realize it or not at the time.
Do you mean the attitude of the post or the attitude of the headline? I don't know where the headline came from, but the original post is logical and interesting. If that didn't go down well in an interview then it certainly wouldn't be a place the poster (or I) would want to work.
Well personally I wouldn't have minded being employee eight, or even employee eighty, at Google, and right now I'd be feeling pretty silly if I'd turned that opportunity down just because they wanted to ask me about golf balls in buses.
Not everyone who asks these problems is an idiot. Given the amount of dick-swinging that goes on from Google employees and the famed interviews one gets from the big tech companies I can imagine a lot of younger managers reading these and thinking that it's how you get hold of great engineers. The simple fact is that a lot of the time interviewers aren't experienced at interviewing, and most of them will spend longer formulating a process than an interviewee will spend preparing for said interview.
Anyway, surely the ideal thing to do would be to play along with the interview, answer the stupid question as best as you can and when the interviewer ends with "any questions for us?" you ask them why the felt it was necessary to test you on a general-purpose brainteaser with no relevance to the job when they could have asked something more relevant, like "x". This way you show that you roughly know what you're doing, you're aware of how the interview went and you've provided a question that will make people sit up and think.
This is very true. How to get past first interview, without ever knowing what you are talking about: Take 15 interviews where you fail miserably, record them all - at some point you have a pool of questions/answers that are good enough to get past the first interview.
If anyone refused to answer questions like this in an interview I was running, I would ask them to leave and tell them they wouldn't be hearing from us. I don't care what your qualifications are, if you are an asshole who will not take directions, then I don't want you in my department.
"How much would you charge to naturally irrigate the sahara desert"?
"How many golf balls can you fit in a bus"
"How many piano tuners could be employed in Seattle?
These questions do have everything to do with a programming job, it tests your ability to take an abstract problem and break it down logically into reasonable parts. It's called a Fermi problem and if you are incapable of thinking through them, I don't want to work with you.
You're really saying you don't want to work with people who:
1. Don't know the volume of a golf ball
2. Don't know the volume of a bus
3. Don't know the amount of a bus's volume taken up by seats and other protrusions
I think that's a little silly. I strongly doubt a single one of those will make you a better programmer. Likewise, problem 1 is a physics problem. If you're applying for a job programming physics simulations or video game engines, it could be relevant, but otherwise I don't see how knowing the physics of water would be helpful in the day-to-day experience of programming.
I see how the underlying skills of solving the problems could be helpful, but in practice it seems like brainteasers tend to be more trivia than problem-solving.
You are not tested on your ability to remember the golf ball diameter or equation to calculate how many spheres of diameter d can fit in cube c.
You are expected to demonstrate you can roll the concept around in your mind and make progress toward answering the problem. When you do interviews you'll find people who literally can't think to save their own lives. Their minds have atrophied to the point that the only way they can solve a problem is to have their hands held by a competent person and told "now i need you to find the volume of a gold ball", at which point they would have to be instructed how to do even that.
There are some idiots out there who have brains that have gone into coasting mode. No new learning, no new problems, everything they learned up till this point is the only thing they are capable of. You want to weed these people out quick. This question does that wonderfully. The incorrect answer is "ummm. I donno, and i don't know how to start.... Hold me".
However, i hate when i get questions like "how many golf balls can you get in a double decker bus" and "whats your most embarrassing moment", both of these were questions i was actually asked in an interview. My answer to the first was "I dont know and i'm not going to try and work it out because 1) i dont have enough information and 2) its not relevant to this position. I'd happily answer hypothetical questions about situations that could plausibly come up during this position, but working out number of golf balls in a bus doesnt show you how i would solve the problem of the database running slow or validation checks not working correctly." For the second one i simply answered "I have never been embarrassed in a professional setting as i mainly just do my job and i'm not embarrassed to say 'I dont know' or to ask for help. The only times i've been embarrassed were in my personal life and those are stories not appropriate for an interview."
Needless to say, these were questions asked by the CEO of a startup i thought was absolutely amazing, i had sat through a tech interview with the CTO before the CEO came in and felt i did well with the tech questions. Once the CEO came in and asked me these questions, i wondered how an awesome startup was achieving this success behind the leadership of total and utter moron. I decided not to work there as i lost all respect for the CEO.