Our industry's hiring practices are absolutely obnoxious. In an ideal world, you could trust someone's resume. e.g. "I've written large production programs in C." Ok, great, so then clearly this person knows C so there should be no need to dissect code or run them thru some linked list algorithm and see if they know how to use pointers correctly.
Yet, we do. Ok, then what about open source contributions (if they have them)? Well, sure, but we don't really know, for sure, if that code is their own or if they indeed truly wrote it, can we? So we can't count on it. It helps but it can't be an automatic "this person passes the technical" signaler.
So maybe the person shares actual code and walks us through it that they wrote, even if not open source? Again, we can't know for certain it is their own code and not some friend who wrote it for them 2 years ago that they claim is their own. Or was a teammate's, or what-have-you. So we can't count on that either.
Apart from getting a candidate to actually pump out some code, even trivially, we don't really have a way to de-facto verify that the person says they can do what they say they can do, short of a personal reference from someone, say, in the company already that knows and has worked with the person in the past.
I absolutely hate this practice of having to constantly "re-prove" to the next employer that someone knows how to write software. It's extremely redundant and yet we keep on doing it, with no real hope of actually making it better.
> Our industry's hiring practices are absolutely obnoxious.
Agreed. The solution is to not work at a tech company, but instead work in a tech capacity at a real company.
- No silly games (before or after hiring).
- Better benefits (I'd rather have proper health coverage and a pension than a room full of toys and vaporware stock options).
- Immensely more job security (Henry Ford didn't have an "exit strategy").
- People respect you more as a professional with a needed skillset, and not a throwaway cog in a machine who can be replaced or offshored on a whim.
- And in many real companies, proper internal HR groups focused on making you a better person and a better employee, and not some farmed-out commission-based headhunter group that makes money on employee churn.
> In an ideal world, you could trust someone's resume.
That's not an ideal world. That's the real world. Meaning not SV, and its wannabes.
Real talk? I'd be concerned that, at a non-tech company, I'd be seen as a IT rather than a software engineer. And the conventional wisdom goes that IT is a cost center.
Some of the best career advice I've ever gleaned is to work on something that visibly makes money for the company. I get to do that every day at my tech company. I don't know that I could say that if I worked in a technical capacity in another industry.
The best "work environment" tech job I ever had was at an investment bank. Didn't disrupt anything (quite the opposite) but got great experience with databases, unix, C, scripting, working with internal clients, determining requirements, managing projects, etc.
9:00 - 4:30 daily hours, no nights, no weekends. Full benefits, good salary, good yearly bonus, quiet office, great people who were there to get a job done and then go home.
Had to wear a shirt and tie but that's not the worst thing in the world.
I'm gonna second this. Started my career as a software dev at a commodities exchange. Pay was excellent for the area I lived in and my experience level, 9-5 was 100% standard and enforced by the team culture, on-call was nonexistent as dedicated support people handled that, bonus was better than the ones I got while working at Microsoft, and got to work with some of the best software engineers I've ever met. Incredible job security, no one got let go or put on any kind of PIP as everyone was pretty self motivated--even if they weren't the best. All in all, I would highly recommend this as a good career path.
I left because it was my first job out of college and I wanted to see what else was out in the world. But it stays a fond memory of a solid job.
There's a very strong correlation "you have to dress like an adult", and "you will be treated like an adult".
Yes, a lot of adults wear sweatshirts and hoodies now. However, if you wore a shirt and tie to a high school, they would make fun of you for dressing up like your dad. We have collectively agreed on a "grown-up" attire.
In other countries, it's very common for boys to wear a shirt and tie to school. Even here it's part of the uniform at many private schools and some charter schools, and it's the norm for children at formal events. It's not 'grown-up' attire, it's 'serious' attire.
I wore a tie to school every day in high school. Then I did it again at a well know startup. In both instances someone pulled me aside to tell me "keep dressing like that and you're going to stand out."
I attended catholic high school in nyc from 1965 to 69 and we wore a sport coat and tie with dress slacks. Hair length inspections were conducted. It didn't kill anyone as far as I know.
The way I've always seen it, if you insist on being able to dress the way you want as a means of self-expression, then you really don't have anything worth expressing.
Dressing for comfort, or to suit the physical demands of your job are valid motivations. I wore a shirt and tie at my first job (back in 1987), but within a few years, I didn't bother any more because no one else did. Nowadays, "business casual" is fine by me.
"If you insist on being able to dress the way you want as a means of self expression then you really dont have anything worth expressing."
My initial reaction was to guffaw at the ridiculousness of such a statement. Expression through fashion and attire is pretty much the entire reason we dont all wear the same thing as each other every day. I feel the exact opposite as you: people who dont dress with the awareness that their appearance is a statement to the world they walk into are missing a massive opportunity to affect their day. How people treat you. How people respond to you. Etc.
But since I'm growing less cynical in my old age, I would care to learn more about your perspective if you care to share.
Well if you think about it in terms of the man-hours that must have been wasted doing "hair length inspections", it's probably equivalent to at least one person dying young :P
The zeitgeist in tech and in culture generally is a moving target. Look at dandyism in the 19th century, no one would argue that is how grown-ups dress today.
I wore a blazer, shirt and tie for every year of school from 11 to 18 years old, everyone did. In fact the last two years we a full tweed jacket that spelt like wet dog when it go wet and was nigh on indestructible.
Which, of course, tech companies don't value at all. From what I've seen, they much prefer to hire a fresh grad over somebody with a long record of proven performance.
So, I agree that one should seek work in tech at a non-tech company.
Apply. I would like nothing more than a flow of applications from smart software engineers with BSs over scientists exiting of postdocing and MCFs.
The skill I look for over all others is an understanding of the standard principles of software design, and principles of testing is a bonus. I want to see that the candidate is honestly interested in our language to the extent that they are familiar with the obvious pitfalls of the standard language features. Candidates who prepared by writing option pricers and passing leetcode challenges won't necessarily understand any of these, because they are simply writing functions.
I assume that anyone who got through a rigorous engineering undergrad has the mathematical preparation to perform as a quantitative developer. This won't be true for all roles, but it's true for all of the QD roles that I have contact with at BB.
If you are interested in a dev role in finance, and see requirements for basic financial knowledge, risk, etc. -- as long as the requirement isn't quite specific, such as experience pricing specific option types, pricing complex products, or with a specific product, I would just ignore it and apply. Many of these concepts are quite simple compared to what SEs are trained to model and implement, and the people on the team who need help know that.
I've seen GPU and C++ skills requested for a Pandas shop. Just apply.
I think hiring managers and HR think they will attract better candidates by adding these things, but it is counterproductive.
From what I can tell from salary surveys, we pay an entry-level dev better than many others. >150 guaranteed all-in first year, and better than that afterwards if you just do your job. Promotion path is good because modern skills are in dire need.
They're large companies recruiting pretty much in all area of technologies. If you have experience in something, you should be able to get an introduction through a recruiter or find an existing employee to reference you.
The only trouble is that finance is only in the NYC area. You can't be there if you are in the valley.
"The only trouble is that finance is only in the NYC area. You can't be there if you are in the valley."
There used to be some in Chicago. Bank of America, Swiss Bank, First Chicago. Swiss Bank is now UBS and I don't know where they do their trading system development. First Chicago got absorbed by JP Morgan Chase. B of A might still do some trading development there, but I have no idea.
Back in the 90s Swiss Bank and First Chicago used NeXT for trading apps, and Swiss Bank had Symbolics machines here and there.
That might be the difference between "investment bank" and "bank" because I've heard stories about near-unhirable people who've been toiling away with Java 1.5 (as late as 2016) at (German) banks. They hated it, wanted to get out, but they'd lost touch with current tech unless they managed to sneak in some off-work hours with non-ancient stuff :/
Working at a (German) investment bank still using Java 6 and Perl 5 extensively in 2018, I can assure you it's not just retail banks. Legacy technology mires are everywhere in the enterprise world and often the premia to work in those places is precisely for dealing with this kind of old technology and decades of technical debt.
In any case the same bank with Blockchain and big data lakes will also be working with mainframes and COBOL, so generalisations about technology aren't particularly useful when it comes to organisations that are so large.
Real talk? There is nothing wrong with being in IT. Imagine if someone said "I would be concerned I'd be seen as a carpenter and not a woodworking artisan".
You're a carpenter. You nail boards together to serve the business interests of the company. Even if the boards are SQL queries or ReactJS or whatever.
Companies need tech. You can bring value by making everyone's jobs better with your technology - sand off those rough edges on everyone's workflows. Demonstrate your value.
> Imagine if someone said "I would be concerned I'd be seen as a carpenter and not a woodworking artisan".
Carpenters build houses, woodworkers build furniture. IT people reinstall operating systems, fix printers and networking issues and software engineers write software. They have a different education, different job description and different pay scales. A woodworker wouldn't make a good carpenter nor a software engineer a good IT person or vice versa.
There is absolutely nothing wrong in being an IT person or a carpenter, they're reputable jobs that put food on the table. But if your trade is being a woodworker or a software engineer you should be slightly worried if you end up being put in a carpentry or IT. Or vice versa.
Disclaimer: I'm biased because I'm a software engineer and a woodworker. I'm good at my job but I can't fix networking issues or build houses.
At any decently sized company, there is no overlap between IT (helpdesk and desktop support) and engineering jobs. I worked at a regional company with six people on the security/compliance team and not once was I ever asked to fix a printer or troubleshoot a laptop. We had dedicated people for that.
It's actually at the smaller companies (like startups) where you'll be expected to do that. I worked at a company with <20 people total in the entire "IT department" (encompassing all computer-related things). Even though my job title was security analyst, I was in Active Directory, I was configuring Cisco gear, I had the on-call phone for the entire department, my phone was in the helpdesk rotation, I'd be dispatched to satellite locations to troubleshoot their wireless.
In my current job as a consultant, I've worked with startups where their engineers are answering support emails and replacing their own hard drives because they don't have a helpdesk or a desktop support team.
'exDM69, I'm not arguing with/against you, just piggybacking off your distinctions to help the conversation. If you're afraid that moving from a tech company to any other enterprise will have you putting on many different hats, it's probably not true. Unless you move to a tiny company, which is true for tech companies as well. Any real enterprise company will have dedicated IT support teams and you will be reprimanded for trying to perform your own IT maintenance since that's not your job.
IT fixes peoples computers. Software Developers don't do that. We aren't Network Admins, we don't know how to set up or fix AD and we don't manage hardware. IT is a different field. I write software to process data and mostly I don't interact with hardware or troubleshooting arbitrary software directly. If I am offering support I am a level above IT. I understand that IT needs to be done but it is a very different job and should be treated that way.
... and that is wrong with the software development today. Our generations of developers started with playing games, using computer, reinstalling, configuring computers, then we started to mess with computers, os, hardware, network, routers, programing came in somewhere in the middle (we wanted to make a game, demo scene, ), cleaning malware, then we started automating the computer tasks, learned multiple programing languages with a clear vision - "I want to learn C, coz then I can do anything" :D, switches multiple computers, security started to become an issue (winnuke (tm) :D), riding the irc splits, taking over irc channels for fun, started to study networks at low level, got first data loss, learned to rescue data, switching controllers on hard disk, studying prolog as it was hilarious, got first job, ... when we started to develop commercial software we had accumulated decade of knowledge from multiple platforms and OSs (I have wrote my first program in simons basic on c64 at age of 11, put together my first 386 from components). We have tinkered with anything coming under our hands, software development was just another skill and we use our knowledge from IT in development and vice versa.
Today? At 20... "what will I be doing for a living? Hm, I did some html and copy pasted js, I WILL BE A DEVELOPER and be RICH". Went to college, learned to develop. But I don't have any foundation, web and sql is all I know, and I am sleeping with Programming patterns book under my pillow and looking at latest programming hype (silently laughing at year of nosql databases and year of blockchain, now AI, and eagerly waiting for the next silver bullet that will solve ALL our problems :D)
IT is not a different field, but the level of today knowledge IS, also the attitude.
A friend of mine is having a successful software company and he said to me once: "If I get a guy with the attitude and background we had, I will hire him even if I wont be hiring at that moment. They became so rare, that it is worth having him on a bench, just to be there when you need him".
Why, I have a good view from here :) And I can take great photos of scenery, now I do understand what the photos are showing is not something that everyone would love but still, those are photos, you can shoot the one making them but the scenery is not going to change by that, or we would just shoot all war journalists and there would be no more wars.
But joke aside, what is the issue, do you disagree?
I started on other peoples computers personally. I poked around and played with linux and some networking and when I went to school for it I realized that it was too late to learn it from the ground up. If you are still trying to figure out how a driver is written in C then you aren't up to date on any current programming paradigm. While I get that 20 years ago you needed to care exactly how memory was handled to properly optimize your code, now I am writing data processing jobs that take 10-15 TB of memory spread across 30 computers. If I fuck up 1 MB of ram per computer it really doesn't matter and it is way more efficient to just add another machine rather than pay me for two months of work to fiddle it down to perfect.
The scales I work at are more about Big O than anything. If I make a mistake that is N^2 vs Log N then I wait a week for a job to finish verses 1 hour. That is an issue. If I add on 10 min because I inefficiently use swap space it doesn't matter. I need to pay attention to the higher abstraction rather than the minute details. Sure I can fix a computer but if I do that rather than stopping a job from doing puts across regions it will cost the company 12000 dollars for each hour I am removing malware.
I would argue that demand has increased dramatically, but the supply of natural-born developers (eg. those tinkering since childhood, not just following the money) has not.
In my experience, Software Developers that can't configure/diagnose/fix hardware/networking issues are usually pretty mediocre at development as well. And vice versa for "IT" guys that can't program. Specialization is for insects.
And, like, literally everyone since Adam Smith and his pins.
Assessing how good people are at skill A by trying to determine how good they are at ever-so-slightly-related skill B is what led google and the rest of the valley down the ridiculous problem solving interview quest of the aughts. Take your manhole covers and suck on them!
Abstraction is a skill, one that’s difficult to assess in an hour-long interview, but one which allows me to say “I’m great at C# but I’ll call someone when the printer is broken” - because there are only so many hours in the day.
Rather, we manage peoples computers, and the environment that allows them to function. The misconception that IT is only there to fix, is because we're invisible to you when everything works as expected.
I worked in a tech startup, where everyone did everything, and where doing everything was considered high-status: it meant you were competent at everything. Or perhaps the other way round: trying to avoid any computer-related task, for whatever reason, was perceived as an admission of incompetence, therefore low-status.
(I didn't agree with that philosophy. I am humble enough to admit that I am not an expert at something, e.g. setting up printers; and lazy enough that I want to avoid such work if I can.)
I also worked in a non-tech company with sufficiently large internal software development department, where as a programmer I focused on developing software; and if any part of the technical infrastructure stopped working, I just called the internal tech support.
> where everyone did everything, and where doing everything was considered high-status: it meant you were competent at everything.
In my current (non-IT/software dev) workplace, the CEO and the manager pride themselves on how all our staff can do everything. We're all sold on it with the line that "..all your future employers will be amazed at how much you can do!" Unfortunately, everything we produce is second rate junk as a result of this, as we are "all responsible for quality control," and the customers pay appropriately.
It's not that it doesn't work, clearly it does for some businesses, it's just that someone has to be responsible for making sure that the product is fit to be presented to the customer, and if everyone's running around doing everything, nobody's really doing anything.
I feel your pain. That is exactly where this kind of reasoning goes.
There is always something you don't know, no matter how "senior full-stack" whatever you are. And when that happens, and the company culture does not allow you to admit it, the only solution is to use google, stack exchange, download a book or two, and perhaps ask a friend -- but neither of that makes you an expert overnight (and sometimes "overnight" is literally as much time as the agile process used in your company will give you; just because someone else, who actually is an expert in that stuff, could do that in a day).
The more I know about the things I specialize at, the more I am aware that you can't reach that depth of knowledge and experience by giving three smart questions to the search engine, and reading the three resulting articles. And by analogy, I assume the same is true about some of the things I don't specialize at.
But I have kids to feed, so sometimes I just bite my tongue and produce whatever is possible within given constraints.
This company culture also leads to playing games where everyone claims to believe that of course anyone can easily do anything -- but there are tasks that everyone is trying very hard to avoid: "It's not that I couldn't do that, I am just very busy now working on some other high-priority stuff that only I can do. Perhaps John would like to take this nice and simple task?" "Uhm, I am also in the middle of, uhm, something. Perhaps Joe could do this?" Joe doesn't have a plausible excuse ready (or is not present at the moment), so he is assigned the task. The meeting ends. If anything goes wrong, it's all Joe's fault.
In long term this reinforces the status ladder within the company, because the high-status guys are in the best position to refuse the tasks outside of their field of competence (by claiming they have more important stuff to do); the low-status guys get stuck with the tasks, produce mediocre results, and that is taken as a proof that they really deserved the low status. (If they are inexperienced 20-somethings, they will quite often buy it, and feel guilty about their incompetence. Then some moment later they change job, and realize it was all actually about the company culture.)
I'm at a "tech" place, but we're very small (8 developers).
Laptops, printers, email, document storage and so on is part of the software development team's responsibility -- so we outsource all of it, and spend a little time on it roughly every 6 months for a new system or similar.
"Hey, I can't print". "Here's the helpdesk number for the people we pay for printers". (Some people were annoyed at first, but understood when they saw the developers' salaries compared to what we paid for the "boring" services.)
I think I've made myself misunderstood. I'm not intending to put down IT work or the people who do it. But from a career perspective, if I want job security and a higher salary, I think it'd be a mistake, in the long term, to be seen as IT. IT is seen as a cost center, which leads to corner-cutting and outsourcing--even when it ends up costing way more in the long run.[0]
Tell that to $250 hr SAP and Salesforce Architects/Developers, or to Baby Boomer COBOL programmers who are given ever increasing incentives to post-pone retirement.
IT is by far more lucrative in the long run. It's just the structure of RSUs and stock options that gives start-ups the illusion of relatively larger comp gains.
No idea what a 'salesforce architect or developer' is, but how would a COBOL programmer be considered IT staff? (In the context of this thread where IT staff == support staff)
Many non-tech employees view IT people as basically “computer janitors with bad attitudes”. Heck, I’ve worked in tech for a decade and I can count the number of people who have a positive view of their IT department in one hand.
To be fair, In my (admittedly limited) experience contracting at BigCorp - the IT department's procedures are often a bottleneck in a workflow or solutions.
We all know they are following business procedure, but there will always be some unconcious tension with departments that hinder a solution (I see marketing and legal butt heads all day).
> Imagine if someone said "I would be concerned I'd be seen as a carpenter and not a woodworking artisan".
It's entirely reasonable for a carpenter not to want a job where they just nail pieces of wood together. Not really building anything, not really using skill. Just nailing a few boards together all day.
Nailing boards together is building something. Carpenters aren't pissed off that they didn't grow the trees themselves and and chop them and haul them and plane them.
Many companies these days make a distinction between IT, digital, and product.
IT runs the network, phones, issues and fixes laptops, maybe provides servers, often is in charge of information security. May own the financial and business intelligence technologies. Unless it's a very enlightened org, this is often the "cost center" team.
Digital talks to the world in a modern way--runs the websites, blogs, social media channels, paid digital ads, etc. Often housed within a larger communications and/or marketing division. Technically a cost center but the visibility makes up for it in the eyes of executives.
Product builds the technology that makes money. This would be like the e-commerce platform, ad network, mobile apps, etc. Executives value things that make money.
Often, all three groups need developers. The tools and roles may differ, but you can be a developer in a major company without feeling "stuck in IT."
This isn’t true any more with software eating the world. I work in movie distribution for instance, which was traditionally sending reels of film, but is now all RSA, certificates, HSMs, remote device management, logistics, Analytics, ticketing, databases and distributed computing. The software team gets pretty much the same treatment as everyone else, but everyone knows it’s the future of the company and sets a pretty high bar.
Speaking of which, our interview process is to ask you to solve a challenge on Github, but you’re free to keep it open source as an indicator of your skills. And it’s not work for us, these are obviously toy / proof of concept problems that we solved ages ago.
The solution to the first point is to go to a place that already has an IT department.
The second (making a bottom-line impact) is probably a non-issue. If it's not a tech company, that means they are likely failing to use tech fully, in places that really could benefit from it. Automate stuff that needs it, and quantify the time saved as:
(time it took before - time it takes now) * number of times each person has to do it per week * number of people * what those people get paid / what you get paid = time T you saved in a week, in terms of your own hours.
Each time you do something, add its T value to a running total. Likely an hour here, an hour there. Eventually it hits 40 and you and the managers can clearly see that your employment is paying for itself. Keep going and you're making them money.
Most larger companies have clearly defined boundaries between IT and software development. And with the push towards automation and security at every layer of the stack, even IT has a growing role for software architecture and development skills.
I've worked at a non-tech company as a software developer. At that place the department (we were rolled in with IT) was viewed as just another cost center. It's not great.
> I'd be concerned that, at a non-tech company, I'd be seen as a IT rather than a software engineer.
Depends on the company. Where I am, IT is in charge of maintaining the computers and networks so that the software developers can develop software in and for other departments.
No, that's nearly every company. But if there's 10 software engineers working with 50 MBA's who drive the business, none of them can tell the difference and treat you as such, which happens more often than not outside of a tech company. They are the ones making the company money, and you are the programmer doing that "easy programming stuff for IT nerds". It's not that rare of a situation
You're not just comparing a tech company to a non-tech company. You're comparing a startup to a non-tech company.
When you compare a non-tech company to a big tech company like Google, the benefits and comp arguments switch to the favor of the big tech company along with the respect and job security.
The only advantage to non-tech companies when comparing across big ones is the simpler interview.
The several hundred other auto makers in Detroit did, they went bankrupt or sold out for peanuts right before going bankrupt.
Google, Microsoft, Amazon, Netflix, Apple, Facebook, Cisco, Oracle, Intuit, Adobe, PayPal, nVidia, Intel, etc. don't have exit strategies either.
For the vast majority of businesses, you're either going bankrupt, or you're going to eventually sell as an exit. One in a million businesses turn into the next Ford scenario.
It depends on the company. MOST non tech companies imho treat all tech workers like IT, with the exception of embedded data/analytics people IF the company is even forward thinking enough to have that.
You compared the best examples of corporate with the worst examples of tech.
Pivotal and Amazon would not meet the rate and benefits that my current real company was paying, let alone the one that just poached me for a 15% raise. She's spot on.
Have you considered that you might both be talking about different sectors of tech, and different countries? The EU / UK / US / etc. tech markets vary wildly in these things.
Of course he could be talking about Europe but there are very few European "tech" companies. Tech is a horrible descriptor of course, he could be talking about 1000 different things.
OP seems to be referring specifically to tech startups with dubious financial situations but then goes on to say that all tech companies are bad to work for
I used to hate it, but have now performed enough interviews of supposedly senior devs with decades of experience.
I've literally seen someone who did firmware for the space shuttle grind for 45 minutes on fizz buzz without making progress. Like, I'd they struggle for ten minutes or so, I take a step back and say "Ok, screw syntax, let's just vaguely talk about what needs to happen and mock up some pseudo code.". Even then, grinding for another half hour with no progress.
I’m going to show some vulnerability… this happened to me this week. I’m probably like a lot of you here: professional software developer for 20 years, have shipped countless products in multiple languages / technologies. Work has appeared on tv and print media multiple times. Have generated millions and millions in cost savings and revenue for employers and clients.
But I totally failed a technical screen.
It’s been a week of reflection as a result. I nearly canceled the interview to begin with because the entire process felt like a cattle call - totally cold, the company had shared no information about themselves or the role. There were ten minutes of rapid fire (“you have one minute to answer this question.”) followed by the proclamation that the next 45 minutes would be spent on “as many coding problems as we can get through.”
Within the first few minutes I was able to describe a general solution to the problem, but over the next 40 I totally struggled to produce a working solution. Now, sitting here I could likely write it in 5 minutes. The problem was:
--I was completely frustrated by the experience from the beginning
--I was in a code editor that I wasn’t familiar with
--I had limited access to documentation
--An interviewer looking over my shoulder harassing me
--I realized that while the problem was simple, there were things I’d just never needed in that particular language before, so didn’t have a complete understanding of what was available to me.
--I focus on design before implementation, where the interviewer just wanted me to jump in and bang out code.
--I realized that the last time I wrote this particular bit of code was 20+ years ago, as a college freshman, and certainly not in the language I was using today.
So I agree with the other commenters: we should all check our premises before we disregard another experienced programmer as as hopeless hack.
I wrote a long blog article this week about what I think the interview process should be, will be posting it soon. There are ways to fix this process.
There's no shame in it. Same thing happened to me last week, and I was furious about it. I've resolved not to take any more live coding interviews, on the grounds it is uninformative, degrading hazing. We're composers, but we're being tested as if we're live concert pianists. Then companies complain that there's a talent shortage! This hurts everybody, and it needs to stop. For my part, I will refuse to participate any further.
> I've resolved not to take any more live coding interviews, on the grounds it is uninformative, degrading hazing.
Depends on the job.
I'm currently hiring and the live coding exercise is a must for the position because I want to see how the person problem solves while under stress.
Because stuff happens. My team works on mission critical systems that can have issues that we sometimes must resolve quickly which is very stressful.
If you can't handle the stress of me watching over your shoulder while you code then you can't handle the stress of getting a critical bug fixed immediately. Maybe that's not true for everyone but I haven't heard of a better filter.
Not every position is like this. But that's the big thing I think people miss. Different software positions require different skillets beyond just tech stacks. And the interview process should measure for the particular software engineer skills needed for that position.
I've also hired people who if you just reviewed their code after the exercise you'd think they were terrible. But I'm not testing whether they are good at coding exercises, the test is a tool to see their ability to problem solve under pressure and to see their level of experience with the particularly tech stack.
> If you can't handle the stress of me watching over your shoulder while you code then you can't handle the stress of getting a critical bug fixed immediately.
Those are two very different kinds of stress.
In one situation you're stressed because you feel like your every action and step is being judged. Every moment you spend floundering you feel is being docked against you, you can't help but worry about how the person breathing down your neck is expecting you to approach the problem.
In the other situation you are a part of a team, you have each others backs and trust each other to make sound judgement calls. You're working together for a common goal rather than judging each others every movement.
Your hiring method sounds like hazing, you're putting interviewees though unnecessary stress to see if they crack. Stress that isn't the same as they would feel on the job, at least I hope you don't also treat your employees the same way.
Exactly. The previous poster doesn't seem to grasp the nature of stress and its variants relative to the social or work situation.
To echo your point, fixing a bug with someone you just met standing over your shoulder clock ticking, is nothing like fixing a bug at your regular place of work.
In an interview, if you don't perform the task well, you can fuck off back to the street where you came from. In your job, you get to ask colleagues, consult previous project code, refer to in-house or external documentation, and calmly analyse to figure it out under your own "in the zone" steam.
Soo...when a mission critical system at your team goes down, you get the newly hired guy with no previous knowledge of your code, put him in front of a dev environment he doesn't know and keep pestering him over the shoulder while he tries to get the system back up?
> Because stuff happens. My team works on mission critical systems that can have issues that we sometimes must resolve quickly which is very stressful.
This doesn't sound like something a new hire should be responsible for and more someone who would eventually, after proven themselves able to, participate in.
Interviews are already stressful; you don't need to add to that by putting candidates into even more stressful situations just for kicks.
To flip it around, would you be willing to let the candidate review the details on all technical employees exit interviews and then ask you about them? Maybe review some completed financial audits from previous years? so they can see how the company reacts under pressure?
> I'm currently hiring and the live coding exercise is a must for the position
It's not.
> because I want to see how the person problem solves while under stress
Why, it's a totally bullshit proposition. The stress of not having memorized some algorithm is not the same kind of stress as a live system going down.
You do it this way because you aren't able to come up with anything better.
Would you be OK with interviewees giving the interviewers a quick coding test? You know, to make sure they won’t be working with people who can’t carry their weight?
Right? I wouldn't feel comfortable working with colleagues who couldn't instantly come up with a novel sorting algorithm to some bizarre scenario I thought of to stump people. So maybe I should "counter" each coding question with my own. It'd be like a shootout. This isn't a dysfunctional culture, right?
Yes depends on the job, but if it's a stressful environment, you need to train for it. OTJ. That's a management issue, not a coding issue.
Otherwuse these exercises would just paint a broad stroke that handling stress is common & generalized, when in fact it's unique and case by case in most situations.
You appear to view stress as a good thing, or, at the very least, an unavoidable part of the job. It seems likely to me that you deliberately manufacture stress by deploying mission-critical systems that have issues. Seems there might be deficiencies in your pre-deployment testing regime -- a stress factory for sure.
tl;dr: If you're a stress factory there's no way on earth I'd be willing to work for you in the first place.
I agree this point is valid, but I think it's much more exceptional than whiteboard interviews are. I've personally never applied for such a job, since I typically work on consumer products at BigCos with a glacial development pace.
People have already hammered you for this, but a Merciless Judge actively looking over shoulder while you're working is a completely different kind of stress than trying to get a fix out the door ASAP.
Only way to code under pressure like that is with practice. I turn into an idiot non-coder in these situations, but I've gotten much better over time, to where I always pass a live coding session. In the real world, I spend a fair amount of time thinking and teasing a problem apart before I actually start coding. This is hard to do in an interview - even if the interviewer says "think about it for a few minutes", they will inevitably step in and offer a suggestion after 30s of silence. It's like you have to code by pure instinct. Which is doable with practice. But it's hugely skewed towards people who have the time or desire to practice such things.
This is why advice for passing the Google/etc. interview usually boils down to: Quit your job for a month and do competitive coding, and you will pass with ease.
This. I noticed that I have gone for over fifteen years through some pretty tough jobs, but never needed to write any Leetcode binary tree problem from scratch in thirty minutes in my work while being questioned. I normally freeze up because I worry about current best practices every step of the way.
In the end, I scored a job with a FAANG company literally because I didn't want to work there. A friend warned me away before the interview for work-life reasons, but I decided to go ahead with the interview anyway as practice and I nailed it. There's no reason to stress if you don't want the job. Another friend who worked there later convinced me to take the offer.
What bothers me is: you're basically the same coder both before and after competitive coding practice. You're only more marketable with the same skills.
>What bothers me is: you're basically the same coder both before and after competitive coding practice. You're only more marketable with the same skills.
Yup. You keep keep measuring that thing..I don't think you're measuring what you think you're measuring.
I am sympathetic to this but I have dealt with very senior candidates that could not demonstrate any coherent thought about a given problem over a whole interview. Others don't get code done but at least they can express a clear understanding of the problem and have an intelligent conversation.
I don't care if they can write the code but I want to see some level of engagement with the problem and some understanding.
So a little more on the interview. I've got a laptop setup with eclipse, all rigged up with an integration test and filled out function signatures. I show the interviewee what to press to compile, and the subsequent failing tests. The code builds in it's initial state, but the tests fail. I also show them on the desktop that the original source is squirreled away in case they fat finger and erase everything (I've totally done that too, lol. Source control is a beautiful thing in the real world). Then I peace out for a few minutes to give them space (I know I'm close to an order of magnitude worse at typing when someone is looking over my shoulder).
Given the negative response we've had in the past about take home assignments for senior devs (their time is pretty valuable, so I try spend the same time they do), I'm not sure what else to do to accommodate.
It's not entirely obsession/neurosis either. Several of those things are valid training/familiarity needs and/or safety/ergonomics/self-care needs.
For instance, I nearly avoided very early carpal tunnel issues in graduate school by entirely relearning to touch-type on the Colemak keyboard layout. Sure it only takes a few minutes to switch to the layout on Linux and macOS these days, when I remember where the keyboard preferences are, but it still takes Administrator access and a software install in Windows.
Even "easy", that's still a cognitive load distracting from whatever the actual question was and starting into the actual problem. More importantly, at this point in my career, it's something that I'm going to see this as an immediate sign that employee ergonomics may not be a concern for the person giving such a test, which may speak to other aspects of the work environment.
Todd, is that you? Sounds a lot like the process used by someone I worked with years ago.
It sounds to me like you’ve done some careful thinking about this, and it doesn’t sound unreasonable. One of my beefs really gets down to the interview process turning into a one-way grilling, dehumanizing the candidate into a “code monkey.” It sounds to me like you are trying really hard to evaluate their technical skills, but in a way that is collaborative.
> dehumanizing the candidate into a “code monkey.”
Bingo.
A hiring process that attempts to convert the multidimensionality of humans to a handful of numeric variables is essentially trying to hire the best drone out there, not the best human fit for the job.
I learn more about candidates from the types of questions they ask me, and their reply to open-ended questions I ask them, than from any technical grilling test.
A dev manager I work with insists that one of the best indicators, along with all the usual things you try to suss out about a software development candidate is whether or not he does any software development in his spare time. Not everyone who's good does, but there's a good correlation between people who would be good at writing software, and people who do it for fun.
So I would take writing software for fun as a plus, but would not take not writing software for fun as a minus.
Pair. I do this. I drive, with my machine, on my setup that I'm comfortable with. They navigate, and tell me what to do.
This gives me a pretty deep insight into how they approach a problem, but also how they interact with others. There's no adversarial element, and it gives them the chance to see me screw up to give them the mental space not to sorry about stupid errors.
The one worry I have about this is that it's more vulnerable to my own biases than a more disinterested process would be, but if I'm consciously aware of how that can play out, I'm in a better place to counter them.
I find trying to describe what I want another person on a computer to do an incredibly frustrating experience. I can't imagine how awful it would be under interview conditions!
It happens to a lot of us and sometimes fear that is just how the industry is. I have heard senior engineers just stop interviewing and often switch jobs via their network/friends.
> I wrote a long blog article this week about what I think the interview process should be, will be posting it soon. There are ways to fix this process.
I really hope that's true. Every solution I've heard so far is unrealistic at best, like using take home assignments, as if no one would cheat or complain about those.
Perhaps it's because you insist on testing their ability to live code on the spot as a performance piece, and misinterpreting that as testing their ability to code. If you're turning down someone who wrote space shuttle firmware, because they "can't do fizzbuzz", then you are administering a terrible test and getting a false negative.
I have 20 years of industry experience. Everything on my resume is the truth. I've worked on serious code, for serious projects, at serious companies. At work, my performance reviews have been stellar.
When I do live coding interviews, I'm being tested with tasks that are utterly remedial relative to what I've done professionally over long periods of time. Yet I flub these remedial tasks. It could be because I'm nervous. Could be because the interviewer keeps interrupting me mid-line, so I can't think. Could be they are too terse, and I can't drag enough information out of them. Could be the problem is slightly more complex than I can accomplish under any circumstances in a live performance, but that I could complete correctly in a closed room.
These interviews are degrading, disrespectful, and they tell lies to the misguided interviewer. What do they think they're testing the candidate for? Why does it matter whether you can code something remedial, live? Is live performance part of the job function? Do other professions do this? We should end this practice.
> If you're turning down someone who wrote space shuttle firmware, because they "can't do fizzbuzz", then you are administering a terrible test and getting a false negative.
Or... large companies, particularly defense, excel at hiding mediocrity in their ranks, and it's easy for someone who produces no or negative work to be handed around rather than fired.
The reality here is that everyone is supposing their particular bias explains what happened.
No one seems to have asked what actually went wrong in the interview. Was the interviewee truly incompetent? Were they nervous? Were they fine at code but terrible at interviews? Was the language or environment unfamiliar? Had they just got off a plane after zero hours of sleep?
Without data, opinions are worthless. And generally, there's too little data about the relationship between interview performance and job performance, and too much unthinking mimicry: "We do this because everyone else does."
That is why I advocate for a hypothetical "bar exam" for the software programming profession. I also posted a question concerning that: https://news.ycombinator.com/item?id=16702389
Going through a series of technical filters is not a problem in and of itself. The problem is that each company has its own set of filters, many times with skills that don't necessarily transfer to other interviews.
With the "bar exam" approach, you only need to pass 1 exam for N number of companies, leverage that to fast-track through a company interview. With the current approach, you need to take N exams for N companies, and return to square one if you fail. There is no extra layer of validation that lets you keep a "ahead" placement in these interviews. Company specific interviews should focus on basic soft skills, and if they mesh well with the team.
I'm an experienced programmer- not yet considered senior level, but have worked at several companies -and having a hell of a time going through my latest job hunt.
I have been applying for jobs for 3 years straight now, and it's telling that the only offers I've gotten were three short term contract gigs- one from a past employer who's already familiar with my work, and two from small studios with a very informal process where they just looked at my resume and Github projects and with that, they were very convinced that I'm the man for their job. I took these jobs mainly to add more skills and to fill up the gaping time hole in my resume.
But for the "regular" full-time jobs at businesses, big or small, local or on the west coast (I'm a midwesterner), I have not gotten an offer. I've been through many tests and live coding going through a lot of things. Triplebyte expects you to be a speed demon, but at least they give detailed feedback on what you did wrong.
So what is with the saturation of applicants and tendency to make the hiring process slower and slower? That's what these coding tests and interviews do, they make the process slower.
It's not 2009 anymore- unemployment is reasonably low, so this current phenomena of rigorous testing shouldn't happen. More programmers need to discover and embrace the power of collective bargaining.
I'm intrigued by the on the spot performance aspect, so I wonder if anyone has performed a skit at an interview. It's a tempting idea. Some people are intrigued by live performance, especially the kind with constraints like this, people like Frank Abagnale for example.
Probably, most likely on the Ali G show. If you know enough inside details on the typical interview pattern at a company this would be easy to do, and you would probably get the job.
The other option is to check references, do credential checks, and simple bullshit screens during interviews, like every other professional interview.
Literally every other profession has this challenge! Programmers are unique only in their seeming inability to realize that they are not unique.
Accountants are not asked to solve double-entry accounting whiteboard problems. Finance professionals are not given take-home banking projects. Only programmers seem to be unable to be reasonable interviewers.
Every accountant attending an interview can produce a certificate from an accredited national group saying that they have taken an in-person, proctored exam on double-entry accounting, which is the 'credentials check' you refer to. There's nothing like it for programmers.
There are plenty of options. Programmers only just have to pick one, or create a new one for all that it matters. For instance, NCEES that runs the Professional Engineering accreditation that every other Engineering discipline utilizes, has a Software Engineering PE. Tomorrow we could grandfather in every Senior Dev with 5+ years experience as a PE and then start requiring it, and encouraging kids to train for it and acquire that accreditation.
One major proctored exam scales a lot better than every company for themselves reinventing an almost proctored exam.
That is because it is an actual __Engineering__ exam, not a programming one. Testing for programming skills in an SE exam is like testing a mechanical engineer's CAD skills. I would argue that the majority of "SE" jobs do not actually entail engineering but software development (or construction), which is only a small subset of SE.
The computer engineering exam has tons of the underpinnings covered, with a whole section on relevant math. Where as the software one pretty much only covers process. The one algorithms or discrete math question they have boils down to just 'what is big o complexity'.
I guess the question is what is the necessary "base set" that shows proficiency enough in learning that you could throw someone at a problem and expect them to solve it. Keep in mind that to keep a PE active there are also lifelong learning requirements.
Certainly what I see in the PE today seems like a good starting base. Having some idea of Big O complexity can be a great place to start when working with algorithms, and certainly from my perspective I'd rather have an engineer with a solid idea of Big O complexity trade-off issues than a bunch of rote memorized implementations of classic algorithms.
Also, don't forget that a lot of discrete math and algorithms bubbled up into the Fundamentals of Engineering test that is the prerequisite of the PE test:
Which isn't to say that the PE can't be improved for Software Engineering to better meet industry needs, but certainly the current PE still stands as a good starting point that the industry could try to hone to a better tool if they wanted.
And? You think that a certification exam provides more evidence than a university degree? (which, let’s face it, most programmers have).
No, this is a made-up difficulty, caused by the fact that programmers think there’s a silver bullet solution to evaluating competency in an “objective” manner.
I don't see how that is actually doable all the time. In my short career I have moved from embedded c / c++ game engine / full stack / DBA / backend. All at different jobs with the previous manager not knowing I actually knew how to do the next job, if they were asked anything other than generic questions I would have sounded like a fraud.
This is not to take away from your point, since I agree with you, but both accountants and finance professionals have stringent licensing requirements which (in theory) could provide a baseline expectation of competence.
That is the reason they get away with not testing them, because they have a universal test they can look to. Software jobs are pretty fluid, so unless we have a massive standard test that cover anything from embedded c to FE with react, it seems that kind of test is out of reach for software people.
Civil Engineering has a huge gamut in specializations and in the last century has seen quite a lot of technological change; programming isn't as special as it thinks it is here. A credential test doesn't have to cover everything, it just has to cover the foundations and get some sense that a person is capable of life-long learning and improvement.
Yes. I told the interviewer he could call any of my past managers, knowing they would take the call, and knowing they’d give a glowing report. He laughed!
Well...fuck that guy. There is really no better measure, assuming you have the basic social skills to see through a con job/scam. I mean yah, you could always be tricked into calling someone's buddy or something, but with any depth or tactful questions that is rarely going to fool anyone.
The other, other option is just refuse to do either. I get all my work from friends I've worked with in the last 20 or so years. I mean if a company insists on these hiring practices, and they continue to get bad applications because of it, the probably aren't going to be around for long anyway.
Yeah we try to have a limit of 4 hours for this kinda thing. We understand that it is the interviewee's free time but when you need to screen 12 people for a job it is not worth a Senior Engineers time for a small startup.
Getting the right team is one of, if not THE key to startup success.
Yet you consider it not worth a senior engineer's time to evaluate the candidates for the next hire, and the presumably with whom they will need to work - productively?
Check your org's runway and be sure to have your resume ready when it gets close to the end. I wouldn't expect take-off.
Sorry I assumed that the word 'screen' was universal. This is the first step, it is a waste of a senior engineers time with a candidate that is not even close to ready for the job.
Yes, I suppose 'screen' is a bit ambiguous. Doesn't seem like a pool of a dozen is the first step. I'd want the Sr Devs involved at least out past the final half-dozen if not dozen -- not for 4 hours each, but at least a chat about what they've done, problem-solving approaches, etc.
The takeaway that's relevant to you is the section titled "Interview outcomes are kind of arbitrary". Specifically, things like:
As you can see, roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview.
If you set up a typical tech-company interview process, and if you equate "didn't pass this" with "complete impostor and utterly incapable of coding in any capacity" (as most people with your approach do), well, congratulations. You're almost certainly rejecting a bunch of qualified people, simply because you're terrible at interviewing them and can't actually tell from your process whether or not they're qualified.
The most generous thing I can say is that it's not entirely your fault. You adopted ideas and approaches from people who seemed authoritative but turned out to be the actual impostors in this situation. But now you have a choice: you can reject that, now you know the problems, or you can double down. I suggest not doubling down.
Agreeing with and expanding on the interview studies... In most cases, the job that one ends up doing has very little to do with what you were tested for in an interview. Languages usually have some sort of approved/well know implementation of common data structures and algorithms. When you are asked to do some BFS/DFS/binary tree work in 45min or less you are unlikely to do that for the job. Someone has already provided an implementation. I venture, you are most likely to see it bought up in your next interview (whenever that is).
The question is how can you test for experience in writing code that is maintainable, readable, 'instrumentable' and contributes to a better system. How often does a candidate refer and defer to a decent third party library instead of feeling they need to (re)implement a well known data structure or algorithm? What have they done to help automate things so they can free themselves and their peers from minutae (aka how willing are they to put more work on machines?). What more do they think they can do. How well do they mentor or are ready to be mentored? Given a sample project, how would they go about running the technical side of things. If they focus on code, you should try to motivate them to expand to other areas (without telling them what they are). How cognizant are they of the ecosystem; or does their vision end at coding + unit test.
Lots of intangibles that cannot be measured with a github profile or a white board interview but probably contribute more to picking out better candidates.
Sure. But, of course there is a difference between having a conversation about the overall trends of a practice versus a small subset of personal experiences. It's also important to define what you mean by "can't do FizzBuzz". Like they can't even do a basic loop or they don't get it perfect, in the exact way you think it should be? Because those are very different as well. I think it's fair to say that if you get a senior Dev in that can't even write out an if-else statement checking for 3 and 5 you have a problem.
> It's also important to define what you mean by "can't do FizzBuzz". Like they can't even do a basic loop or they don't get it perfect, in the exact way you think it should be?
Like literally can't do a basic loop. I'm not looking for 'my solution', I'm looking for any solution.
> I think it's fair to say that if you get a senior Dev in that can't even write out an if-else statement checking for 3 and 5 you have a problem.
And yet, I can tell you from experience that there's someone out there who (at least was on the team that) wrote firmware for the space shuttle that literally can't if-else with modulus despite having a great looking resume. And they are far from alone.
Why didn't you check references? I mean wouldn't you consider it a failure that you got a candidate to the interview table that was so bad? Assuming you aren't making it up, the guy was obviously lying on his resume, that could have easily been solved by checking his references.
References would be personal references - someone with a cell number and call the college he attended; they should also verify enrollment and completion. You don't want to be calling the HR department. Also, a secret is if the reference (not HR) only verifies employment, that's a negative review. They are concerned about liability. If it was a good employee, the references have no qualms about saying so; it's the negative aspects that carry legal liability.
You need to ask for those, not just prior companies. My resume has a "References available upon request" at the bottom and I'd happily give them. Is that no longer practiced? It used to be a requirement.
> References would be personal references - someone with a cell number and call the college he attended; they should also verify enrollment and completion.
Yeah, right. Personal references aren't worth squat because nobody gives references that are going to badmouth them.
The worst hire I've ever seen had a highly positive reference from a previous job.
Most companies, period, will have a policy of only verifying dates of employment. They don't want to get sued from telling the truth about the candidate and costing him the job offer.
Haha, I'd love to see it if that's true. I love cool approaches to computation. FWIW as long as you can back it up with knowledge of what the relevant tradeoffs are (and don't scoff at a more KISS attitude for doing actual work), you're pretty much guaranteed to get a thumbs up from me if you give an answer off the beaten path.
That's awesome! I'd probably follow up that, since I generally interview for real time, firmware jobs, we try to clearly bound execution time, and therefore frown against more functional techniques like that. But you clearly know how to program well and would most likely get a thumbs up. : )
Admittedly most of my phone-screen solutions aren't tuned for performance; they're tuned to make people scratch their heads.
My code for "detect if a number is a perfect square", though, is O(sqrt(n)) time and O(1) space and uses no floating point operations. Which isn't quite Newton's method, but also saves me having to memorize an implementation of Newton's method and appropriately causes confusion. Here it is in Python, with obfuscation help from itertools:
To me the problem usnt fizzbuzz, its “find all matching subtrees in a binary tree” in 45 min at a whiteboard.
I dont need to hit the book to write fizzbuzz, and i could probably code up dfs and bfs without prep but i dont walk around ready to retake my undergrad algoriths midterm. Especially when i really dont even know what the topic will be.
Im aware that the question i posed isnt terribly difficult and perhaps it does say somethingabout me that i need a bit more time to refresh my data structure to do this at a whiteboard.
But i do want to be clear that fizzbuzz is not representative of the 5+ hour whiteboard grilling ive gone through at nany of these interviews.
The problem with the BFS/DFS questions is that you're playing a forced game of knifey spoony: They expect you to write it in the recursive fashion and then try to stump you with "oh but how does that do on large structures?!" .. Stackoverflow. Then they'll try to force you to do tail recursion.
Said space shuttle firmware engineer probably never had to solve a pop-quiz problem on a whiteboard. It's much more likely that he/she deliberated for weeks about a simple change with the rest of their team, and perhaps even with the person/astronaut they might kill if the change had a bad bug.
Who cares if this person can't solve fizz-buzz on the whiteboard? That's probably not even a relevant skill to the job opening you're filling, or a skill they have ever needed to hone or practice. It's like determining a marathon runner's worth by timing their 100 yard dash.
If I ever need to do another coding interview again, I'm gonna troll the person asking the question by pulling out my phone, googling their question, and consulting the stack overflow post in the search results to solve the problem. And why not, this is how I (and the vast majority of people) will actually get the job done in a real situation.
Developing software requires high level/architecture type decisions that you mention in your first paragraph. It also requires solving many small fizz-buzz size subtasks, which are necessary and demonstrate competency.
A marathon runner doesn't need a stellar 100 yard dash time, but they should at least be able to jog 100 yards.
> Said space shuttle firmware engineer probably never had to solve a pop-quiz problem on a whiteboard. It's much more likely that he/she deliberated for weeks about a simple change with the rest of their team, and perhaps even with the person/astronaut they might kill if the change had a bad bug.
> Who cares if this person can't solve fizz-buzz on the whiteboard? That's probably not even a relevant skill to the job opening you're filling, or a skill they have ever needed to hone or practice.
Thing is, shuttle-style development is also probably not even a relevant skill to the job opening he's filling. Odds are, that job requires something more akin to FizzBuzz ('I want to do something. How do I do it?') than to shuttle-firmware.
The point I'm trying to make is that solving a fizz-buzz-like pop quiz on the board is a very different skill from sitting at a desk with internet access and collaborating with team members to solve an engineering problem, even if they reduce to fizz-buzz-like solutions. For this reason, the whiteboard pop-quizzes are not really useful in evaluating how well the candidate will perform on the job. I've worked with terrible team members that have aced their interviews / exams, and I've worked with amazing engineers who crashed and burned spectacularly in their interviews and do so regularly.
Also, another thing to consider is how this kind of interview encounter negatively affects diversity and the experience of candidates from different cultural backgrounds. If we treat this form of interview a sort of hazing experience that doesn't really inform on their aptitude on the job, what kind of traits does this process _actually_ select for? Would be an interesting study.
That sounds to me like fight or flight response. Happens to me sometimes, and when it does I can’t code worth a damn, in spite of having a decade and a half of experience on the top projects at some of the top companies in the industry. This, in fact, happened to me the first time I interviewed with Google: I bombed that pretty spectacularly. The second time I applied I already had a couple of very generous offers elsewhere, so I didn’t care as much and was less nervous during my interviews, so I passed.
Hence the take home interview. At least at the last company I worked at we began doing these because we could tell that the stress was filtering out some otherwise decent candidates. At least that was one of the reasons, it also gave us a chance to talk about technical solutions the candidate produced in a more stress-free way.
I don't think this assertion that the technical interview is inherently filtering out great candidates is being very honest (you responded similarly to another comment of mine the same way). One quality being tested is "can you try to do the thing that we're asking for?" That's a measurement in competence and the willingness to get something done. If said candidate is saying "this test is ridiculous and a waste of my time!" then my common sense instinct will say that they're probably not a great fit even if they're the greatest programmer in the world. At least on the take homes I've issued they shouldn't be taking more than a few hours. These stories about creating applications that take 3 days are certainly problems and recruiters need to lighten up a little, but I don't think the take home is inherently flawed just by its existence.
Most senior devs have a family, value work/life balance, and currently have another job. I don't expect them to take their work home past 5 while they're working with me, so why should I expect them to do that before I'm even paying them?
Finding a new job isn't part of your current job. It is over and above. If you aren't willing to put in a few hours after work for something not related to your job, why are you even bothering to go looking for a new job anyway? Are you job hunting on your current employer's time?
Almost too obvious solution: do an initial phone chat to filter qualified candidates into a small pool and then pay your final round of candidates a nice rate to complete a take home project. Bonus points if you can make it something that you can actually use at your company.
This is a terrible solution at many levels. Paying someone who isn't on your payroll (yet) needs a lot of bureacracy on both sides (e.g. taxes).
And even if it's paid, it's still a not-insignificant time investment for the applicant. They're probably applying for several gigs and take home assignments would quickly turn into a full time job, and filing the tax paperwork could take more time than the coding.
And finally, assigning a job that would actually get used at a company pushes up the complexity level, the need to understand the context. This would also make me very suspicious about the motives of the company.
In my company, we give a "fizzbuzz" level assignment on the whiteboard (or their own laptop if they prefer). The purpose is to weed out the applicants who can't code at all (makes a surprisingly large portion of applicants). Additionally, we've noticed that the ones who are good programmers will ace these tests.
I don't think that whiteboard assignments or takehome work is a good way to assess how good a programmer is. A 15-30 minute smoke test with a binary result (the applicant can or can not code at all) is good to filter out bad applicants but not to distinguish good from excellent.
The take home exam is quite a bit more of an efficient filter than a blanket phone screen. Anything less than a 20 minute call isn't going to tell me much.
Personally I think it'd be fine to e-mail back and say "sorry I don't have time to work on this with my current schedule, could I do an in-person interview instead?" That's me though, I would hope the rest of the industry is willing to work with people. Even just the response to the e-mail would tell me that they're competent enough to know their time management and that they have limited bandwidth. At some point you need to make a sacrifice of time though. Even if your hotel and plane ticket are being paid for to fly out to the place of business, you need to take that time off to go in for the interview.
I'd prefer to do a lot of that at home personally.
> Personally I think it'd be fine to e-mail back and say "sorry I don't have time to work on this with my current schedule, could I do an in-person interview instead?"
What happens instead is that you just don't ever hear back from candidate.
>I don't think this assertion that the technical interview is inherently filtering out great candidates is being very honest
There are plenty of people telling you the same thing, you just don't want to hear it. Here's a data point. I would never do it unless I was absolutely desperate. I mean about to lose the house desperate. There are just too many other companies around.
I mean if you are trying to pay junior rates and maybe pull a decent mid-level guy, it may be a good tactic. If you are at all interested in top talent, you'll turn many if not most of them away. Unless your company offers something no other company in the area offers, tops generally have no time or patience for that, especially if they just handed you a loaded resume with tons of references.
If you're just a boring ole company like all the other companies, believe me, you're turning top people away.
at least the take home test is similiar to a real world issue. Tons of issues with it but it's better than the live code remotely (live coding on paper somehow clicks well for me)
Yeah, I’d vastly prefer that, but it should take no more than a couple of hours, not days. In fact we use a 1-hour take home to screen candidates, with great results. The assignment is pretty simple, but you can’t find a solution on the internet. We have found this to be way more effective than a phone screen (which we also do, to let the candidate also ask questions early on).
When I interviewed at Google it was 2006 and they had a reputation for rejecting nearly everyone. I was still a student! So I rocked up to the interviews calm as calm could be because I didn't believe for even a second that I'd actually get the job and really, the whole thing seemed like an entertaining and potentially interesting mistake.
I did that with the google phone screen. It didn't help that the interview really took a "well that's interesting" approach and basically killed the interview.
It sounds to me that an assumption is being made because they didnt meet the expectations within a fixed time period. The danger with this is that this is not a real world scenerio of how two people would approach a task or problem. It’s also important to recognize that everyone is different with regards to how they process information. Stress has a real biological effect on how the brain processes information, and who doesnt feel stressed during an interview.
I’m somewhat autistic and struggle to process verbal information. But I try not to share that with people because I choose not to be labeled by it. It’s unfortunate that in our society that we have to conceal things like that.
All of us could be better at trying to understand people around us and accepting them for who they are instead of scrutinizing them for what they are not.
Solving a problem within a fixed time period is the best real world scenario for programmers. No project has an unlimited budget, so no one gets unlimited time in the real world.
"Solving a problem within a fixed time period is the best real world scenario for programmers"
The funny thing is an artificial examination style setup totally freezes me up. I'm a valued contributor and write original technically non-trivial code day in and day out.
The last time I did an interview a few years ago my brain completely locked up. The interviewer was very polite, and it could have not have been any less stressful situation. Yet, somehow my brain knew I was in an interview, and it was time to lock up all that valuable technical information ... somewhere where my conscious mind could not access it.
Which is really weird. Usually at work the occasional unexpected stress put's my brain on turbo-charge, and ... it's so beautiful to experience it, everything has perfect clarity, I know what needs to be done and I just do it so much faster than regularly.
My "on-interview" brain is complete opposite of my "million-euros-are-at-a-risk" brain at work. I have no idea why.
"Plan to throw one away." This was one of the major thesis statements of The Mythical Man Month, by Fred Brooks. It's one of the hallmark texts on software development. I don't disagree with what you are saying - that we need to be cognizant of time and money - but there is a scary shift happening in software where more emphasis is put on time-to-market above all else, where we are boiling the frog of consumer expectations. Millions of people lose their personal financial information in a hack, and the entire population just shrugs their shoulders. I'm left wondering if this would be the case if we put less emphasis on continuous production deployment, and more on the care and quality of our processes and systems.
Lots of real world problems are troubleshooted in less than that time, under more pressure, and have far more variables to consider/are far more difficult, like build failures. The scenario here sounds like it's as relaxed and simple as you can get without it being so easy it tests nothing. Are you assuming the interviewer always yells like he's Sam Kinison?
> Lots of real world problems are troubleshooted in less than that time, under more pressure, and have far more variables to consider/are far more difficult, like build failures.
Yes, and they're troubleshooted by people who are already intimately familiar with the systems they're troubleshooting. Nobody in the real world writes an app for a requirement they've never seen before from scratch in 45 minutes.
> Yes, and they're troubleshooted by people who are already intimately familiar with the systems they're troubleshooting.
So you're asserting interviewees shouldn't already be familiar with simple problems like reversing a list? Come on, give me a break.
> Nobody in the real world writes an app for a requirement they've never seen before from scratch in 45 minutes.
Being asked to reverse a list in 15 minutes isn't even close to "writing an app from scratch". It's not even theoretical - just look at all the people who make it past these interviews. I doubt they would call it "writing an app from scratch". Could you try any harder to misrepresent things here?
> So you're asserting interviewees shouldn't already be familiar with simple problems like reversing a list? Come on, give me a break.
Though it is clearly a simple problem, I've not had to reverse a list in any of the code I've written in the past 20 years. So I am not 'familiar' with the problem. I think I'd be able to solve it pretty quickly under normal work circumstances (even crises), but as many have said here already, working on something you've never done before with three strangers looking over your shoulder is a whole other dimension.
Many here and elsewhere have related the 'brain lockup' effect. Take me as another data point. It's real and it totally wrecks the interview for the interviewee and the interviewers, who lose out on a potentially good employee.
> Though it is clearly a simple problem, I've not had to reverse a list in any of the code I've written in the past 20 years. So I am not 'familiar' with the problem. I think I'd be able to solve it pretty quickly under normal work circumstances (even crises), but as many have said here already, working on something you've never done before with three strangers looking over your shoulder is a whole other dimension.
What's your point? You just want to repeat what everyone else has already said?
And how low do you think the bar should be? What would be reasonable questions that actually still separate people who can code from people who can't, since reversing a list is too hard according to you?
> It's real and it totally wrecks the interview for the interviewee and the interviewers, who lose out on a potentially good employee.
Or they hire someone else who might actually be just as good or better. It's not like companies have an unlimited number of open positions.
> And how low do you think the bar should be? What would be reasonable questions that actually still separate people who can code from people who can't, since reversing a list is too hard according to you?
I'm not saying anybody should lower any bars. Just don't immediately fail somebody who struggles with a simple problem. Don't just automatically declare them incompetent. Help the candidate be the best version of themselves. It's in your best interest as an employer. Try to figure out ways to make your interview atmosphere as realistic as possible. Some of these people who choke on trivial problems may have aced the same test on a better day.
I was responding to the words I quoted from your post. I was not aware that somebody had already made the point that even apparently simple problems may not be 'familiar' to all candidates.
> It also isn't even close to "solving a real-world problem".
It's not supposed to be the same thing. It's supposed to test your coding and problem solving abilities, in the limited time you have. What better alternative can you come up with? It's easy to just complain...
> It looks to me like you are the one misrepresenting things.
Hah, yeah right. If it makes you feel better, sure.
I worked with people who were hired with "can you do the job?" test and with people who were hired by "everything is wonderful, you are a special candidate that we just talk with" test.
The second group universally cannot perform under production pressure.
No projects are budgeted to 45 minutes. The real world scenario for such limited time is debugging in production issues. It is a completely different scenario. The people who jump in are mostly those who are familiar with the services or application that are showing issues, those who have a clear understand of the architecture of the company, those who have skills on their tech stacks and know where to look at and how to analyze, etc.
> It sounds to me that an assumption is being made because they didnt meet the expectations within a fixed time period.
Of course, you have a limited amount of time to interview people. Please let me know when you figure out how to give someone an infinite amount of time to answer questions during a 1 hour interview.
> The danger with this is that this is not a real world scenerio of how two people would approach a task or problem.
That doesn't mean it's not a reasonable way to test basic skills. It's even too easy to test that by itself. What good alternative do you have?
> All of us could be better at trying to understand people around us and accepting them for who they are instead of scrutinizing them for what they are not.
How exactly would you structure a 1 hour interview so that it's not, "scrutinizing them for what they are not"? Or are you just having fun being preachy here?
I realize that you’re doing the best that you know how given the time pressures you have as an interviewer. There is no perfect answer to this problem.
Here’s the thing though, you are “leaving money on the table” when your interview process deselects great programmers because of a mistaken belief that Test A is a good proxy for developer talent. When HN developers are telling that Test A is a bad proxy for developer competence, perhaps, just perhaps, you might consider trying to improve it with the advice given.
The upside of being able to accomodate developers who “lock up” in stressful interviews, is that you now have access to great developer that are (according to your competitors) “impossible to find”.
A lot of that's true or might be true, but how do we improve things?
> you are “leaving money on the table” when your interview process deselects great programmers because of a mistaken belief that Test A is a good proxy for developer talent.
That's true for any non-perfect interview process at any job. The problem isn't that people don't know this, it's that it's hard to accurately measure who's good.
> you might consider trying to improve it with the advice given
Hah, as if the advice is something amazing! Which of these pieces of advice could we use that wouldn't make things worse in some way and also have a chance of being accepted by existing coworkers and managers? There's some great pie in the sky ideas, but nothing without serious issues.
> Pete Holiday, an Engineering Manager at CallRail in Atlanta, used to use homework as part of job interviews before realizing that he was ruling out good candidates. Some told him they didn’t have time for homework. Others may have never gotten to that point. “It’s way more inclusive to just have someone come to the office and talk to them,” Holiday said. “You’re not counting on them having time, or a computer at home. We have candidates with sick family member, single parents. Without the homework we can cast a wider net.”
So is homework worse than whiteboarding onsite? Who do we believe, some article or the crackpots here?
I wasn’t referring to the in-office vs. take-home aspect, just the “stress test” nature of the white boarding test. (And I was admittedly unclear about that in my post and I’m sorry.)
Think of ways you can de-stress the candidate while extracting useful information. Ideas:
1. Have them bring something they wrote with them and describe it in the interview. A lot of developers will “come out of their shell” when they start explaining something that they worked on.
2. Show them some code and have them explain it to you. I had a manager open a C book once and had me explain some code.
3. Pair program on a problem that the interviewer doesn’t know the answer to. This situation is a lot more like a true work situation.
One thing I would say is just talk through problems with candidates as simple as that sounds. Try to grok how they think through software problems. Nothing mind-blowing there, but what I'm saying is how someone codes fizz-buzz tells very little...or to me this is like hiring a driver by taking them out to a parking lot and doing doughnuts. If you can fathom the idea that they can drive a car (at all) it seems like it is much more enlightening to hear about how they organize a schedule, clients they have picked up, problems they have solved...how they react and comment to situations you pose to them,etc. I guess the key here is to do this conversationally, and not like you're grilling them. Or you have to be a good question asker and a good listener..but if you are, or can become one, I 100% believe you can get the information you need in a humane and efficient manner. But even as I type this, I'm wondering if the ability to read people is why this so hard for software interviewers...or maybe it's for lack of that simple skill that our industry has resorted to silly proxies like fizz buzz, etc.
> One thing I would say is just talk through problems with candidates as simple as that sounds. Try to grok how they think through software problems. Nothing mind-blowing there, but what I'm saying is how someone codes fizz-buzz tells very little...or to me this is like hiring a driver by taking them out to a parking lot and doing doughnuts.
That's what people do, or should be doing, along with the whiteboarding. I don't think anyone just has them do fizzbuzz and that's it, like you seem to be implying. That won't even fill an entire interview slot.
> it seems like it is much more enlightening to hear about how they organize a schedule...
Sure, and that's also something you ask about, but that doesn't weed out the people who can talk the talk but can't really program, who do exist.
> But even as I type this, I'm wondering if the ability to read people is why this so hard for software interviewers...or maybe it's for lack of that simple skill that our industry has resorted to silly proxies like fizz buzz, etc.
I think it's because people want real evidence of skill, which is pretty easy to detect via whiteboard in most cases, not just the interviewer's gut feeling.
The amount of times I've heard interviewers go "Oh I had this super experienced guy who but in the interview he couldn't even do some simple programming problem". To me that is a failure in the interview process. I've seen this when I've interviewed people. If people don't do well when their resume implies that they should, I'll discuss it with them and say we want to get some kind of confidence in their abilities. Then work out a way to do that. This has worked out really well in my experience. I've ended up with devs who are quite different from me, one person of note (who we employed), was much slower at solving problems but far far more methodical and exhaustive in approaching problems and often notices very subtle details (which is a good thing in firmware development!).
The only way I could see that scenario could ever occur is if the candidate outright lied on his resume. A quick solution is to check references (call the school said person graduated from, call references, call references via their company line, etc).
Sounds like the interviewer didn't bother checking references.
The majority of shops that firmware devs come from (and particularly defense) have a policy of only verifying the dates of employment and nothing else.
References would be personal references - someone with a cell number and call the college he attended; they should also verify enrollment and completion. You don't want to be calling the HR department. Also, a secret is if the reference (not HR) only verifies employment, that's a negative review. They are concerned about liability. If it was a good employee, the references have no qualms about saying so; it's the negative aspects that carry legal liability.
You need to ask for those, not just prior companies. My resume has a "References available upon request" at the bottom and I'd happily give them. Is that no longer practiced? It used to be a requirement.
In my experience, candidates who have experience in dev work but could not pass our interview tests always end up in a dev role somewhere else shortly afterwards. Even some of them go on to find much more success.
Either there are a lot of companies which employ unskilled developers who cannot even code fizzbuzz, or interviewing is not a great way to understand specific types of people.
We have to separate whether or not someone doesn't know how to do a job from whether someone doesn't know how to interview.
This is one of the biggest things I try to guard against. Some people just interview better than a lot of other people, but that doesn't mean they do anything other than interview well.
Oh totally. Fizz Buzz shouldn't take you the whole 45 minutes, so the remainder is just a conversation where I dig down in their experience on a hunt for bullshit.
I think FizzBuzz is about as hard as I would make a technical interview. Maybe some basic questions about database design; what's an index; what's an array, etc. The dig down and hunt for bullshit is the most effective way for a tech person to interview another tech person in my opinion.
This is always my approach too. Pick a few elementary questions about a technology and ask a developer how to do them. Simple stuff, like how to build an array with integers 0-5, or iterate over a map.
I agree w/ you here. And I think a lot of people have some personal experiences encountering this kind of situation. It seems baffling, right?
And yet, you have to really wonder ... what did that person do writing firmware for the space shuttle? Did they really do that? Or did they somehow lose their mind after they left and now couldn't do a simple set of modulus operations on a FizzBuzz test? How can that happen? Even reading this kind of thing makes me totally scratch my head.
Brain freeze under interview conditions? One thing that I have noticed is that taking in an interview seems to engage a different part of your mind from coding. I get quite into the flow of chatting, then have to jump into code. Its quite a context switch.
Plus the pressure of someone staring at you.
Then you feel stupid for not answering straight away.
The you get self conscious and start doubting yourself.
What was the question again?
I don't think there is anything especially unusual about it at all.
Off the top of my head the space shuttle firmware was written by a team that adherred to a very systemic process to ensure it met the spec/reduce the number of bugs, it was often cited in old software engineering papers/books in the time period as the best organization in the world, IIRC in terms of software engineering measurements like defects per lines of code. It would be important to know if someone came from that at the beginning of the space shuttle program when they were creating the standards for work, starting out or if they came on later after the process had been perfected.
I worked in a different contractor in NASA/Clear Lake at the time but even before I read those papers the organization responsible for the shuttle firmware was held in the highest regard.
I avoid doing interviews, but I've suggested fizz buzz a few times to people I've worked with because it's so simple but once outside the environment of your usual IDE seems surprisingly simple to screw up.
Whenever I've suggested it a usually more junior guy has scoffed at how trivial it is... I'm yet to then have one supply a complete and flawless solution. These are from people I work with, respect and happily rely on to be able to code. Their recognition that the problem isn't so trivial for themselves is the only thing I've got out of it - I'm no closer to understanding if it's just a poor test or not.
There first time I came across the problem was as a candidate and I screwed up the upper bound of my for loop just because I was slightly thrown by starting from 1 instead of the almost obligatory zero. Naming things, caches and off by one errors.
My worst candidate experience was only a couple of years ago when they wanted me to reason about booking cinema tickets over the phone (like it's the 90s?!) I've never booked cinema tickets and I think they thought I was joking, but they went at me for at least half an hour on what users would need to supply other than location, film title, date and maybe time, to begin the process of booking tickets. Why they didn't just tell me what attribute they were after I'll never know - eventually they just wrapped it up.
Historically, and if we're talking about someone with 30 years programming experience, this history is very relevant - programming has been a popular field for smart but antisocial/unsocialized nerds who had very few if any friends growing up and have spent much of their life alone at a computer.
For a person with that background, simply sharing a relatively small private room with a total stranger can be overwhelming. It doesn't matter what trivial test you're giving them, if they're not able to think. The test is failing to actually determine if the person can do the technical part of the job, instead it's testing how the person performs in close quarters shared with unfamiliar people.
The whole giving homework for technical vetting is a great way to get around this class of problems.
It's not like you forego the in-person interview and don't learn what you can from that. The point is, by including a take-home portion of the technical interview you can get a better sense of the individual's technical abilities.
If, in addition to what seems to be a technically competent individual, you find they don't perform well in-person, then you take that data and weigh it against where your priorities lie for the position.
There's a huge difference in knowing "technically great, performs poorly with an audience" vs. "technically poor, but we only tested them with an audience".
Furthermore, it's not uncommon for people who experience anxiety during the in-person to loosen up as they acclimate to the office environment and their peers. These people can actually be some of the best contributors, and it can be a very costly mistake to misjudge them. This personality type tends to be quite loyal and averse to changing positions, because the whole process of interviewing and acclimating to a new environment is so stressful to them.
You seem to have a chip on your shoulder in this discussion, are you having a bad day?
Not at all, I'm enjoying myself and the debate (while finding it kafkaesque that people are finding ways to defend failing fizzbuzz on an interview). How are you doing today? That might be the only legitimate point I've read in this chain so far - if you hire people too socially impaired to write fizzbuzz, they'll be less likely to job hop.
Now we're tacking on an additional aspect of the job that we didn't know about before. If the job requires a social interaction with people outside of the company then, of course, it becomes a factor. Let's not be silly in an effort to just be right and prove someone else wrong.
I know several really, really good programmers with pretty intense anxiety. Automatically ruling them out because of a poorly-formatted interview process would be a big mistake.
I mean, they're people. I'm still close friends with a few of them, and I didn't like some of them. It's no different than working with anybody with a disability or an illness (or kids, or elderly parents, or any of the million other human things that can make you a less-than-full-functionality worker bot) - you make the necessary accommodations, work around issues when they arise, and live with it. Specifically, it involved toning down a bro-y, combative work environment (which was a change that I really appreciated).
I worked with a guy who had pretty bad autism and social issues. He was a great guy to work with, incredibly smart and a really nice guy.
He wasn't going to come to any works nights out or even lunches and like you said certain accomodations have to be made, e.g. access to a private quiet room for him at certain times... Able to work from home when he had to etc.
His father had to go to his interview with him, luckily the company allowed this. We have since gone our seperate ways but wherever he is working now got a great developer. I'd happily work with him again.
You could extend your attitude to any sort of disability couldn't you. Why hire someone in a wheelchair, when there is someone who could walk up the stairs in half the time?
You can take the guy who starts frothing at the mouth when you ask him to code fizzbuzz, I’ll take the wheelchair guy who can code fizzbuzz during the interview. Deal?
In your head and the head of those with extreme anxiety it is quite different. You're meet people who might be nice or not and chat a bit, you answer some questions, maybe there's a free lunch involved, you either get a job offer or you don't and you continue your search, and your life continues on without the sky collapsing
You could say that for almost any job, but somehow you don't see the same hyperbolic whinging in all of those fields, do you? And your claim isn't even true. It's very similar to the kind of code discussion you might have on the job. "How would you solve this?" "What if I try X, Y, and Z?"
This whole thing reminds me of a shitty superhero spoof movie from the 90’s - Mystery Men - one of the characters’ superpower is that he can turn invisible when nobody else is looking and he is naked
Not if you can find others who are just as good. And what's your better suggestion for filtering out people who can't code and not filtering out the good ones with anxiety?
Take-home tests. Lower-stress environments for coding tests - working on their own hardware, working remotely, no panel interviews, no actively combative/aggressive interviews. Also, cool aspect of that: all of these options will make the interview process more appealing for everyone. And if the justification for angry/combative/aggressive interviewing is "well, that's how our culture is", I would say the company has a bigger problem to fix.
But people are already crying over take-home tests being like working for free for the company, and then there's no way to prove they aren't cheating, which is a showstopper. That by itself is definitely not a "better suggestion for filtering out people who can't code".
> all of these options will make the interview process more appealing for everyone.
No for people who can competently handle an onsite interview and don't want hours of extra work at home.
> And if the justification for angry/combative/aggressive interviewing is "well, that's how our culture is", I would say the company has a bigger problem to fix.
Maybe you being so disingenuous makes you feel better, but interviews aren't "angry/combative/aggressive" just because someone asks to see some code on a whiteboard.
I've participated in a lot of interviews on both sides of the process.
In my experience, the whole process goes a lot smoother when there's a take-home exercise. It turns the in-person interview portion into a technical discussion about a recent mini-project unencumbered by NDAs and trade secrets.
If you can't quickly determine if an applicant "cheated" on the take-home exercise through a simple discussion, then you may want to recuse yourself from performing interviews.
If an applicant can't make enough time for an appropriately scoped exercise, I think it's likely they're unqualified or they don't value the opportunity enough to make time. Those are undesirable qualities the recruiting/interview process is intended to filter out.
I think you should consider that most interviewees expect to be asked any question from their entire work history which doesn't leave much room for anything else. It's probably worth separating the problem-solving interview and the techno-archaeological trivia game into separate days.
The hiring process is a response to the fact that employers cannot trust a single word candidates say about their experience. This isn't paranoia. Literally, most candidates we see cannot code, even when they supposed have a decade or more of industry experience.
If the industry wasn't so full of lying incompetents, it wouldn't be an issue.
Others have definitely mentioned this experience. For me, I have encountered this very rarely. More common, IME, has been people who seem like they have a TON of deep experience in something but then, when pressed or inquired, only have shallow experience. So, in short, I don't run into people who claim to code but can't as often as I do people who can code at least a little bit but quickly are unable to deal with more complex technical problems, or more expansive knowledge of the language they prefer or use most.
I'm in that group you interviewed. Didn't know what perm-gen is although I've seen it in errors when java used up all memory. Next question was about Hibernate's different caching levels.
I've been rejected enough to eventually learn all these. Then finally when I got hired, questions were the very basic define static, inheritance etc.
I knew it was a matter of numbers and should not mind the devastating feeling of rejection. But damn, you can never know how it feels experiencing tons of rejection until you do.
The hiring process is a response to the fact that employers cannot trust a single word candidates say about their experience. This isn't paranoia
Here's the thing tho': my CV is honest and my track record is pretty good: some small companies and some name-brand, well respected in their industries, known for probing interviews, over about 20 years. There may be some bad actors in the global candidate pool. But seriously, asking me these questions is like tearing my CV up in front of me and calling me a liar to my face.
I am not in the market right now and hope not to be for a good while, but I dream of being asked a stupid whiteboard question, solving it, then throwing the board eraser at the interviewer's head and walking out...
I see on this board more than any other where people will agree that taking time to hire correctly is by and far more cost efficient than hiring quickly, but incorrectly. I've definitely gone through a lot of experiences now where someone looks like a rock star on paper, have a great cultural interview, and we don't give them a ton of technical exercises, only to find out that they can't hack it or, even worse, are dragging down the rest of the team. I really wanna trust people on this stuff, my instincts are to appeal to the greater good in people, but I've been burned enough to not trust it anymore.
The problem is that a lying liar will also have a resumé like yours. How can I — who have never met you, and have no-one I trust recommending you to me — differentiate between gaius the awesome developer and gaius-prime the lying liar? I've got to administer some sort of test to distinguish the two, and the likeliest sort seems to me to be one which attempts to discern whether one knows the sort of stuff gaius would know, and gaius-prime wouldn't.
Which makes asking for a resume, rather pointless. There would likely be a lot more acceptance of fizzbuzz take home tests and so on; if that was instead of writing up resumes (replace one time suck with another instead of adding a second one in). However that makes it hard to justify paying recruiters.
Given how Da Vinchi had a resume its amazing we still use them. (His was better thoigh, it had his name and address, dates he worked for people and their names and address)
>Which makes asking for a resume, rather pointless.
I wouldn't go that far. It still weeds out the honest-but-unqualified with minimal effort - it's basically a good early filter. This actually works for the candidate too - how much would it suck to 6 hours of interviews and then get told you don't have the right degree or some such?
I can tell within 10 minutes if someone is genuine or not just by talking through real, representative scenarios. If the job really involved implementing red-black trees from scratch, then and only then bring it up!
It's so funny, people who support that garbage just aren't hearing it. This forum probably has some of the best developers in the world chatting, and these guys just refuse to change their ways, regardless of how many people tell them it's a bad idea.
It's a bit rude but until senior-level-role-interviewing candidates can stop falling over on the basic questions why waste time with harder ones? The "candidate with an MS degree in CS and x years at SomeCo, can't do FizzBuzz" isn't a myth, it really happens to people giving interviews.
I've been having to give some interviews lately (though unfortunately they have to be within 'process') and my basic question is to write some code (with a computer) that outputs the endpoints of a line segment which bisects a plane between two input control points. (I give all the equations needed and the code is done within an application already set up.) It's still managed to trip up some people with experience who seemingly don't have the concept of representing a line mathematically (BS or MS in Physics, how?). We also tested it on intern candidates under shorter time constraints and a couple internal people, they did much better, I think all but one actually got the basic line done within half an hour even if edge cases remained. If we're given longer time constraints the problem and application framework scales to more interesting subjects like generating n-point voronoi diagrams and their applications, but we haven't gotten there with anyone yet...
> "candidate with an MS degree in CS and x years at SomeCo, can't do FizzBuzz" isn't a myth
I've interviewed many candidates. It happens to find people that cannot code at all.
Yet, one does not propose trivial coding challenges like FizzBuzz. Very little is gained from finding out that a senior developer can do FizzBuzz and then fails at any more complex coding. You simply go for more complex questions straight away.
If you can't tell in a ten-minute casual conversation whether or not someone is grossly lying on their resume, you probably shouldn't be an interviewer.
That's not realistic. You and I are complete strangers; we can have a talk for 10 minutes where not a single word that comes out of my mouth is true and you'd be none the wiser.
If you are a programmer, you can tell; just ask them about their previous projects. Dig into their answers and ask follow up questions. What data structure did you use to do this part? What in the database was designed poorly and why? (Every database has a design issue).
I mean if you were at a party and some guy came up to you chatting, claiming to be a senior whatever at BigTechCo, you'd start chatting him up and you could tell if he was full of BS pretty quickly.
I think so, I personally prefer the "look at this already written code, what does it do and can we find any bugs or issues?", less stressful and more collaborative, but good luck getting any non-startup size company to let you do it.
I do something reasonably similar all the time at Google for iOS interviews, few hundred lines of obj-c with some issues in them (actual issues, not typos)
As someone who does a lot of interviewing, I tone down the technical stuff as the experience and responsibility level increases. I think a lot of the annoyances you mention still make sense for an entry level position, because there are a lot of folks who actually can't write code who apply for those jobs.
For candidates with many years of experience, obviously they can and have been writing software effectively, so for me it mostly comes down to whether they could work well on a team, interact with clients successfully, and be capable of mentoring juniors.
I have a PhD in Computer Science and 10 years in the industry, and I've never not been asked to do some trivial programming task in an interview. Thanks for showing some common sense, but most interviewers are just lazy. One time I couldn't implement a skiplist, and the feedback I got was "we were concerned about your problem solving skills".
Even recently, I've gone to job interview type situations and the interviewer has literally asked me "Do you understand what Object Oriented Programming is? Can you write good quality Object Oriented code?"
This despite that my best known essay is titled "Object Oriented Programming Is An Expensive Disaster Which Must End" a fact the interviewer should know if they'd looked at my resume or spent 10 seconds looking up my name on Google. Wikipedia now lists me as one of the critics of OOP -- https://en.wikipedia.org/wiki/Object-oriented_programming
I'm lucky in that my position is comfortable, so I can laugh off stuff like that, and either educate the person I'm talking to, or end things quickly by explaining that there is probably some kind of cultural mismatch.
But it shows a remarkable laziness on the part of the people who, in theory, want to learn more about me.
I was exposed to this essay really early on in my career and it completely changed how I thought about programming. Thank you so much for writing that; I still link people to it today.
Thanks for this, I added this essay to my reading list. Breaking people out of the OOP worldview is currently one of my biggest professional challenges.
That would be easily countered with "So... I read your essay and it intrigued me. As a baseline, to understand where you're coming from... can you...?"
As it should have been. The fact that you have a PhD means nothing to Google, Facebook, and the like (except possibly a slightly higher base salary than a new grad w/ only a BS)
The question is, can you whiteboard? They will be testing that. If you can, then all is golden and you probably didn't need to waste those 4-6 years getting that PhD because what you will be working on will have nothing to do with your topic of research (unless it was AI/ML).
I didn't have to answer any hackneyed technical questions or take any tests to get a job. The most technical thing I did was explain how a data structure I wrote works, which came up naturally in a conversation about what kind of programming I have done.
If you can demonstrate that you understand programming concepts and can learn, then it's not much more to assume you can write code. If you need someone who can hit the ground running with some specific technology, then you might need to go deep into it, but not to find out if someone can program.
Google does not hire people by simply assuming the candidate can code. You do not get the benefit of the doubt, and in fact, you must prove otherwise. They want to see you write code on the whiteboard and solve the technical challenge given.
And as we all plainly know: what Google does is what we all should do. That's probably the one company that's never done something unnecessary or wasteful.
I assumed OP was referring to white board coding questions when he said "trivial programming task", which Amazon indeed does ask you to do throughout the interview process.
>One time I couldn't implement a skiplist, and the feedback I got was "we were concerned about your problem solving skills".
I think you're overthinking the feedback you got.
If it was at one of the big names, they have the luxury of being picky. They can't hire every qualified candidate, so they have to reject most of them. As the interviewer, the company policy likely dictates the interviewer give a reason. Which puts the interviewer in a bind because it implies that there was a good reason for rejecting you. So they'll put canned statements like what you got.
I recently got rejected. And one of the reasons given was something that 3 of the 5 interviewers had explicitly told me they don't care about. I recognized it for what it was - they feel obligated to give some reason, so they nitpick after the fact even though they told you they were not going to nitpick.
When I was younger, I often wondered how it was that New Senior Guy was totally incapable of actually programming, but very good at handwaving and bullshitting, and being patronizing. Now I know how those people got hired. Thanks.
When I was younger, I, too, thought I knew everything and that the senior guys were full of it. As I grew professionally, I developed a degree of humility and EQ and understood how little I actually knew.
Sure. It's a common meme. It certainly happens. Consider though, that the fact that it happens sometimes means that genuine frauds are able to trot out this meme, as you have just done, and immediately dismiss the complaints of the junior.
You might as well have said, "Well, you know, when I was a young girl I once had a crush on an older man, and when he rebuffed me I badmouthed him on Insta and all this talk of older men coercing young women in employment is probably just schoolgirl crushes, broken hearts and vindictiveness."
It's true that young people can be cocky and arrogant. But there are unscrupulous people who use this meme to maintain their position, usually to the detriment of the young people. If your interview system is unable to identify such people then you will create a hostile environment. Hopefully, your good people will leave. Unfortunately, many will stay and blame or doubt themselves. I was very lucky in two cases to be at a start-up with a strong CEO who wanted to see results (which I could produce, but which the bullshitter could not).
I was also lucky that in those two cases, the CEO correctly interpreted my emotional expression, which ran the gamut of anger, isolation, helplessness, victimhood, righteousness, denial, negotiation, not as a lunatic but as someone in a fundamentally awful position, and why.
So in one specific case I'm thinking of, the senior guy in question tanked two consecutive projects and lost a major client, while in another, the CEO eventually called him out by asking to demonstrate doing himself what he claimed he was mentoring his team on, and firing him when he failed (the guy handwaved, boasted, and prevaricated but couldn't actually do what was asked). That CEO was fucking great.
I'll use this as an opportunity to vent about agile. It promises developers a larger voice, but in my experience, it heavily favors management and product management types, since they are usually the ones in control of the ticketing/tracking systems. They aren't under the sprint deadlines that developers are, so they are able to better organize and build their cases for what gets done. Even if developers can build a case, it often gets sent to the bottom of the backlog. Developers are forced to take on technical debt due to the short-term product oriented thinking. Is this because of the design of agile, or just the misapplication? Considering it's a cargo-cult management technique, I think it is part of the design.
2 week sprint deadlines are entirely too short. say there's a larger project, it takes a lot of overhead to split everything up into neat 2-week releasable pieces. After the slicing and dicing, the big picture gets lost and garbled, adding more stress to developers.
Third, and most importantly, it offloads responsibility and ownership from management (especially upper management) onto lower level employees. Why should managers do their job if agile teams are "self organizing" and "self managing"?
On top of this, I'm not convinced product or business types are better able to design a good product than most developers are.
My favorite is when I take the time to break something management puts tasks 4, 6, and 10 into the sprint because the hours fit. Disregarding any dependencies or natural order.
I think the real problem is that we, as a society, have lost the ability to manage. Agile being associated with these problems is a symptom, not the cause. It won't fix bad developers and it won't fix bad management. But if the developers and management are good, then it can work fine.
> 4. Removal of vacation time (unlimited PTO means you have no vacation days)
That's just not universally true at all. Using my current employer as an example, they have been very good about encouraging employees to take advantage of unlimited PTO.
Managers take PTO and encourage people that haven't taken it in a while to do so. Even the CEO goes in front of the company at all hands every once in a while and tells people "I just came back from a week vacation, you guys should too, remember to take time to recharge". It's something you have to get right in the company culture early on, granted.
I personally am so spoiled by this that the thought of having a set number of vacation days gives me anxiety. I don't want to count vacation days. It's not that I take a ton of vacation, I'm not a big traveler. But I like knowing that I can go to my manager and say "I'm taking friday off cause I'm going on a road trip this weekend" and he can just say "cool, just put it on the calendar" without worrying about counting days.
That said, I know this is really dependent on the company and have been in the opposite position at a company where PTO wasn't counted but was culturally discouraged, so I get where people are coming from. But I wouldn't just flat out dismiss the idea like so many on HN do.
6 weeks is not standard in the US. I'm used to seeing 15 days of flexible pto/sick time and 6 holidays. Also, due to how the pto time would accumulate per each pay period, it can take many months to save up a whole week. I'm much happier with unlimited pto, as I now take 4-5 weeks spread out however I prefer.
I know, I was trying to add some outside perspective. Where I'm from, 5 weeks is the legal minimum, and everyone has the legal right to get 4 consecutive weeks off in the summer. Consequently, since everyone does this, and expects this, companies adjust for it, and most importantly: No-one bitches about it. There's no masochist culture around not taking vacation.
The problem with "unlimited" vacation is that it's of course not unlimited. There is a limit, it's just not written down on paper, it's not part of your employment contract. It's arbitrary. At one point, either your boss or your boss's boss is going to say no.
And if you get told no, you have no legal recourse! If you get fired for taking "too much", you can't sue for wrongful termination!
Fixed vacation time protects you as an employee. Why doesn't your company just give everyone five weeks of vacation and have that written down in the employment contract?
It's exactly like bonuses vs. salary. A bonus can always be withdrawn for any reason or no reason, but it's very difficult to lower someone's salary.
Technically it's scrum that causes that, not Agile. Scrum is a way for people to make money off Agile via consulting. Srcum is rigid, Agile is the opposite.
This discussion takes me way back - it reminds me of people in the Eastern Bloc talking about communism. A widespread meme was that communism was allegedly good "in theory", it was just the practice that somehow got derailed.
> So maybe the person shares actual code and walks us through it that they wrote, even if not open source? Again, we can't know for certain it is their own code and not some friend who wrote it for them 2 years ago that they claim is their own. Or was a teammate's, or what-have-you. So we can't count on that either.
Then you're not doing a very good walkthrough. I can generally spot someone who didn't write the code they are presenting almost immediately. Occasionally, someone might fool you, but it's pretty rare.
As one of my interviewers said many moons ago: "Yes, his claims and credentials sound overblown. But after reviewing his stuff and interviewing him, he either is exactly what he claims, or he's the best damn liar I've ever seen. Either way we should hire him--the only question is whether for engineering or marketing." :)
Our vigorous interview culture also allows us to not be as credentialist & elitist about what school you've went to. As a result, someone can get a job a google without having to have gone to a top ten comp sci school, like you would have to with law or business.
Yea I was trying to think of a way of putting this that didn't sound too trite, but I think you've nailed it. This process is decent way to combat nepotism or BS'ing your way up. There's a bit of elitism to "if you can't hack the technical interview, you can't hack the job", but this is why some of these alternate ideas have been tried like the take home exam.
I'm confused. On the one hand you say that the hiring practices are obnoxious, but then you agree that the only way to verify someone can code is to have them "actually pump out some code".
I now have two steps before an in-office interview: an online programming test, followed by a screen-share coding test. When I just did the screen-share test, I can't tell you how many people claimed to be java programmers and their IDE was eclipse, and then they didn't know how to start coding or run a program. "I know junit", ok, write a junit test and run it in eclipse: deer in headlights. Such a huge waste of my time. So now we do an online programming test, and many applicants fail to write code that compiles, let alone passes the test.
There are good applicants out there. It seems like they are vastly outnumbered by frauds. I have no other way to describe these people that cannot program, and just show up with a bullshit resume hoping for a paycheck. Before I was responsible for hiring, a man was hired on our team who could not program, despite a resume showing years of java work. Now he gets to go to the next place with our company on his resume. These frauds are hindering us finding the genuinely talented.
Let me play devil's advocate for a second. Knowing how to write code isn't binary, of course; there's a wide range of skill even among those who have contributed to large production systems. If I hire someone who seems great on paper and turns out to be a dud, then I immediately let them go. It's bad for me, and bad for them. If a 10 hour project-specific coding task reduces mis-hires by 50%+, isn't it better for both parties?
Less automated / efficient hiring practices could also contribute to higher developer salaries, which would in a roundabout way, provide some compensation for what seems like "uncompensated" time spent intensively interviewing.
> If a 10 hour project-specific coding task reduces mis-hires by 50%+, isn't it better for both parties?
Does this fully replace the onsite? If I'm fully employed, interviewing with three or four places, and expected to take a day off and visit each of them, adding another full day of work on top of each is a big hurdle.
To keep going with the devil's advocate: If one of your competitors says "ok based on your initial chat with the director, we're gonna bring you straight to onsite" and you say "do this ten hour problem" guess which I'm going to favor? A hiring manager confident in his ability (probably learned through trial and error) to sniff out bullshit is a positive signal to me, the candidate. Intense homework tells me one of two things: they have way too many candidates for the role and there isn't enough data to screen by hand (this makes sense for seasonal stuff like entry-level, not so much elsewhere IMO), or they don't really have a solid picture of how to interview. And in the latter case, that probably means if someone else on the team or in the company suggests a bad but speciously-attractive technical design, they're also going to be unable to call that out.
I cannot agree enough. I'm okay with most of the interviewing practices out there, but there has to be some kind of balance between the company's needs and mine.
One of the most mind-boggling interviewing processes I've been through went like this:
1. Write a program that satisfies the given requirements. Be sure to use the specified design pattern they explicitly requested. Include a suite of unit tests and instructions for building and running the program. Do this within one week.
2. Phone call with the team to discuss your program. Explain why you did certain things in a certain way. Discuss the design patterns you used, especially the one they requested.
3. A "coffee meeting" with some of the team members to discuss the cultural fit. They described their culture and the environment, and also talked about what I looked for in a job, my motivations and what I would do in certain hypothetical situations.
4. Full day of on-site interviews.
The reason I call it mind-boggling is because this process wasn't outlined beforehand, so that last step came as a total surprise; at that point, I had assumed they had enough information to decide. When I explained that I couldn't accommodate them within a week at least due to my current job, their proposed solution was to instead do two half-day interview rounds after normal working hours. This is where I decided to politely drop them, not so much because they wanted to make me go through an interview loop after a full day's work, but because they were going to make their employees do that, too. I was already concerned about the adversarial nature of their work culture -- anyone can nominate anyone else to be fired because "they don't belong there" -- and this just convinced me that I wouldn't fit in there.
The moral of the story is: your interview practices send a very strong message about your company. Be careful what that message is.
>If I hire someone who seems great on paper and turns out to be a dud, then I immediately let them go.
If you keep hiring duds, you shouldn't be allowed to hire people. In spite of what the headhunter firms and cookie-cutter websites would have you believe, interviewing and hiring is a skill, not a punchlist.
>If a 10 hour project-specific coding task reduces mis-hires by 50%+, isn't it better for both parties?
Only if you pay me for my ten hours. Otherwise, you've already indicated that you and your company don't value me or my time.
I've hired about 50 full-time w2 on-site developers over the last decade and maybe 3 or 4 of them were total duds (due to extreme mis-representation of technical skills) once employed. I'm not sure how this ratio compares to others but it certainly seems to be an issue that the industry talks about, so I'd be surprised if it were ultra rare.
> Only if you pay me for my ten hours. Otherwise, you've already indicated that you and your company don't value me or my time.
My favorite way to hire is short term consulting project into full time position. Not every candidate is willing or able to do this, but it has produced the highest success rate in terms of hiring signals. By the way, this underscores that it's not a money problem - hiring the wrong person is a lot more expensive than paying five candidates to produce something meaningful (and find the best one of the five).
That's pretty good, I know people who've had worse results.
Some of them were former coworkers who leaned very heavily on algorithm problems and whiteboarding because they were terrified of people who couldn't code at all (though wouldn't that be the easiest kind to let go?). But they still missed sometimes. Both on people who'd studied purely for algorithm questions and got through, despite having little practical ability; and on people who could knock out a task but had no ability to reason about its place in a larger system, so created more work for others than they contributed themselves.
I agree with your approach. As a junior developer I got my big break by agreeing to join as an hourly contractor while I proved myself, with the hope of being brought on full time should my performance be sufficient. Lucky for me, the company was not lying and they brought me on and the rest is history. We all have heard those horror stories about companies who claim to do the contract-to-hire thing but have no intention to make good on the promise, so this is a risky path for the candidate.
> Our industry's hiring practices are absolutely obnoxious. In an ideal world, you could trust someone's resume.
Working for nothing __is__ obnoxious.
Not to belittle anyone but...a plumber is licensed. An electrician is licensed. Even a hair stylist is licensed. Perhaps licensing a programmer / developer / engineer would be over doing it. (Read: Yes, it would be. But the examples do help.) That said, can't there be a formal certification(s)?
If the time from these "take home test" was invested in a more universal certification process, wouldn't that be to everyone's benefit? A recruiter with value add, if you will
The employee gets a clear picture (read: context) of where his/her skill set stands, or doesn't.
The employers get to mitigate risk and fear.
All parties gets to reduce friction and time.
Clearly, no one is happy with the status quo. That's a problem. It's also an opportunity. Hint. Hint. ;)
> Not to belittle anyone but...a plumber is licensed. An electrician is licensed. Even a hair stylist is licensed.
Shitty plumbers are licensed. Shitty electricians are licensed. Shitty hair stylists are licensed. A license says nothing about current competence. It says the person demonstrated some minimal level of competence at one point, fills out their paperwork every year, and hasn't fucked up royally enough yet to have it revoked.
Every time this subject comes up someone inevitably points to licensure as some sort of panacea. Licensure does not replace skills or competency assessments.
Fizzbuzz and other interview coding tests are being used to test minimal competence. If a license proves minimal competence it solves this particular problem. Sure, the licensed may be shitty at their job but at least they can do the job.
So because some licenses don't have ongoing re-certification requirements, that means all license are bad? Couldn't it just be that a "programming license" has regular testing and re-certification?
Because a license/certification test is the exact definition of "skills/competency assessment". If the licensing test doesn't meet those needs, that's not the fault of licenses in general, that's a fault in the test and can be fixed.
> So because some licenses don't have ongoing re-certification requirements, that means all license are bad?
That is not at all what I said. I was addressing how licensure does very little to prove competence for the three trades specified in the post.
> Couldn't it just be that a "programming license" has regular testing and re-certification?
How many licenses require recertification? Why do you think a programming one would? Continuing education requirements do not count for this, as they are mere completion measures.
> Because a license/certification test is the exact definition of "skills/competency assessment".
Yes, and this test needs to be passed once. The longer ago it was, the less it says about the person now.
Is that true? Wikipedia says (at least in the USA) you need 1200 hours of supervised and practical experience and you have to have a bachelor's degree. Following their source link, it looks like you have to apply for the licensing test 12 months before actually taking it, too.
--edit: I think you might be talking about a nutritionist (https://en.wikipedia.org/wiki/Nutritionist) which is not regulated in any way and is not a medical job. These are the people who will talk to you about essential oils and homeopathy and Herbalife.
And yet you see plenty of people that don't give any weight to certifications, and clearly just about no one sees a degree as any sort of indicator, beyond a simple line item that must exist for HR to pass the resume along.
That's 4+ years of study right there, that we did to prove that we could do the job, but no, fuck that, some of those people freeze up during an interview and a handful of those people we brought couldn't demonstrate their programming skill, so we decided we can't trust that for ANYONE.
Sorry, go invent some new way to certify yourself. How about 2-4 years of intense study and a new test, like a bar exam that lawyers take! Or just get your PhD! Just study for half your life, go waaaaay into debt, and take the exact same salaries we are offering now, because we don't reeeaaaaally want to pay you as much as we're paying now, so we're sure as hell not going to offer to pay more for that extra effort.
If you thought a day long test was too annoying, then your only alternative is spend even more time and get something we're just going to ignore again the instant we make a bad hire with someone with that license!
(For the record, if I could take a single certification test and skip all this interview bullshit for the rest of my career, I would have done that years ago. I hate going through the technical interview gauntlet, and I'm one of the good programmers, at least once I'm on the job).
You can tell if someone knows what they are doing by talking to them and having them walk you through what they have worked on.
Also, in the United States, you could fire someone easily if they were really incompetent. Considering how much time and money people spend per candidate to interview, it's probably much cheaper and more efficient to hire faster and fire faster. Some companies are spending tens of thousands of dollars per candidate they interview!
Hiring is a skill, and an overly long or drawn out process is a sign of bad hiring, not good hiring. The second part of the equation is to train people up and mentor them. Hiring is not enough. The second big part of good management is to actually get the most out of people and to help them learn and grow.
Now if it were only the tech industry that had long, ridiculous hiring processes, maybe we could say that it is necessary, but many companies and many fields have similar processes. I believe that many people have mistaken length of interview process and time spent with rigor.
You really can't. There are some people who have a skill for talking about what they've seen others do, and for memorizing what others said. If that person has sat in a post-mortem for some failure, they'll be able to tell you all the things that went wrong, how "they" fixed them, and why. And yet they can't code.
I'm not hiring people who can talk and not code. I'm hiring people who can code (and, ideally, talk). The only way to verify that they can code is to witness it happening.
Well, in that case, what you definitely want to do is optimize your entire interview process around the 1% of people who can pull this off, instead of just firing them when they can't execute on the job.
I agree with you, but in reality, a large enough company could face (and lose) a wrongful termination lawsuit if they do this too often. The burden of proof for this is surprisingly low and engineering pays well enough that many of them can afford good lawyers.
Plus there's the added risk of hiring "duds" that are also minorities. And letting too many of them go will make it look like formal discrimination. Which is a much, much more expensive mess.
And then there's this issue of finding the duds. Easy for the people on the ground to spot, but less so for management and HR, which will require some form of reasonably accurate performance evaluations.
Couple days ago I watched youtube and stumbled onto airline pilot channel. Every company does extensive tests, psychological and technical and most of time simulator flight even if you had thousands hours under your belt. They can kick you out of interview because you are not dressed well enough, like wrong tie to wrong shirt...
I think if they can talk you through some code its more relevant than if it was written by them or not. Maintenance of other peoples code is usually more difficult than understanding what you wrote yourself.
The best interview I had from a technical perspective was when I was asked to bring in my laptop with some of my code. , talk a bit about it and add a quick feature. No pressure as I knew the code and the framework. I felt like I could show off what I was good at. It would have likely caught out impostors as well.
"Maintenance of other peoples code is usually more difficult than understanding what you wrote yourself."
This is true. Often the types of things that folks look for though are nuances and style that come through with what someone writes. Do they have comments in their code? What standards are they following? Do they have test cases? How well do they handle exceptions? Etc.
The structure and accoutrement that comes along with code / work product can be quite telling.
The above being said I'd say what you did in the "best interview" was very appropriate as well. What I've generally seen out there is a "homework assignment" is often a one size fits all as an attempt to eliminate qualitative bias ...
I'm a lot more minamalist about comments, documentation, and unit tests with personal projects at home than I am at work. The stuff I work on at home are for my own enjoyment, and usually the projects are small enough and my time to work on them short enough that I don't put extra time making sure everything is ironclad. Good descriptive function and variable names and decent structure is about all you can expect with that code.
> Yet, we do. Ok, then what about open source contributions (if they have them)? Well, sure, but we don't really know, for sure, if that code is their own or if they indeed truly wrote it, can we? So we can't count on it. It helps but it can't be an automatic "this person passes the technical" signaler.
Actually "open source contributions" isn't just about having some code dumps on GitHub.
First of all you've got the stream of commits, which is very gradual and shows you how the project evolved and since when. And you also have the issues, the pull requests, the interactions with other people. Those show you how the candidate communicated and collaborated with others.
You know if the code is his own or not because it's usually all there, everything that's needed ;-)
The industry's hiring practices aren't broken. What is broken is our ego.
The reason we have to do is that you will find time and again that despite having a good resume and even interviewing well, that you will find that you've hired a person who literally has no idea what he's doing. I've seen it too many times to ignore.
I've also seen people that can ostensibly get the job done, but will write code that is so byzantine, so incredibly cryptic and opaque that no one else can understand it. I'm not talking about code that is incredibly clever, although that can be a problem, but code that is simply orders of magnitude more complex than it needs to be. From management's point of view, they've accomplished the task, at least in the short term, but to the people who have to deal with their code, it can require more work to make fixes or improvements that it would take to rewrite it all from scratch.
The real reason that it's so hard to hire a good developer, IMO, is that there are very few good developers, but a huge number of bad developers that have managed to convince their bosses that they are good, or at least competent, because there are also a not a lot of good development managers.
Can we really say that software development, as an industry, is really any better than it was back in the days of "The Mythical Man-Month"? Sure, the tools are lot more sophisticated and powerful, and the hardware has exceeded any reasonable expectations, and even imagination in some cases, but I don't think our ability to develop software has really improved all that much.
I used to work as a chef, and the problems really aren't that different. People describe their experience on a resume but knowing whether the guy will be able to do their prep efficiently etc is just impossible without seeing them work.
The solution? Sure, come in, do a shift. See how you like the kitchen and the environment. We'll just give you some cash for your time.
Interestingly this goes away at a certain level of seniority, but I always liked it as a simple way to get a feel for a potential hire.
In other professions like law or medicine there's a professional organization that requires members to take a difficult test to become certified to practice. In those fields nobody with 20 years of experience is asked if they can FizzBuzz on an interview. I wish we could get the algorithm whiteboarding out of the way one time after graduation and then have every company trust in the certification from then on and instead focus on experience and real-world skills.
> Apart from getting a candidate to actually pump out some code, even trivially, we don't really have a way to de-facto verify that the person says they can do what they say they can do, short of a personal reference from someone, say, in the company already that knows and has worked with the person in the past.
And it's enough for someone with good social skills to get a great recommendation without skills.
True, yes. Although in this case, I am referring to perhaps a software engineer (who you already know to be good since he is there and in the company and on the team) who would personally vouch for someone else's skills. At least in a case like that, I have trust that their vouching for someone else is legit.
But this tends to be rare in practice, of course, because the pool of potential people being hired is very large.
I've hired and it is astounding the number of people with CS degrees and years of experience that can barely program. Until there is some sort of professional IT association like the BAR for lawyers this type of treatment will continue.
Unfortunately there are about 1000 competing bodies trying to be this association. It seems that the tech crowd would rather in-fight than work together to be treated seriously.
And the number of coding schools that have popped up in the recent years and the fact that every econ major and their mom is trying to become a developer now isn't helping either.
>Well, sure, but we don't really know, for sure, if that code is their own or if they indeed truly wrote it, can we?
Passing off someone else's open source as yours would be a hard lie to keep under wraps, an easy lie to uncover and I've never heard of anybody doing it to score a job.
That's the answer, and lots of companies do it. If you're going to fire, fire quickly. One problem with that is the high cost of onboarding, which is (IMO) caused by HR making their own job security by pushing for long hiring processes. If you just spent 3 months searching for candidates and getting them at a desk, you don't want to spend another 3 months if that didn't work out. So you have to cut your hiring lead time to a more reasonable level first.
An informal network of trust (which in fact already exists, it's just that many companies and many engineers don't use it), i.e. personal references, referrals, open source contributions, a project portfolio.
Many participants in the game unfortunately insist on hiring and marketing by three-letter acronyms.
I have 0 stars on Github. Don't use twitter. Don't have a blog. Only a few coworkers I even talk to.
I spend my time writing code and interacting with my family, not trying to win popularity contests. Now if you're looking to hire celebrities instead of programmers, you're maybe onto something. I'm not a self-promotion artist. If I were, I would work in advertising.
Being trusted by your peers is different from trying to win shallow popularity contests.
That kind of aversion towards self-promotion might be a reason why we arrived at absurd phenomena like homework assignments for interviewing processes in the first place.
With no way to draw upon previous work results or third party trust how else is someone supposed to assess a candidate’s skills?
If people did previous work, you should look at it. There's a way to do that. Now, if you can't because of the industry's propensity for using the law to hide information, then it seems that may be the cause of the industry's inability to find relevant information about potential employees.
For the software industry ( for other industries or use cases that might certainly be different) I think it is. It’s just one possible indicator but an indicator it is.
If I come across or meet a new person from this industry and that person has a Twitter account I routinely check the follower count.
A somewhat large number says to me: “Many people seem to think that person has something relevant to say.”
More realistically, a somewhat large number (>1000) says to me "this person spends time and energy on self-promotion". Some of the smartest people I know have Twitter accounts with just 100-300 followers because they don't aggressively blog or anything.
Even if that were the case, which I don’t think it is because the underlying problem is something else, the current predominant interviewing process does little to alleviate that problem as well.
I don’t think we can magically solve inequality by trying to fix the interviewing process or by denying there are social networks.
Actually there are historical examples of where changing the interviewing process solved inequality. All it took was a curtain.
You see there where few women in orchestras and it was said that this was because they didn't have the talent so couldn't get expirence, rather than being unable to get expirence due to prejudice. Then blind auditions started, interviewers would sit in front of a curtain, candidate would come in behind and play their music. Suddenly when the interviews could only judge on performance the same women that they said didn't have the talent where the ones they were choosing and that inequality was fixed. Now there might not work if the underlying cause is something else but it has worked.
”Professor Michael Hiscox, a Harvard academic who oversaw the trial, said he was shocked by the results and has urged caution.
"We anticipated this would have a positive impact on diversity — making it more likely that female candidates and those from ethnic minorities are selected for the shortlist," he said.
"We found the opposite, that de-identifying candidates reduced the likelihood of women being selected for the shortlist."
The trial found assigning a male name to a candidate made them 3.2 per cent less likely to get a job interview.
Adding a woman's name to a CV made the candidate 2.9 per cent more likely to get a foot in the door.
"We should hit pause and be very cautious about introducing this as a way of improving diversity, as it can have the opposite effect," Professor Hiscox said.”
That's not whataboutism, that's valid criticism of a flaw in your idea.
>Should we therefore resign and stop trusting each other altogether?
This is a strawman, that was not suggested. What was suggested was to have an interview process which doesn't involve informal/subjective trust. The best programmers don't have huge twitter networks or github stars - some might, but not all. That's no criteria for filtering.
Well, perhaps. I don't want to suggest "licensing" because that's kind of a naughty word around here but man, it sure would be nice if something like that could take the extreme repetition and time wasting of "technical screens" out of the interview process completely.
Licensing probably could work in this industry but it would be a tough nut to implement. For starters, as others have mentioned, with so much change happening, re-licensing would probably be needed, and of course that's a huge cost, a pain, and overhead as well. But then again, as I think about potential solutions, I am kind of coming up dry. Really need to mull on this some more.
It can just be a general aptitude test done in several popular languages of your choice with some level of customization based on what sort of programmer you want to be.
We won't get an agreement about the scope of the certification or even what answers are right. Too much change, too much opinions, too little rigor/science.
Most programming happens in a team environment. It can be difficult to separate the impact of a good programmer from a substandard programmer in a good team.
GitHub and code samples can be useful, but not every candidate will have shared code on GitHub (in fact, most don’t) and retaining copies of code they’ve written for a previous employer is a major red flag.
The biggest issue with using prior experience is that it’s hard to compare the ability of programmers who work in different languages and different domains, it can be like comparing apples and oranges.
>It can be difficult to separate the impact of a good programmer from a substandard programmer in a good team.
I believe it's actually easier. If everyone is given part of the whole and 4 out of 5 people get theirs done properly and 1 is flailing, everyone, and I mean everyone on that team knows it.
> Apart from getting a candidate to actually pump out some code, even trivially, we don't really have a way to de-facto verify that the person says they can do what they say they can do, short of a personal reference from someone, say, in the company already that knows and has worked with the person in the past.
How do hospitals make sure that their surgeons can do surgery without having them do surgery? Do they have some magic eightball?
How do salespeople get interviewed? If one claims "I sold X million dollars worth of widgets more than the rest of the sales staff combined" I doubt you'd be able to confirm that easily.
Or what about non-technical middle management? Or HR?
If it's anything like white boarding, then they would bring in a game of operation and have the surgeons pull out all the pieces. If the buzzer goes off, well they just aren't good surgeons.
Very apt. But I'd really like to know why a medical degree means "can perform real medical work" while we're OK with a CS degree meaning "well, she has successfully completed the degree requirements, but hey, actual software development is not part of it".
However, CS is close to something else: Economics. People spend 4+ years learning about economics, but don't know shit about how it really works. They found a solution for that: business schools.
CS at a University versus CS at a technical college.
Unless things have changed drastically in the last 20+ years, CS degrees involve plenty of software development. I remember having to build a calculator in Pascal using queues in CS1. We'd have to code all the classic algorithms (Merge sorts, traveling salesman, shortest path, etc).
I much rather have homework tests than intensive, arbitrary and stress producing full day interviews.
If you are going to devote 40~50 hours a week for a company thats going to pay you 6 figures and you are not willing to work a day or two, you have your priorities crooked.
And yes, just having a resume is not enough to figure out the good from the bad ones. If we knew that perfectly , there would not be any need for any interviews of any kind.
Why not to have a 1 hour technical conversation and figure out if the person knows what you are talking about? Talk about data structures/algorithms/design patterns/OOP 10 minutes for each, plus language specific points if you want and it should be enough.
> we don't really have a way to de-facto verify that the person can do what they say they can do
We do, it's called the probationary period. 30 or 60 days, or even a bit more. If the person isn't working out, or is not a good "fit" you can say goodbye. And it goes both ways. Candidates might find that the actual work or work environment is not what was portrayed in the recruitment process. If they don't like your company, they can leave.
Scenario: you have two offers on hand. One offer is for a full time position. The other is for "let's see how the next one to two months goes and then we'll decide if you still have a job or not." Which do you pick? All things being equal, the first offer carries significantly less risk as an employee and is clearly the better choice. If you're an employer who really wants to hire engineers, why would you give an offer that will automatically filter out a large portion of qualified candidates?
If I had the choice between a full time position with 3 days free coding prior, or "let's see how the next 1-2 months go" with no free prior coding, I'd chose the latter.
But the reality is in many (most? all?) US states, employers can fire you for any reason. Sure you can nominally fire back with a wrongful termination suit, but the reality is you have little recourse. So really there is no difference in those two offers, other than one is being more explicit about their personnel practices.
What do I do when I quit my current job, move across the country, and then get fired 30 days later? I would likely never accept an offer at a company that tells me up front there is a good chance I will be let go in 30 or 60 days.
There is only a good chance if you misrepresented your abilities and aren't able to get up to speed in 30-90 days. If that is the case, you deserve to be let go.
Is it, though? There are so many other ways that someone might not be a good "fit." Maybe you found another, slightly better candidate who is willing to accept a lower offer. Maybe a manager or individual contributor feels that their position is threatened by the candidate. Maybe they don't attend weekly happy hour.
I can’t imagine firing a competent employee for any of those reasons. Hiring (and firing) an employee involves a lot of work and is a big onvestment. You don’t throw that all away to roll the dice on someone who might be incrementally better.
There are times it makes sense to do the high-risk ones: devs that are unproven, without great academic and work backgrounds, looking to get a foot in the door. Companies that are not well funded looking to find diamonds in the rough, since they can't attract people who already have proven themselves. I've been that dev myself, based just on a strong sense of how the CEO and myself were on the same page based on our conversations.
But it's definitely not a risk I'd take again from where I am now. At a more established place with people I know who vouch for the folks involved, there's a much lower chance of a new FTE getting canned within a few months for anything short of egregious behavior.
Bingo. This is how it was done before. 30 minute interview with some quick test questions (what's an array or linked list or what does ++ do), then if it sounded good, hire them. If they didn't work out, fire them. Simple and effective.
Of course I don't disagree. So true. I meant in the context of the interview / hiring process, short of hiring them on and thinking about it through the lens of a probationary period.
Are programmers more dishonest than say artists? I imagine an artist could easily fake their portfolio too (especially if they were a digital artist). Maybe the artist would be found out much easier than a programmer. I think the fact that programmers have to re-prove themselves is more a symptom of non-technical people having no earthly idea of what it is that programmers actually do. For a programmer, its the same with other professions. Programmers can't differentiate between the skill levels required to perform a protein assay in a BSL-2 facility, to operate a fermentor in a class 10 facility, to work on tissue culture vaccines. However most people can somewhat differentiate between a nurse, a doctor, and a surgeon. Seems like the more specialized your skill, the less ordinary people can appreciate your skills.
You are 100% right. In design work roles, people like to look at and review people's portfolios. They do it all the time. I suppose that maybe at least in that case, it's easier to verify that their work is their own? I don't know. I definitely do not think that programmers are any more or any less dishonest than most other professions.
Because its a good practice in this industry to get rid of sub-par employees as potato-torpedos for the comeptition to hire- by praising them in the highest register?
It's unreal that will people will bitch and moan about having to invest a couple days, maybe even a whole week in cramming/prepping/interviewing, to perform on-demand, one time, for a job that realistically will earn them $100k, $200k, $300k, $400k+ a year. Meanwhile people that work harder than us 5 days a week for the entire duration of their career will make 1/2, 1/4, 1/6th, or less of income, with none of the perks, flexibility, mobility, etc.
I mean good lord. People are asking for an objective/verifiable measure of a skill that's entirely in the domain of your supposed education and/or work experience. How much more straightforward can things get? Would you prefer that it be based on ... who you know in the company? How much you can bullshit your way through a soft skills quiz? How much buddy-buddy rapport you can establish with your interviewer?
These are objective measures on a set of borderline "well known" problems, ... I can't imagine a simpler thing to prepare for. There is no guessing. The amount of hoping someone just "likes" you is at a bare minimum. The impact of resume fluffing is minimized.
I literally can't think of anything more fair or objective.
I can't even fathom people that aren't willing to make these miniscule one-time efforts where there are literally buckets of money waiting on the other side of the gate. Money that most non-tech people will never ever dream of obtaining.
I'll take all my downvotes now, but I'm willing to karmically die on this cross. :P
Many programmers make nowhere near $100k and so many programmers do a ton of extra unpaid work; and week-long interview preparations are just one of them. Just last night, a fellow programmer I know is being forced into "mandatory unpaid overtime" for the purposes of learning new things for his company. He has to learn more crap, together with coworkers, in his own 'freetime', regularly, unpaid. And I can tell you he makes nowhere near 100k living near a large city. That may be his problem for, but if the popularity of companies having "mandatory unpaid work" AKA slavery increases we will all feel these things even more so.
That being said, I can't think of anything more viable than actually testing the programmer in question with code. I would personally give them a computer and have them do it right there on the spot in the interview but that is just me.
The problem with your friend's situation is not a bad interview process, it's that they're being taken advantage of, and for some reason not improving their own situation, in a market where a good candidate is calling the shots on the terms of their employment.
99.9% of candidates are not calling any shots in this employer’s market. Unless you are a celebrity, widely known as a top candidate, the employer is pretty much going to be dictating the interview process, the job description, the compensation, and the terms of employment.
If this were an actual employee‘s market, the friend might not be so underemployed.
Software is 100% an employees market. Everyone wants coders, and can be hard to pin down good ones. I think software engineers either don't know what they are worth, or are uncomfortable negotiating comp.
And looking for remote jobs that are not US only is fun too.
'We are totally a global company that only works remote. such awesomeness, but yeah, you have to be based in this very specific time zone though' ¯\_(ツ)_/¯
Have you ever had to do homework as part of a job application? I ask because in my experience, the companies asking for these tend to be a bit shady.
For instance, one company asked me to do this extract/load to Spark and analyze data thing. I did it because their e-mail said that they would reply to each individual with feedback. Guess what? 3 weeks later I get a canned response!
Another job assignment took me 4 days but landed me an interview that went really well and gave me a chance to talk about my code one-on-one with a real profession (I consider myself a newish developer). After the second interview, I'm told I almost definitely got the job and that the position pays a whopping $15/hour in NYC.
Right now if you offered me coding work for $15/hour, I would probably take it and be quite happy for a while...but if you put me through the wringer like that, make me explain all 500 lines of codes for 3 1/2 hours after I spent the entire work-week bootstrapping your next project, I expect you to at least negotiate around the minimum salary I wrote about on my my resume! And if you're going to flat-out reject me, that's fine but at least offer me feedback on my application or code!
I really enjoyed doing the coding exercises, it forced me to consider things I wouldn't think about when coding for myself. But I have to say, the whole thing is making me a little wary of companies that do this. I'm not saying I should have a job but a little reciprocity and honesty goes a long way.
Yes, for example, I spent two days implementing a typeahead search results system for Etsy, including writing my own ingestion pipeline, map-reduce framework, index builder, and request processor. Of course Etsy already had their own stuff, this was just a toy done in Java. It was fun. Got offered the job but didn't take it for personal reasons that cropped up concurrent w/ the interview process.
"I spent two days implementing a typeahead search results system for Etsy, including writing my own ingestion pipeline, map-reduce framework, index builder, and request processor."
Seriously? You did all of that? For a take-home interview problem?? Oy. This is why companies get away with this crap...
I'm going to say something harsh, but you really need to hear it: you failed at your number one responsibilty. It doesn't matter that you enjoyed the project or got the job. You need to maintain some level of professional self-respect in these situations. Professionals don't give away their time for free.
If Etsy asked you to do this, they were being abusive. If you did it because you enjoyed it, you signaled to them that you're desperate for the job, not very busy with other interviews, or willing to do work for very little in return. Or some combination thereof. None of these are great signals to send when you want to be treated with respect, and catastrophic signals to send when it's time to negotiate salary. Your time is not worthless.
If you're capable of developing software, being offered $15/hr /even without any interview at all/ is just broken. This is not about the interview process, that's just an old fashioned screwjob. I'm really confused by your situation. What are your qualifications and seriously what kind of compensation are you receiving right now if you are employed? If you want to talk, email me at my username at gmail.
To put things into perspective, I made $30/hr 23 YEARS AGO as a summer intern (coding) in my hometown city's IT department. In an "inland" state. Not even at a monied startup. This was before the dotcom boom, and it was a "government job", which never pay as well as private work.
I'm trying to be a fullstack developer and I'm not working at the moment. I also haven't been sending out a ton of applications (about 6 or 7 so far) because I'm enjoying the ability to work freely.
Anyways, check your inbox I'll send you some more details. I'd love to hear your take.
Alright it sounds like you're applying for higher caliber workplaces, whereas I'm operating somewhere around the junior/mid-level. My feeling is that the article rings more true the lower you are in the foodchain.
I'm guessing Etsy didn't offer you $15/hour, right? :)
People in "offshored-to countries" are making ~$15USD/hr ... if you are making this anywhere in the primary market for doing code work, you're getting massively^3 underpaid ... nevermind NYC
Frankly you have no idea about wages in the world outside of your country. $15/h is ok for dev in my country, maybe not the most experienced one but still ok. And WE are offshoring a lot of the programming jobs to countries with much worse pay.
They said it would be like that for 3 months and if they were satisfied they would make me an offer...but they would not say what the salary range was!
I almost took it just to get my foot in the door but decided I could do better with 3 months of polishing up my portfolio.
After you've been burned by these tests once or twice, you realize only the dumbest of the dumb would even consider taking such tests. Employers are so disrespectful, most times they won't even look at your submission. Buckets of money my ass. This is free work that often doesn't even get you an in-person interview. Companies that use these tests are really optimizing for the worst engineers, the most desperate, and the ones with the least common business sense and respect for their own time. If these companies could at least be trusted to even consider all submissions, maybe then it might be worth it, but having to do dozens of these tests just to get a regular interview (because no hiring process hires simply off one of these tests) is beyond ridiculous. If a company wants actual work done, they should pay for it. That's the only way to make sure the exchange is fair considering how untrustworthy and uncaring companies are these days. Writing code for a few hours or a few days just to find out the company's not even interested in reading it, is absolutely the stupidest thing I've ever done in my career, a career that has not been a stranger to stupid things at all. In the future, I will use the idea of this type of test to weed out all the people who do not have enough self respect to say no. That's the only proper use of this type of testing.
Personally, I think it's fine to expect employers to verify a candidates skills. But they should also limit their verification to the minimum amount necessary.
People are just arguing where that line should be drawn. Whether it's hours, days, or months of verification. Some careers do require months of unpaid internships in order to get jobs. And it's not because it takes months to verify a person's ability, but because companies will do anything they can to pay people as little as possible for their work.
It's wrong if the company benefits in any way from your interview/probationary period. (Probationary periods are just wrong period IMHO.) Interviewing is a huge burden to most companies, so if your interview questions are entirely synthetic, it's ONLY a cost to the company to interview you, which they should naturally want to minimize on their own. But you know what's even more expensive than interviewing? Dealing with bad hires. So I think a company is within its rights to do it what takes to ensure a quality hire. But they should not gain any benefit from this process - doing so is crossing a line - other than potentially being able to hire you, or failing that, at least make a good impression on you.
You're not refuting my point though. I agree they should do what they can to ensure they get a good hire, but I also thing they should do the minimum amount necessary to do such.
I've worked on many great teams whose interview process was little more than asking intelligent, pointed questions about previous work experience. Of course, this puts the onus on the interviewer to know what they are doing.
I personally keep a list of elementary questions about various technologies for interviewing. So when someone says they've worked with ElasticSearch or Python, I ask them how to do a few rudimentary tasks in a few of them and can pretty quickly tell how much they've been fibbing.
I've had companies both accept and reject my hiring recommendations, and every person that's been hired despite my objection has been poor quality. So I know at least anecdotally that it's a viable strategy and it only takes ~30 minutes per person.
The whole point is that these people DO make $100-$400k+. At the high range, that's $1000 for a day of work. Imagine asking someone "Hi, please give me $1000 application fee for this job." With a whiteboard interview, at least you get to talk to the engineers directly. You could argue a company that is willing to spend it's engineer's time doing hiring is spending more than you are, which is not true for a homework assignment.
If you're only applying to one job and you get it, sure. But when you're looking for a new job and sending out dozens of resumes, it adds up pretty quickly! On top of that, it makes rejections all the more dispiriting when you spend hours or days working on one job application.
One that hit me hard was a homework assignment that I absolutely aced. It was timed and I was super proud of myself for not only getting all of it completed including the bonus questions with what I considered very good quality code.
But I hadn't talked to anyone in the company prior to them giving me the homework. So they are impressed and move me on to the next stage, I talk with them and immediately find out that 60+ hour weeks are the normal for them several months out of the year. So my time on the coding exercise was wasted, a simple phone screening where they gave me details of the position would've saved me from that and I now insist on a quick chat with the hiring manager to "find out if I'm right for the role" prior to completing a non-trivial exercise.
IMHO, if you can't land, say, 1 out of 3 interviews, there is a systemic personal problem, not just a bad day, and "the problem is u" -- sadly you will just need to work harder because something in your skill set is not up to snuff.
Thanks for clarifying that your opinion was humble while insulting me. I was thinking of when I was getting my first job out of school, when I did three final interviews and landed one. The thing with the homework is that it typically comes well before the final interview and often even a phone interview. Often it would be the very first step in the process. I only had one application require a substantial (say, 4+ hours) amount of homework, but I had a lot more give 2 to 3 hour assignments.
That was annoying, but considering I was unemployed and job searching full-time, it was manageable. If it becomes more common, as seems to be happening, it'll absolutely become an issue for anyone who has to do more than a couple of job applications, which probably includes most people trying to move up in position or relocate.
My last job hunt (only a few months ago) I probably approached/applied to around 50 companies, got about 10 interviews and 1 offer. I don’t think that’s too far from most tech people’s job hunting experience in today’s environment. If you are getting 1 offer per 3 interviews, you might be “punching way below your weight class.”
This ratio depends on how ambitious you are on upgrading from your current position. Imagine arranging all open jobs into layers. At the lower layers are jobs that you can do with your eyes closed. Towards the middle are jobs you're sure you could do, but would take some ramping up. Higher up are jobs that might be more rewarding, and you think you have decent chance of doing them well if you work hard to improve your skills once you're in.
Your 1:3 odds will be different depending on your target. Maybe the commenter to whom you replied is trying to get jobs paying 50% more than their current job, but for each of these jobs there's only a 3% chance of getting it.
Fair enough. I wrote my comment assuming lateral or slightly upward movement, not a level skip, which I personally would never try to do between jobs, unless it was moving from a little fish in a big pond to a big fish in a little pond type situation.
I agreed with your main post, but landing 1 out of 3 interviews? I know several people who got offers at both Facebook and Google out of school who did not hit a 1 out of 3 ratio of offers to interviews
Ok. It was a SWAG. My point being that a systemic failure to land a coding job is a problem with you, not the interview process(es).
My sentiment in my original reply is that I hear a lot of complaining about "i was asked to code binary search" interviews, which seems to come with an implicit but unstated "...and i failed to code binary search, therefore it is bullshit" statement attached to it. Somehow I feel like if it ended with "and I coded binary search because I can generally code on-demand or because I specifically practiced it, and I got the job!" -- like there would be less complaining happening...
I mean, people don't wanna do whiteboard coding ("it's not realistic"), they don't wanna do take-home coding ("it's too much work") ... are people willing to do ANYTHING to get a dev job?
I've seen a situation where the person literally refused to do an interview, just stated their desired comp, take it or leave it. Are you fucking kidding me?
> My point being that a systemic failure to land a coding job is a problem with you, not the interview process(es).
The statistics on this speak a far different truth - one where most of the time, you're going to get rejected no matter how good you are, because you're competing with hundreds of other candidates for one position.
The homework doesn't get you the job though. You may need to go through 10 of those to find someone who wants to hire you and you also wanna work for them. Let's say you're currently employed, where are you gonna find those 10 weeks? Minuscule one-time effort? You never switch jobs?
There are lots of professions that make just as much or more as your average software developer. Accountants, sales people, bankers, lawyers, dentists, managers ... the list goes on. Software developers are middle class (or at least what's left of it).
Is the homework a strong enough signal? What if someone helped the candidate? What if they won't get along with the team or the culture? How close is it to the real work the candidate would be doing? I wouldn't base a hiring decision on homework. So this is basically just a screen, a very expensive (to the candidate) screen. Who are you screening? Anyone good enough that doesn't need to invest a week to get a job, that's for sure.
If you feel homework is a good indicator and you ask someone to do that for you, pay them. $200/hr sounds like a good starting price.
The reason the take-home assignment is unfair isn't because of a lack of objectivity. The unfairness is that it allows employers to make candidates spend proportionally more of the time, money, and effort required to generate signal from noise. Companies that are that aggressive at cost-cutting at your expense are also likely to have other strategies - say, giving the coding assignment to almost anyone and dropping the top 95% by salary requirement.
The short of it is that it's a candidate-unfriendly measure, and in my experience has been correlated with having other red flags.
Your post assumes that programmers are always applying for well paying jobs at great companies and have a great chance at any position they do this homework for. Realistically, there are 50-100 people applying for every position in the US, many of whom are qualified. So 50 devs are going to do a day of homework, doing two months of collective work? And then you have to do it again for every other job you apply for? Get outta here!
Happy to discuss this, though (and i assure without any downmodding!) I disagree with you in part. One issue is the amount of time required early in the process. A whole day of whiteboard exercisis plus prep time is a lot to ask wheon only, say, 4% of the people who go through it are offered a position. Keep in mind that unlike the bar exam or med boards, devs go through this every time they apply. This pushes a tremendous inefficiency out onto the field and almost certainly deters people fromapplyingfor jobs at all.
Which brings me to my next point- who is doing the majority of the complaining? Tech companies are constantly bemoaning the shortage of good developers. Im ok with a company not hiring me because i wont spend 40 hours on their unpaid homework assignment or exam. Seems they need to accept that just as i may not be hired, they may not do any hiring, and their own processes are the reason.
They also never mention the salary level they can't pay people at which is odd. In a shortage that's the first thing you'd want to advertise (as it would encourage more people to enter the industry) it's almost like they can't find good developers for cheap.
I think you make some really good points. But I would add 1 caveat.
I'm absolutely willing to lose a couple days for a job interview, if I have a decent chance of getting that job.
If a company is asking me to complete an 8 hour test and they have already received 100 qualified applications that have also completed that test, I should know how many people have successfully completed that test.
Giving a free 1/2 week of work for a job you have a > 10% chance of getting is very different than being asked for that kind of work for a 1% chance.
What is unreal is that these homework assignments don't get the job, they get an interview. The interview is then still the standard whiteboard coding and system design sessions. If passing take home meant getting the job, it would be more palatable. In the end, these companies suck at developing reliable evaluation instruments and it shows in how they don't trust the homework assignment to hire on its own.
Amazon, Facebook, Google? Those are hard salaries to get in my experience. I'm reasonably well respected in my field; attending conferences, speaking, leading meetups, and tirelessly researching computer science in my free time. Can't get a job at those 3, but going to try and take the hedge fund route I guess (I'm in NYC).
That's total comp. Salaries top out at what you'd expect. It's bonuses, stock grants and discounted stock purchase programs that can push you up there.
When I pass the tech interview I fail leadership, when I pass the leadership I fail tech. Kind of amusing. Amazon and Facebook are still interested in me but I'm not interested in studying algorithms for them again for a while. I like studying algorithms but am more interested in specific algorithms at the moment, not random undergraduate ones. Google has me on the 3 strikes list.
Let me put it this way, people that came to BigCo's from FinCo's in my experience (I mean, trading/quant, not banking software) generally were 90th percentile+ performers at BigCo's. Don't know about the other direction.
I'm guessing, but I'll guess that you live somewhere on the coast. I work in research and I make a very good salary in my city, but I only make ~65k/year. My roommate, with many more years of experience, makes 70k/year, and the only people at his company that make more are the senior architects and executives.
As is quite normal in this industry, both of us occasionally have to put in large amounts of unpaid overtime in order to make deadlines. That doesn't translate into more money or career fulfilment for us, because the city we live in has a very low ceiling for this sort of thing. Not everyone lives in LA, SFA, or NYC.
> You can't know for sure that dad/friend/partner didn't write the assignment either.
I don't follow what you're implying here, sorry?
> How about a 4-5 hours coding rush in the office, where you have to add a few functions or features to an existing codebase?
I work at a company where we actively keep a small set of tasks groomed for new hires. It's extremely difficult to keep a set of meaningful actually-in-the-codebase problems groomed at all times and totally infeasible that the person will spend less than a day coming up to speed on the infrastructure etc. The overhead of doing that is really prohibitive from an interview perspective.
I do think that for the most part, the coding problems need to be synthetic, then, not an active need in your codebase. I am definitely a believer in longer more open-ended coding challenges than 45 minute white board jams though.
The best option I've seen were recently-solved small problems. We would roll back a few commits, take that code into the room with the candidate, and pair-program a solution or partial solution in a fairly tightly-scoped time period, and then review the solution with another interviewer. That way, we've got an idea of what to work towards, usually still a pretty solid sense of where the problem exists in the context of the overall app, and we hit a couple of crucial skillsets - collaborating on a project, reviewing code. It wasn't perfect, but it worked reasonably well.
I think that we "benefitted" in some sense, because interviewees sometimes came up with better ways to solve some of those problems. Generally, we extended offers to those people, tho. I think it's just a slightly easier approach than trying to develop useful synthetic challenges - I haven't really seen a synthetic challenge that felt like it accurately approximated what day-to-day life is like in a company, which as an interviewee I am most interested in.
I think this is fine as long as it's a bottle problem -- something for the interview that is not of benefit to the business. If a company is asking you to do unpaid work on their existing database without paying you, it's unethical.
I think in general asking you to do a coding assignment in the office is better than asking you to do homework because there is a cost (in terms of time invested) to the company. For homework, a company can ask dozens of applicants to invest hours of their time at minimum cost to the company, even when some of bthose applicants had no chance to begin with. Here, they're forced to be choosier.
Does it have to be a day? Multiple days? I limit case work to 2 hours, on something we've already solved. Too many of these interviews apparently seek free labor and ideas.
What's really simulated by going off and building something on your own start to finish? You miss out on all the collaborative skills required to ensure you're writing the right code.
If you only care about hiring programming, motherfucker (google it) then by all means.
I agree the investment of time is worth it when you have a strong expectation of landing a high-paying job. That's one caveat to consider. Another: as long as the take-home test comes _later_ in the process, after a more traditional interview, when the company and interviewee have had a chance to evaluate each other.
There ought to be a formally-named logical fallacy for the "we shouldn't attempt to fix this problem, because someone somewhere else has it worse than we do" attitude.
1. You're forgetting that the candidate has to do this for just about every single job they're applying for. You're also forgetting that they likely have to do this while doing their normal day job.
2. You mention the money. Tell me, what is the exact dollar amount that one has to make before they have to stop complaining about this?
Look at other fields for examples. Few have to interview like devs do. Other professionals may have continuing education like conferences, but that's not the same.
Imagine nurses or docs getting asked to take vitals for an interview.
I mean, programmers also don't go through a medical school equivalent or residency. We don't take boards. I'm married to a doctor and we were together through her time in medical school and residency. If I had to choose between skill checks in the interviewing process and the 8 grueling years of hell that was med school and residency, I will take option 1 without a second of consideration.
There are developers who went through a master or PhD at a top school, with a similar difficulty and debt to medical school.
There are companies who will only hire you if you got a degree and relevant experience. The big tech companies never mention it but most of the workforce had a long education.
>>> I've never seen post-grad requirement for a dev unless it's a cutting edge research position and they need PhD's from the problem domain.
I've seen it in almost every place I worked for in London. The floor is either only people with degrees or only people with no degrees.
The degree is never announced as a requirement, it's rather selected automatically during the interview. For instance in programming, some questions about state machines and graph traversal will do a wonderful filter.
Master's and PhDs in CS are typically not focused primarily on coding skills though. These individuals have a degree and publications that speak primarily to their conceptual expertise, which is important and significant but if the job they're interested in is mostly coding, that's an entirely separate skill. I know CS PhDs who didn't write a single line of code for their thesis.
Going back to the medical analogy, just because someone has a medical degree doesn't mean they're a good surgeon. They'd need additional certification to prove their surgery skills.
Cooks do a stage after a phone / in-person interview where they work in the restaurant for a day or so (multiple days if they are expected to work with different people).
Yes. As a former cook, it's only the high end jobs that do this, not the chain restaurants. I think that is some of the problem with our industry, imagine if Applebee's started doing this because The French Laundry does? But, I think that this example is a good one, because even a chef with 20+ years has to demonstrate their abilities in an interview. Where should that line be in our industry?
People have to go through exams and other assessment before they can put the title on their CV. They've been tested multiple times on their ability to 'take vitals' and other basic parts of their job.
Anyone can call themselves a software engineer, even if no one has ever tested their ability to reverse a linked list, or solve fizz buzz, or even write 'hello world' in a language of their choice.
Many fields not only have extensive education requirements, like law school for example, but May have mandatory internships at low wages for several years.
This kind of comment is exactly the kind of thing that makes some people think that software developers have no idea just how good they have it.
I don't get it. The whole industry is complaining about not being able to hire developers, yet there seems to be some kind of competition who can come up with the most awkward hiring process.
Requiring homework is a great way to deter the kind of people companies should be most interested in hiring: those who are not actively looking for a new job.
I find this trend especially surprising considering that the U.S. has at-will employment where you can fire anyone anytime (at least that is my understanding, I'm from Germany). Why not get people at a desk quickly and evaluate them on the job?
All of the complaining is just a tantrum fit of an emotionally immature industry who, just like a spoiled child has had it very easy so far.
"Why aren't there 100x more people than I actually need (because we only hire top 1%, right?) with 100 years of working experience with this technology that I just dreamt last night that want to work for free?"
Developers have a similar attitude, despite currently being one of the most privileged segments of the workforce save perhaps investment bankers. We have a straightforward professional path that does not require massive debt and burning half of our youth doing academic busywork and entry level drudgery - like it happens in almost any other professional field with similar compensation.
The industry requirement to prove that you can produce software is not unreasonable, coding is a craft and selecting developers is like selecting painters: ability to bullshit your way through an interview is orthogonal to the skill.
I might assert that what developers have is not a privilege. A privilege is something granted out of generosity or grace by another party, perhaps even undue (and hence the grace).
What developers have is some "game", which makes them a "player". As soon as they lose their game, they cease to be players. Their relation to the game is exactly their value and nothing less. Tech employers are higher level players who know that the economy takes no excuses, and they will trim the fat, so to speak, of anyone who cannot contribute to their gamesmanship.
Generally, in most industries, seniority is respected in and of itself. But in software, you see disgraced older people who didn't play the game right, and you wonder if they are to decline away out of sight. That suggests this is not grace granted by another party. Does that look more like a privilege granted (by whom? charitable businesses?), or a loser in a cold and harsh game?
I see more question marks over the futures of software people than I see in other industries where other peers have gone. In other industries, I feel there has been more of a multi-factored consideration for the generational passing of the torch. In software, I feel everyone is always in a sink or swim test, and I think older people sink a little more.
A thousand times this. Although I still write code, and I would be comfortable being a tech lead forever, after 20 years I decided to focus my career on management for my own job security. It was either that or specialize in some hot field, but that's not where my heart is. So as a generalist I would be perpetually competing with the ever widening cohort of "senior" devs who have the SV-standard 5-years experience that represents the capping out of their signalable value.
The picture gets worse when you look at typical interview panels at hot SV companies: median-age 26, went to Stanford/MIT, interned at AmaGooFace, base their hiring process on established "rigorous" stack ranking of candidates based on their performance on fixed algo/whiteboard questions. While I can generally do pretty well at these, variance is high and it does nothing to differentiate me from smart but inexperienced people.
I think it's ridiculous that SV treats programmers like sports players with a limited shelf life. Code bases would probably be a lot better, and workplaces more pleasant to work in if experience were more valued. However I hold no illusions about the way VCs work: they require a steady influx of impressionable young blood to exploit with visions "changing the world", so it makes sense that older jaded devs don't fit into that picture. I do think there is an arbitrage opportunity for older talent though.
Spoken like a programmer with years of experience! I feel this all too acutely. My only game these days is to find small companies who are desperate for engineering experience and leadership because their in-house/contract devs aren't getting the job done.
If I went to google or FB or whatever, it would likely be a nightmare of constant sink or swim projects and unrealistic expectations.
"Privilege" is not generally understood to be granted out of generosity or grace. The dictionary definition is "a right, immunity, or benefit enjoyed only by a person beyond the advantages of most" (http://www.dictionary.com/browse/privilege) which sounds like it applies to software developers when compared to the rest of the workforce. One can enjoy a privilege even if it wasn't intentionally granted by another party. And I certainly feel privileged compared to most of my friends and family members who do not work in the software world.
I agree we have a problem with age discrimination, but I can't conclude if our problem is any better or worse than the rest of the work force.
I thought of undue grace or generosity because typically the term is used to criticize the unmeritorious for undue position -- is it not?
What does it mean if I say you are privileged? When one looks further at the definitions listed for privilege, one sees examples discussing kings and royalty for birthrights. If we're discussing whether people are unworthy, then we are on the same page.
To me, "undue grace" and "generosity" imply some actor who imparts such grace or generosity. But that's not required for privilege. Particularly when people talk about enjoying a privilege in the social-justice way (which I believe to the kind of criticism you're talking about), they're often using it to mean a lack of discrimination as opposed to granting an elevated status. For example, if someone has the expectation that they can walk through a store without getting harassed, we may say they enjoy a privilege, even though most people would agree that should be the norm - because there are people who do expect to get harassed. The reason I think it's important to remove an actor who grants privilege is that its causes are often structural and implicit, as opposed to intentional and explicit. And it's less a criticism than it is a way of pointing out that some people's default expectations for how the world works and will react to them is quite different from others.
Put another way, privilege is often used when some people tend to experience a fairer world than others. But expecting a fair world is how it should be! That's not "undue grace", as everyone should experience a fair world.
(Just so we're clear: this is a full-on semantic discussion, and I'm okay with that.)
>A privilege is something granted out of generosity or grace by another party, perhaps even undue (and hence the grace).
That's quite a narrow definition. A privilege is any kind of advantage enjoyed by virtue of belonging to a group as opposed to personal merit. It's not fundamentally imoral and may well be temporary. Certainly no one has to grant someone the privilege of beauty or intelligence.
In relation to other professions , programmers have the strong privilege of being highly in demand in the labour market. This is clearly a temporary situation, and they should absolutely make the most of their "game" towards the tech employers.
What I was underlining is that the tech job market is ballanced, and certainly more favorable towards labour than almost any other.
If you believe that merit is the focus of discussion, then we're actually on the same page and no further definition work is needed.
I saw focus on meritoriousness, undue worth, or whatever is evoked under the sense of "birthright". I'm arguing that it's gamesmanship at play, and not metaphorical birthright, and hence I speak of some of the losers of the game, as well as the higher level players who interface with the economy directly.
Very well thought out rebuttal. I live in the convergence between telecom/network engineering and code, and I definitely see the disparity that you describe.
Programmer is one of the only jobs where you have to actually show work related talent in an interview. Some jobs have strong credentials, most jobs it's bullshitting and resume.
investment bankers. We have a straightforward professional path that does not require massive debt and burning half of our youth doing academic busywork and entry level drudgery
You’ve never worked in banking I see. There’s a 3-5 year grind of doing nothing but updating Excel sheets 80-100 hours/week at entry level. Pay is decent for the 1-in-10 that make it through this stage (i.e to Associate 3 and up) but the hours are still long and the work dull.
And there’s nothing at all straightforward about guessing which language or “framework” will be fashionable next...
Also from Germany. Last time someone told me "we just can't get enought decent devs" was a company with the worst hiring process I have ever seen.
They wanted me to build an app, with backend and frontend, and design (the design had to be "beautiful"), test it, all in under a weekend while I still was working fulltime at some other job, and I told them, they knew.
It was a Deutsch company. The name is GuteFre.The worst take home test ever, of course I didn't do it.
Lol as an American living in Germany I totally empathize with you, it made me smile.
Germany also says it suffers from a lack of developers, but in my opinion what they don't say is that they need more developers to treat poorly and with a low salary.
That's the same in America. You see job posting like:
Developer position, independent contractor, 5+ years experience Erlang, COBOL, Salesforce, Active Directory, IBM Mainframe, and Java required. Starting salary: $40,000.
And they're whining that there is a great developer shortage in America but they did get some offers from overseas that claim to have those skills, if they could only get a H1B...
I don't know why you're being downvoted. This is exactly how employers abuse the H1B system.
Give an employer absolute power over an immigrant worker with no connections, support network, or other job options and their residency status on the line--what could possibly go wrong?
To be fair, such an employer usually can't get a H1B these days. Which is probably for the best because those people were probably not completely truthful when they described their tech experience.
There is also an filtering effect on job sites where good jobs get snapped up right away and disappear from the site, while delusional prospects concentrate over time, causing people to think that those sites have nothing but garbage offers.
Just that firing someone is legal does not mean it is easy. Especially if you have been working closely with the said person for months. And then there are people who are very nice human beings but their performance just isn't good enough. Firing them is hard and pretty bad for team morale.
I'd much rather do that extra work during the hiring process than have to deal with a bad hire later.
Firing someone is always hard, no doubt about it. But I disagree that it's bad for team morale to fire a nice underperformer. In fact, I've seen the opposite happen, where having to drag along a team member was a reason for people leaving.
IMHO, hiring based on homework is no guarantee that you won't end up with a bad hire anyway. It is just another imperfect data point to base your decision on, but one that comes at the cost of massively reducing your number of candidates.
Money where your mouth is; are you willing to pay for it? Would you pay the candidate a reasonable sum, certainly over a hundred Euros a day (after tax etc.) for their pre-interview homework? Given that the cost of a bad hire is orders of magnitude more than that, this seems like a really good deal for the company.
are you willing to pay for it? Would you pay the candidate a reasonable sum
This would be additional income complicating tax, and many companies restrict employees from taking on outside paid work without approval.
No matter how you slice it, these multi-hour homeworks are a stupid idea. And the real reason for them is to discourage candidates to justify getting indentured labour instead. Or to discriminate against older workers with more existing time commitments.
The homework is, in part, a personality test to see how much the company will be able to take advantage of an applicant. This may or may not not be the intent, but it is a result.
This would be additional income complicating tax, and many companies restrict employees from taking on outside paid work without approval.
It seems wrong to reject a good idea because other bad idea will interfere with it. Maybe I'm fortunate; I take extra irregular work in the sum of about 30 days a year, and at the end of the year HMRC and I settle up very simply and easily. I couldn't imagine taking a contract that said I couldn't do any other work; it's by no means uncommon in the UK for people to have extra part-time work that fits around their main job. I think we shouldn't reject a good idea - paying people if you want them to do multi-day interviews - just because other bad ideas (tax systems that can't handle earning extra cash, employers who view their staff as property) will make it awkward for some people.
couldn't imagine taking a contract that said I couldn't do any other work
It’s usually about avoiding conflicts of interest - you can generally get the box ticked easily enough to do something unrelated to your job with them. But “doing an audition at a competitor” is unlikely to be signed off!
But even so - these homework assignments are stupid, and discriminatory on factors unrelated to job performance, and no other industry does it. In fact they yield worse candidates because quality candidates don’t tolerate this kind of nonsense.
And how would this work - you take vacation to do your trial period? You quit to do a trial period knowing it’s still an interview not even an offer?
We want people who are desperate, people who will give up their precious free time to do our bidding, people who don't know when they're being asked to do something plainly ridiculous. We want people who will give up their vacation to try to please us. These are the people who will embrace poor working conditions and tie themselves to us through Stockholm syndrome.
Absolutely. As you say, spending a few hundreds bucks is way cheaper than wasting almost half a day of the candidate and our team when we bring someone in and it ends in a no. For remote candidates it makes even more sense since we end up spending thousands of dollars in travel costs when we fly them in for onsite interviews.
Honestly, keeping someone around who is under par is bad for morale because your other teammates have to make up for the difference. It is hard though, but it's business and if you aren't up for it, you probably shouldn't be in that position.
Lately most of the companies I had an interview with wanted me to be a contractor (most of the time with all the benefits of being a contractor removed, from a law standpoint this is possible), this is a common practice around here, not just out of caution, but for tax evasion as well.
There's not much I can do, it's so common a practice nowadays (literally all of my team members are in a similar construction and we are working on fixed projects from nine to six) that with my next workplace I will do the same.
When the whole industry complains about not being able to hire developers, it's not due to lack of engineering supply. It's due to lack of hiring ability. Most engineers, quite simply are terrible at interviewing, at least until someone gives some actual training and mentoring, but even that is quite rare.
Especially irritating considering I've dedicated tens of thousands of dollars and six years of my life completing a bachelor's and a master's degree in computer science where people whose entire job was to give me tests and evaluate me on them - maybe you'd like to take their word for it?
I went to a large state school for undergrad and tutored people regularly. By senior year, even the bottom performing students wouldn't have had a problem with fizzbuzz or similar screening problems. We regularly had coding problems much harder than that on tests.
Unless there were other factors involved like maybe the pressure caused by the weirdly adversarial hazing process that the tech interview has become.
Yeah, I'd really like to know what colleges these people are coming out of - if they're real colleges and not just paper diploma mills, I'd have to question their accreditation. I didn't go to MIT - I went to a tier-3 (or 4...) school and you couldn't have graduated, with any GPA no matter how low, without having produced dozens of working programming assignments, many much more difficult than any take-home work assignment.
I personally went to college with someone who could not program at all when he graduated. He made a lot of friends and had a lot of help the entire way through.
It's been 7 years and he has never worked as a developer.
>I personally went to college with someone who could not program at all when he graduated.
If you mean can't program fizzbuzz or simliar, how did he make it through tests? Did he cheat? In most classes I had, tests were around 60% of your grade.
I got a TDD fizzbuzz question once. Later learned that the interviewer told people it was the best interview he’d ever done.
But halfway through I got stuck. Something wasn’t clicking. This is stupid. I could do this in my sleep what’s going on? How much time do I have left?
So I did the only thing I could do. I wrote more tests, figured it out and got the job. Became a lead.
Conversely I’ve worked with many people who can handle trivial problem after trivial problem all day long without breaking a sweat. Give them data that isn’t arranged the way they need it though, and they crumble. Convoluted code full of redundant decisions or data rearranged in an inconsistent (or even data loss) fashion. Always had to be rewritten because we had five more features to add in the same place and you can’t build on sand.
This is true, although I've no idea how prevalent it is, outside of my own sphere. Over the years I have had several devs in my offshore team in India who had not just a Bsc, but an Msc in computing - and they are literally incapable of performing even the most basic of tasks.
That's a bunch of hooey used to justify crappy interviewing practices. Now there are people who say that have those degrees that don't, but that's verified by a simple phone call to the school.
And yet, every last freakin employer is requiring a BS in computer science. Combine that with ever longer programming assignments and stagnant wages and These are all signs of a vast talent surplus.
Very well summed up. Can confirm from the PoV of Berlin, Germany.
I've been on an interviewing spree in January and February, totaling around 20 companies. There were maybe 3 or 4 with realistic and respectful hiring process.
In the end I was left with just one company I actually liked.
Out of interest, what do you define as a 'realistic' hiring process?
I end up having to hire new devs for my offshore team in India every 4-6 months or so, and I just haven't found a good way to ensure candidates are any good.
I wonder how much of the shortage is due to there only being a relatively small portion of jobs that developers actually want.
Every time I've worked at a small company that had obvious perks (using a cool language, building a cool product), we had a bunch of applicants to pick from. We could be picky.
But the few times I worked at a company on soul-sucking work like internal legacy tools in VBScript for a non-tech company, they were the ones complaining about the great dev shortage.
They paid well but you also had to jump through hoops to see what it was which is too typical in our work culture. So if the job description doesn't compel someone to apply, GG.
I never see a shortage of people applying. I usually see a great shortage of people I want to hire.
Mid-level position, asking for 5 years of related experience? 85% of the resumes have no related experience. Fresh out of school, that is. The virtual pile gets much shorter when minimal criteria are applied.
When you say you require 5 years of related experience as a minimum bar, most applicants read we don't train or invest in our employees. It's not surprising you don't get the best candidates applying.
If I want a junior person, I place an ad for a junior person. We do quite well with those.
If I want somebody with 5 years of experience, it's because I need someone who has already got familiarity with the tools and can apply good judgement.
I have had both positions open simultaneously on occasion, and discovered the same people applying for both. Some of them were even qualified for the junior position.
I think our contention centers mainly around this word. When I see the word related I envision most standard job offers that have a laundry list of 10+ technologies, not all of them particularly related.
Software engineering is such a wide field with such a huge array of available technology options that if you're limiting to 5+ years in a specific stack you're already massively narrowing down your field to a small percentage of the available workforce. If you aren't offering significant advantages to offset that huge initial filter you're not going to get many candidates you find acceptable.
At the company I work at we hire for "general software engineering ability". You can pick whatever language or tool you want to get through the interview, we don't care. Most strong candidates will ramp up on whatever specific stack way more quickly than you expect.
If I ask for five years of related experience, I mean that if my list includes an object-oriented language with a well-known framework, I expect to see someone who has worked with an object-oriented language with a well-known framework and has perhaps dabbled in the particular one I mentioned.
Instead I get many resumes from people who have never used an object-oriented language to contribute to any software project that wasn't assigned by their professor. They haven't got five years of experience, period.
When I was fresh out of University I had to send at least 2 resumes per week to get my unemployment money. Sometimes there were no positions open that matches my criteria, so I just send it to what I could find.
Are there companies where people don't periodically move on? The culture right now is to hop between companies. It's not necessarily a bad thing. People get tired of their work and need novelty.
Yes, not crappy ones that pay well. I worked at a company where everyone automatically got a week and a half off at Christmas. This was above and beyond our normal PTO. I would have worked there forever, but we got acquired by a soul sucker. That was the first perk to go.
Yes! If the best thing an employer can come up with to say about their compensation and benefits is that they are “competitive“ this is a huge red flag for any candidate who is (or believes they are) above average.
That's probably because the rest of the job posting doesn't sound that difficult.
And you may even say you want "rock stars", completely forgetting that actual rock stars don't need 5 years of experience to do an exemplary job with a new tech stack.
Most job postings are complete BS, cut through that and the quality of candidates will likely improve.
same here, looking for a local web developer, getting people that can't spell the difference between the various css positioning. 8 in 10 couldn't pass a javascript version of foobar test and 3 in ten even got some basic stuff from the assignment wrong (like, printing 0 to 99 instead of 1 to 100)
If you're going to expect people to do real work, you need to pay them.
This is common in a field like journalism. Want to do a one or two day trial of a reporter? Pay them to write for you! Have them work in a real environment and do real work for publication.
If you're not willing to pay people for their time, and it's not an executive position, asking for days of a person's time is asinine. Also, if the homework isn't really real world, I still don't think it's of value. Making interview processes longer doesn't make them more rigorous. Only rigor makes a process more rigorous.
What I tend to do is have people do contract work for me, and if they do good work and people like working with them, I'll offer them a full-time position. Another way that companies get at this is with internships. They are a low-cost way to scout talent.
I find that a lot of companies are mistaking theater for rigor. If you ask someone to do a coding assignment, and it's not the kind of work that your company would actually do, how much value do you get out of it? Can someone work with your APIs? Are they comfortable with your languages and systems?
I have also found that longer interview processes that have more people involved introduce noise, not rigor. This is why I generally try to be involved in the entire interview process for people I am hiring.
Want to work on my team? I'll screen the resumes, I'll do the phone interviews and then I'll bring you in in person. I'll be in on all of the interviews you have with various people. It's my job to hire great people to work for me -- not recruiters, HR or random people at the company. I involve HR at the end to get a sense for a person's personality, not at the beginning.
As a manager, hiring is arguably my most important skill. A manager that can't hire is going to have trouble building a good team that works well together. So, getting the hiring process right is critical.
How do you try to hire people who currently have jobs with strict anti moonlighting policies? (Or maybe the answer is just don't bother, and you can afford shrinking the applicant pool without any big adverse effects)
Has anyone tried offering to pay but permitting candidates to decline payment? (My current and previous jobs have these restrictions, but they also pay well enough that I care quite a bit less than I did when I was a college student / funemployed / etc.)
It seems like it's likely to work similarly to bug bounties: it's not like you stop getting security reports from people who can't accept the bounties, you just don't get the benefit of the incentive, and maybe the money gets donated to a charity of their choice instead.
Not paying for the work doesn’t circumvent most moonlighting policies. The first company still likely claims ownership of your work. Enforcement of that can be tricky, but I doubt that’s a battle most people want to have.
A careful reading of most policies would show that the company claims it does own the code you write during an interview. They just don't care to enforce it because that code never really goes anywhere.
These policies are actually fairly onerous as written. Most of us just get by because the company doesn't care to enforce.
I mean, an asshole company's incentives to enforcing "You shouldn't interview at competitors" are extremely strong. I'd expect that the reason that they don't enforce "You can't write code at interviews" is not that the company doesn't care to enforce it, but that either they can't legally prevent people from interviewing with industry-standard practices (nb, I am guessing this with zero knowledge/training in employment law) or that they know that this will end poorly for them PR-wise, because people won't interview for them.
So those incentives, I think, also apply to take-home projects. Those are also about as industry-standard these days as in-person coding interviews, and so any legal or social pressures against saying that you can't write code in person should also apply to saying that you can't write code for a few hours on an evening as part of an interview.
Meanwhile, the legal situation for "you can't get paid" is very different because so many legal things are different when money is involved, and the social pressures for "you can interview via whatever means they want you to interview, you just have to refuse payment" are going to also be very different.
If someone is experienced, you can talk through code and APIs and get a good sense for them.
But if you can work with someone before you hire them full-time, you'll greatly increase your hit rate. This may not always work, and it may not even work 50% of the time, but the more people you can hire that you have worked with either through internships or contract work or from previous jobs, the better.
> But if you can work with someone before you hire them full-time, you'll greatly increase your hit rate.
Sure but you also filter out a ton of good candidates. I can't do my full time job plus family life stuff and work for someone else on a side project.
And I'm not going to resign from a job for a "maybe" contract to hire type position.
So it's just a non-starter. And most other programmers I've talked to are the same.
It's different for younger people who don't have families but that's exactly my point. You filter out whole demographics with the contract to hire style.
Those people made poor choices for signing those policies. I personally never sign non-compete agreements. I've also seen agreements where any OSS projects you work on need to get approved. I walked away from that bullshit.
You are right, but usually the people who come up with that crap aren't tech people. Most likely they are biz people who just googled NDAs and noncompetes on the internet and picked the harshest one.
> Want to work on my team? I'll screen the resumes, I'll do the phone interviews and then I'll bring you in in person. I'll be in on all of the interviews you have with various people.
FWIW, if I were one of the interviewers on OPs team, I will find this to be micromanaging and will be uncomfortable.
Also, be very sure you understand what you can and cannot say, or ask, when talking with a candidate. HR (in theory) are trained to avoid the landmines.
If I was being given a code challenge/take-home, I'd be more inclined to do it if it came with payment. Even if it's trivial, like $200. That's a small price to pay in screening and it at least shows the company believes your time is valuable.
A lot of companies will reimburse you for travel and lodging where applicable during the interview stage. It's not a far stretch to think that they should do the same if they are having you do actual work on your own time, also during the interviewing process.
> This is common in a field like journalism. Want to do a one or two day trial of a reporter? Pay them to write for you! Have them work in a real environment and do real work for publication.
Doesn't journalism also have a lot of sites/publications/companies trying to get people to work for free full time? I know quite a few pass it off as working for 'exposure'...
Exactly. Just a week or two of contract work will tell you if the candidate is a fit or not. A 30 minute phone screen will tell you if a candidate is worthy of a contract. I think companies don't do this because
a. They don't want to pay and
b. They don't get to gloat 'we have a super rigorous hiring process'
How many engineers would accept an offer which is conditional on "a week or two of contract work"?
Most of the engineers that I've hired came from another job (not all though), and so they need a confirmed offer before they are willing to jump ship.
So this practice introduces its own selection bias; you'd only be sampling from the pool of currently un-/fun-employed, and those who hate their current job enough to quit without a safe landing.
I'd vastly prefer to have a week or two of paid contract work from each candidate before making the decision, but it feels insulting to even ask.
I wonder if having multiple ways you can hire would be beneficial here. Behind Door number 1 is a confirmed job offer after 5 hours of grueling technical interviews. Behind Door number 2 is a 30 day probation offer after an hour long soft skills + tech interview, etc..
Maybe the lesson here is that the "optimal" strategy is to simply not abstract the problem. Its not "a person" you're hiring, its Bob. And Bob is more comfortable doing X or Y or Z, and Bob is best tested for his skills in ABC manner, etc.
> How many engineers would accept an offer which is conditional on "a week or two of contract work"?
Unless you work on Google or something with the same size, more than you expect to hire.
And yes, you will biasing your candidate pool. Every kind of procedure will bias the candidate pool. Again, unless your company is Google-sized, the only important thing about that bias is if it align with the preponderant bias (what is bad) or not.
> Again, unless your company is Google-sized, the only important thing about that bias is if it align with the preponderant bias (what is bad) or not.
If you mean (which is bad) then I agree completely. You want your set of biases (biases being errors in hiring, meaning the things you select on that don't match with what the job needs) to be as different as possible from the rest of the industry. Whenever you can pick up people who are good who aren't getting picked up by the rest of the industry, everyone wins.
That's what I meant. Ideally, job offers should have the largest possible diversity of biases (this is better for both sides). That's very hard to happen on practice for many reasons, one of them being that a few companies concentrate most of the offers.
If you are not one of those companies, your focus should be on making sure you are not biased the same way as them. Other means of avoiding rejecting good candidates have a much lower return.
Agreed. I guess you could make the contract route optional. I mean have the standard interview route or the contract route - ask the candidates to pick what they want.
I agree.. especially if the candidate is just looking for an "upgrade". Personally I left a very stable telco job for a 6 month contract with high chance of renewal or offered FT. The thing that made me say yes was the contract rate was ~50% more compensation than my previous job and I had contacts there. If it was anything less, I wouldn't have left.
What we have done is hired people as contractors when we have overflow work. We'll pay you a high overly rate to do extra work for us each week. They usually already have gigs and want some extra cash. Others might be doing contract work full time. But if we really like working together, we chat about making the arrangement permanent.
But I wouldn't just do this. I think any company of any size needs a strong internship program to build their own pipeline. I also feel that you should be able to hire people without them doing contract work for you or other kinds of assignments. Talking with someone, going over code and APIs, etc. is usually good enough to gauge someone. If someone has years of experience, has stuff on Git and can talk coherently about what they are doing and why, they probably can program.
What I was saying is that if you insist on seeing code first, pay them as a contractor, don't ask them to do a bogus assignment for free.
Companies don't do this because it will wildly cut down their candidate pool.
There is _literally no chance_ that I am going to leave an existing job for a "week or two" contract with a company. If that's part of their hiring process, I will simply look elsewhere.
We don't do this, but we do have people work for us on a contract basis for a period of time while they still have another job. We have a lot of people who are looking for extra cash or maybe they like to just do a bunch of contract projects. But one thing can lead to another and then you decide you both want to work full time with each other.
I agree with your general stance on CTH - but this seems a little sketch. If you're already a contractor and are used to trying to keep multiple projects going, or have some idle time, then by all means. From the candidate side its nice to sign up with eyes open to the bad as well as the good.
But if you're working someplace salaried, it seems fairly dishonest to spin up a short contract. especially when most of the attention is consumed in the process of getting started (environment, implicit policies, platform, etc).
plus on your side, you don't really know how much of that person you're getting. if the work is great, thats fine, but if it seems slow or misguided, is that a fundamental reflection of ability, or because they tried to cram it in between 8pm and 11pm every night?
No one here has a contract that forbids working for other companies in your own time. I haven't seen that in the DC area.
Maybe that's a Silicon Valley thing?
People might have clauses to prevent people from working on similar work for competitors, sure, but I haven't seen anything that bars working on other kinds of projects outside of work.
It should be one of many options. CTH is how I got my foot in the door as a junior guy long ago, but now that my career is established, I would not even consider it unless the alternative was unemployment.
I left my previous employer for a 6 month contract at ~50% increase in salary for the contracted period. If it was a 3 month contract, I would have declined.
There is an additional difficulty in many cases: the law.
Most people are not contractors with an established company/legal status as contractors, so you want to hire them. To "hire" someone formally for a week, though, entails a lot of paperwork and has, depending on the country, legal consequences (e.g. voting rights in worker councils, minimum employment durations, right to sue for wrongful termination)...
The optimal way would be if lawmakers could create some sort of "employed contractor", however I can think of a dozen ways for employers to screw over poor people with it...
>Most people are not contractors with an established company/legal status as contractors, so you want to hire them.
At least in the US, this isn't a problem. You just make out the check to the individual, and then send them a 1099 at the end of the year. No need to have any kind of established company/status, because by default anyone running a business without incorporating or organizing is operating a sole proprietorship--it's basically 1 extra form at tax time.
As long as the person you're paying is doing business under their name, they generally won't even need to open a second bank account (they might want to if they do this more than occasionally).
> You just make out the check to the individual, and then send them a 1099 at the end of the year.
Wow, that's awesome. What happens, for example, if someone is unemployed, then doing work for you and either gets sick on his own or gets injured on the job?
In Germany, for an employed person, the medical insurance would cover #1 and the Unfallversicherung (accident insurance) of the employer would cover #2. Let me guess: the (potential) employee is totally uninsured in the US during such an relationship?
>What happens, for example, if someone is unemployed, then doing work for you and either gets sick on his own or gets injured on the job?
For the first case--nothing at all.
For the second case--generally nothing as long as the person is really an independent contractor.
In the US there are guidelines that determine whether someone is an employee or a contractor. For example is the person flexible to work whenever they want, or do they have to be in the office from 9-5. Does the person have to wear a uniform etc. Is the position short term etc...
So if someone were injured on the job, they could claim that they even though the company said they were an independent contractor, they were actually an employee because of xyz.
Most employers know this so they take steps to ensure independent contractors actually fit the guidelines for independent contractor. And in the case of paying a tech contractor for a few weeks, it's usually pretty clear cut.
The IRS has guidelines on who is considered an employee. If you set their hours, provide equipment for them to use and there's a few other things I forget, they are a de-facto employee regardless of how you pay them: 1099 or W2. This is why a lot of larger companies will not employ sole-proprietors directly. They require you to either go through a contracting company or set up some corporate structure so that you can't claim to be one of their employees.
It's fairly easy. You just have to follow the guidelines. They are also guidelines not hard and fast rules. If you ask a contractor to come in from 9-5 for a week because that's when you're company is open, but it's a short term engagement, you don't buy them a computer, you don't prevent them from working for other companies, you don't offer insurance or vacation, you don't reimburse them for business expenses, and you have a written contract, you're not likely to run into problems.
>This is why a lot of larger companies will not employ sole-proprietors directly.
There are plenty of large companies that do contract directly. For example, Coke does it all the time.
I've see quite a few comments on the workplace stack overflow where a 25-35% premium for contractors was considered generous and considered "normal" for the USA instead of the 200% / 300% id expect in say the UK
Perhaps this is different outside the US, but this is not really a big problem inside the US from my experience. If you're doing "contract-to-hire" you do need to be somewhat careful that you don't treat them as employees before they are, e.g. don't give them equipment (laptop etc), don't require them to arrive at a certain time, probably don't buy them lunch, etc. IANAL (this is not legal advice), but it seems quite easy to keep a software engineer correctly classified.
It's not like the job "Uber driver" where the fundamental nature of the job means you can't be a contractor.
Most people are not contractors with an established company/legal status as contractors
Is this problematic? Here in Portugal I just update my status in the tax authority website, then declare the extra income when filing my taxes (plus possibly pay social security for that month). You don't need to create a company or anything like that.
As far as I know this isn't a problem in India, UK and US. Reporting additional income for tax purposes is fairly simple and there is nothing legally wrong in doing multiple jobs at the same time, unless you are on some kind of Visa which prohibits that kind of employment.
I agree it's the best way to tell if the candidate is a fit. But, I think resume + phone screen (or 2) + an on-site can be almost as good, and not limit your talent pool. The on-site could even include working with the team. What you describe would require me to take PTO from work... to work. Sure I'm getting paid, but I'd tell you no thanks. Others without decent PTO and full-time jobs wouldn't even have that choice
Contract before hire is a practice that is slowly growing, and not fast enough.
I find test paid projects are the best interview technique for detail oriented work, whether it's software or otherwise.
It's far better to select a task that is simple enough for the potential candidate to be a pinball through the code base or problem domain, ideally to solve a small problem that will be directly transferrable into the project they will work on.
I also make myself available to answer any basic questions or assumptions to save them time, because that would be the case on a normal team.
This results in a far lower expenditure of time trying out different resources to find relationships that work.
Unfortunately a lot of technical hiring is still done by non-technical people, or tech folks need to level up their hiring skills.
I, for one, hope it doesn't really grow. There's a lot of abuse in the "contract-to-hire" space, where the company tends to dangle that carrot of getting the job at the end of the contract period to get the applicant to work lots of extra hours at lower pay, only to be told at the end, "nope".
I wouldn't condone a practice like you've described. It's far better to engage a potential candidate at normal rates for a short period to see if it's a mutual fit.
This literally happened to me last week. On a Friday night before my girlfriend's birthday they sent me -- without warning -- an assignment that "should take about 8 hours". I told them to give the job to someone hungrier than me which was the politest way of telling someone to stuff it I could think of.
The company in question advertises here on HN's "Who's hiring?" threads, but the role was brought to my attention by a recruiter.
Here's a little tip: If you expect to take more than 2 - 3 hours of someone's time, you should be paying them. Contract-to-hire and trial contracts are very common in the industry and most people have no problem with them. If you require a significant time commitment to interview someone then you should be respecting their time. I don't want to work for any company that doesn't respect my time.
I had the same experience recently with first-time founders on HN "Who's Hiring?" (Fat Lama). They expected 4 rounds of interviews and a tech test. Delusional.
> If you expect to take more than 2 - 3 hours of someone's time, you should be paying them.
Do you want to work with team mates who have been vetted over less than 2 hours? I've worked with people who couldn't code, it's not a fun experience.
There are a lot of people who can talk a good talk but can't actually code. This was the reason for the FizzBuzz test (which I dislike, but it was invented to solve a real problem).
My current employer would not be able to do contract-to-hire or a trial contract for legal and policy reasons, and I strongly believe that a homework assignment taking approximately 2 hours, when combined with another 1.5 hours in person provides the best signal of all the methods I've tried. That's a total interview process of 3.5 hours i.e. about half a day. If you can find half a day for an in person interview then you can certainly find the same hours split up differently for homework + in person.
Trying to make it about sexism is crazy. Homework is much more flexible as to when you do it than in person interviews. Just because you're looking after kids doesn't mean you won't be able to take a couple of hours when they're asleep or in school. If doing homework is a problem, then I'm sure doing an inperson is even harder.
Our homework + inperson extension is on a topic related to our real field, not just a compsci quiz (which means it gives you a chance to show off domain knowledge if you want the extra points), allows no frameworks (except testing), produces an interesting artefact at the end, and can be completed in less than 100 lines of functional code plus a smattering of test code (although many will do more, sometimes crazily so - for which they lose marks). There are exhortations in the instructions to keep it simple. You can schedule when you want to work on it yourself, work on it with your own tools and hardware, split the time over a couple of weeks if it's difficult to get a block of time together.
The only real question is the time commitment. I've targeted 2 hours, but many spend around 3 or so, and some spend more on it. Setting an 8 hour challenge for my kind of work woud be entirely unfair to the candidate, unless that was the total time spent on the entire interview process.
I've hired with a number of different processes, and this one takes the least total time for the candidate and seems to have given me the best results.
This is the format at almost every single company these days.
1. Initial phone screen with recuriter - 30 mins
2. Phone screen(s) - 1-4 hr
3. (optional) phone live-coding on coderpad - 1 hr
4. Take home assignment - 4 - 8 hrs
5. Full Day onsite interview - 2-3 days if traveling.
Rinse and Repeat for atleast ten interviews minimum you are looking at almost 1 month of pure interview time. Add on practise time for algorithms and puzzles.
This is fed by a weird delusion amidst tech people that all their problems (including hiring employees) are somehow incredibly unique, and that they occupy some special magical world that nobody else has ever dealt with before, and they're so smart for figuring it out. But the fact is they just suck at hiring people, and they should look around and think about how everyone else does it. There are plenty of fields where hiring the right person is serious business, and they seem to get on just fine with considerably less bullshit than this.
Part of the interview process is selling the organization and its people. Maybe companies prioritize phone screenings and interviews first to steer the process and educate the candidate about the company and the position. Maybe candidates would be less likely to try the coding project if they hadn't built some rapport with an employee.
> They should always do the take home test first or after a short phone call.
Absolutely not. If most applicants need to do 4-8 hours worth of homework, they'll never have the time to review the results for every applicant. And making me do 4-8 hours of work without getting it reviewed thoroughly and getting personal feedback is not valuing my time.
In the process above there already was a screen sharing coding test. Adding the homework on top is redundant. You should be able to tell if someone can code in one hour of screen sharing, but you can't really tell if someone is mediocre or top notch from a single homework assignment.
Only once have I been given a homework from an interview and it was a solid 10-20 hours worth of work. I refused to do the work and I got the offer anyway, but I turned it down in favor of another offer.
But you didn't say how long they gave you to do it. Could you hand in the assignment a week later?
I've never been given a homework assignment for a job interview, but I would love it. I absolutely hate the tests they give you on the spot, the last one I did was awful and made me angry! I would much prefer an 8 hour homework assignment. That pressure would be fun, and I get to work with my tools in my comfort zone.
I spend 8 hours playing video games (over a few days), so putting that aside for a challenge with possible good job at the end seems okay to me.
They wanted it done by Monday. Then I got to choose which day of my weekend I wanted to sacrifice: I chose none and I'll begin my new position at a company that respects my time Monday.
To be clear: I've been given take-home "challenges" that take 2 - 3 hours each. Including at the company I wound up going with. I have no problem with these. If you expect me to sacrifice a full day of my time then you can pay me or find someone more desperate than I am.
8 hours really is a lot of time to work for free. They are basically expecting you to dedicate some of your spare time to doing work for them.
To me, this means the company does not value my own time, and it might even be a red flag which indicates unhealthy expectations about your work-life balance.
If they had an assignment which would take about an hour, maybe two, fine. But any more than that and I think you should be paid for that.
> I've never been given a homework assignment for a job interview, but I would love it [...] I spend 8 hours playing video games (over a few days)
The problem becomes, but then when would play videogames?
It's fine to be eager for an occasional puzzle challenge, but don't forget about the opportunity cost. They want 8 hours of your videogame time for something that likely has a low return on that time investment (ie, the homework assignment is [not] productive code; there's no guarantee you get the job as that homework assignment is only input in the process, etc).
That sort of opportunity cost favors the young (with no or small families, and a stronger ability to live on fewer hours of sleep), the eager (passionate for that particular job), and generally those with fewer work/life balance concerns (what if you need those 8 hours to stay healthy or protect your family?).
If you are already employed, you risk burning out by trying to burn the candle at both ends and do unpaid work in your off-hours. There's a possible opportunity cost that your day job's productivity suffers and you potentially lose your job.
Even when unemployed there are opportunity costs at play (if two companies need 8 hours in the next 12, which one do you turn down). The old adage is that unemployment is a full-time job (hunting for prospects, networking, talking to recruiters, dealing with unemployment wage systems, etc), and is stressful enough without adding additional unpaid labor on top of it.
I've done interview assignments a few times. In every instance I made it clear I needed a 2-3 weeks to complete the assignment (which was < 8 hours of work) due to my schedule. They never had a problem, and the interviews advanced to the next stage after they reviewed my submission.
I think most people are ok spreading 8 hours of work over a couple weekends. Paid shows the company is more respectful of your time, but in the end if they're taking up much of your time they should absolutely be flexible, or they will definitely pick-up a selection bias in their candidates.
I've gotten a couple of 24 hour tests - they were released electronically so you are timed from when you choose to start to when you turn it in and both times was graded on how much time it took you - more than 24 hours and you failed. One of them had two puzzle questions and I understood how to one of them in an hour or so but didn't figure out the trick for the other one until I took a shower a couple of days later but since I could only do one of them I immediately just told them I would pass on it.
I'll take a 3-day take-home project over a whiteboard interview or live coding session any day. Working at my pace with my tools in my space? Yes please.
The article makes two good points re: bias and lack of research. To the first I think it's up to us as interviewees to mention this in all of our calls. "Do you have a non-take-home option? For some groups this could be an exceptional burden."
To the second there's no real answer other than a lot of companies need to try it out to see how they fare and then when they have a basic idea of logistics that work run a study on efficacy / predictive power.
Came here to say almost exactly this. I can't speak to the bias issue. I do know that spending three days doing a little chat app last time around landed me one of the best gigs I've had. The process was painless, I would have spent that time coding and refreshing skills anyway, and the project was actually fun. I also know 100% that if the company I work for encountered a candidate who needed more time to work on the project due to child care or other responsibilities that would be quickly and happily accommodated. I wouldn't invest this kind of time as a first step, obviously, and would be turned off by any company that expected me to. I undertook this project after two 30 minute conversations had convinced both parties that there was a fit, and the project was a next step.
>I undertook this project after two 30 minute conversations had convinced both parties that there was a fit, and the project was a next step.
In this case, I don't see it as unreasonable, because you didn't have to do the project just to have a shot at someone talking to you in person.
I would never do a homework project as the first step in an application process. I would do one as the last step, after I had learned enough about the job and the company to decide that I really wanted it, and they had narrowed down the field to me and a handful of other candidates.
> I would never do a homework project as the first step in an application process. I would do one as the last step, after I had learned enough about the job and the company to decide that I really wanted it, and they had narrowed down the field to me and a handful of other candidates.
When I was job hunting last year, the steps typically went like this.
1) 30 min phone call
2) Homework
3) Algorithm Exam
4) Onsite
I typically got rejected at #3 which means the homework meant nothing.
>I typically got rejected at #3 which means the homework meant nothing.
It got you to step #3, but if I were applying to that company, I would not have done the homework. When they told me of the homework assignment, I would have asked, "assuming I do it, what comes next?" And when they replied "an algorithm exam, followed by an onsite interview", my reply would have been, "homework is the last step, or no thank you."
Then again, I have almost two decades of experience and a well-established track record of accomplishments. And I currently have a solid, well-paying job. If I were fresh out of school, trying to land my first programming job, I would probably be more flexible.
In disclosure, all the companies were upfront about the full interview process. That said, in this job market, there isn't much leverage to say no to a prospective job or to make demands of the interview process unless you're already a rockstar.
>That said, in this job market, there isn't much leverage to say no to a prospective job or to make demands of the interview process unless you're already a rockstar.
I don't consider myself a rockstar, or even a prima donna type who would make "demands" of the interview process. But I'm confident I could find a job fairly quickly without jumping through ridiculous hoops just to get an initial response from a company. From my perspective, the software developer market is still a seller's market.
YMMV, of course. Maybe you're in an area where there are more qualified developers than available jobs, and employers have the upper hand to the extent they can pull off bullshit like this.
This is a great point that gets lost because of the widely held “shortage of engineers” myth.
The hiring market is strongly an employer’s market. There is far more talent out there seeking relatively few jobs, so employers can dictate terms pretty much absolutely. If they require a 5 day homework, then it’s “homework or GTFO”. If they require whiteboard hazing, it’s “whiteboard or GTFO”. The last real employee’s-market we had was in the late 90’s!
> The hiring market is strongly an employer’s market. There is far more talent out there seeking relatively few jobs, so employers can dictate terms pretty much absolutely.
Wow, are you in the U.S.? This is pretty much the exact opposite of my experience and perception. I'm sure it must vary somewhat based on your specialty and what kind of work you do.
Depends on where you live and how much you ask for.
There are entire deserts where there isn't a tech company in a hundred miles around.
Even in the most active tech hub, it may be difficult to find a new job, assuming you care about things like work hours, health coverage or a moderate commute.
Even in areas seen as having many employment opportunities, the employers can be very picky, and their criteria are often arbitrary and opaque. I think even on the Bay Area, companies are more picky about who they hire than candidates are picky about what company they join.
Companies usually want people who are cheap, malleable and competent enough. That's a safe profile.
I couldn't agree more on companies being picky. My wife interviewed for a company last year, they said that they interviewed more than a hundred candidate so far in the year and they didn't hire anyone.
It's fair to say that they are not recruiting for real.
I have no issues with doing homework as a filter, but a homework plus an algorithm exam for filtering is annoying, especially since the homework is more related to on-job performance and the algorithm exam isn't. (and in my experience, it's never a weighted average of doing well on the HW and doing less-well on the algorithms: it's always pass/fail).
I'm rethinking what I did on my last hiring position. I sent a take-home test that should take about 2-3 hours to those who sent in a decent resume. It filtered out a little over half of the applicants I responded to, as in half didn't respond. It further filtered out a small segment who didn't do very well on the test. A filter is definitely needed to weed out those who are simply casting a wide net, are looking for contract work instead of a FTE and to those whose skills simply do not match what they've purported in their resume.
I really can't call everyone who has a decent resume, its way too time consuming. If we had a recruiter then maybe it would be worth it, but not personally. Maybe a take-home quiz that takes like 20-30 minutes would be a good filter instead?
>Wouldn't it be funny if the half that didn't respond were the best candidates you heard from, and you threw them away with your take home thing?
In my estimation, it's not likely that all of the ones who didn't respond were better than the ones who did, but it's likely that a sizeable portion of them were. They're not going to jump through ridiculous hoops because they don't have to.
One time an employer of mine wanted newly hired developers to sign a crazy onerous contract with unbelievably broad and vague non-compete provisions and IP provisions. I told him that the only people who would sign it were people who were desperate for employment, or people without integrity who would sign it without any intent of abiding by it. Neither of which were the types we were looking for. So he relented and we ditched it.
It's the opposite effect IMO. A good test is more likely to keep the better candidates and only reject the poor candidates.
The exercise is sent upfront. A good candidate can estimate that it's a reasonable program doable in an hour. It's up to him to decide whether he wanna continue the application or not. Either way, both parties win.
A bad candidate who can't code cannot return anything. Little risk of false positive.
I think the only important thing is to keep the exercise short enough, one or two hours top. It doesn't exclude people who have a family or other obligations.
That's almost like an exercise in economics. There's a middle ground of what the buyer (i.e. the employer) wants and what the seller (i.e. the applicant) wants. Buyer wants all sorts of qualities + a low salary + an NDA and non-compete, seller wants a high salary + a good work place + no NDA or non-compete contracts. Somewhere in the middle both buyer and seller should be satisfied, if one can live without the NDA contract then its not needed in the negotiation. If its a non-starter then the applicant can look elsewhere.
For sure, its something I've thought about too, but at the end of the day you need to do something to get from the many down to the one. Gut intuition gets used at some level and I prefer that to be way down the line after we've gathered a lot of data about a candidate. Take home tests also serve as a standardization measurement. Everyone does the same test so we can at least compare candidates to each other on that level. If they don't do the test then its pretty tough to compare them.
At some point you need to ask the applicant to do something to prove their worth and interest in the company. Take home tests are only one filter for that and I'm honestly open to others.
>I really can't call everyone who has a decent resume, its way too time consuming.
Even if you have, say, 100 resumes you consider "decent" (which to me would be an impossibly high number) then you need to be able to rank them, so you can pick, say, the top 10, and invest some followup time in those.
Totally, by the time we have a phone interview I'm pretty seriously considering that candidate and I want to get to know their personality better to see if they're a good fit, so whatever I can do to shrink that pool in a way that's fair to everyone is necessary.
I love trying to square these conversations about needing to shrink the candidate pool to something manageable with the widely claimed “shortage of engineers”.
That's easy, there is a vast chasm between people who say they can do things and people who actually can do things. This is precisely why these filters exist!
>That's easy, there is a vast chasm between people who say they can do things and people who actually can do things. This is precisely why these filters exist!
Amen. There is no shortage of applicants. But there is a shortage of competent, qualified ones. In fact, I find it frightening the degree to which blatantly incompetent people can skate by in our industry. I recently interviewed someone who rated themselves as a "professional level" Java programmer. (The highest level he could have rated himself is Expert, one above professional level.) The first technical question I asked was, "OK, please write a Hello World program in Java for me.". He totally bombed. It was painful to witness. He had no clue what he was doing, but yet, there he was, asking for a six-figure salary and deeming himself worthy of it.
Supply and demand. Whether there is an actual shortage of a good depends on price.
The extremes:
Is there a shortage of talented engineers willing to work for minimum wage? Of course!
Is there a shortage of talented engineers at a price of, say, $5M a year? I guarantee not.
So there exists a market wage somewhere in between at which supply of talented engineers must equal demand. If you think there is a shortage of real talent you might not be at market rate for that talent.
Using your example, there exists a market rate for an engineer who doesn’t know how to program Hello World. You might be surprised how much that is!
Agree. I'm going to spend days of unpaid time 'cramming' for whatever random whiteboard questions I think they might ask me anyway. I'd rather put the time into showing how the work I actually do might be useful to their team. Makes a lot more sense from both sides IMO.
This is a job interview, not a college exam; if you have to "cram", you're doing it wrong. (If they are interviewing you in such a way that "cramming" would make a difference, they're doing it wrong, and why would you want to work there?)
A lot of whiteboard interviews require cramming. You never use any of the 'skills' these whiteboard interviews ask for in your day job at all. As somebody with decades of experience, your best bet is to get a "cracking the interview" book and run through all the practice test questions. After all, it is basically a little exam like back in school (because apparently that is who these people want to hire.... fresh grads who took their algo class a few quarters ago....)
As somebody with decades of experience myself, I have never experienced anything like you describe. The "skills" involved in whiteboard interviews I've participated in generally consist of "writing code", "thinking through a problem", and "communicating about your process". A more intense interview might also involve skills like "understanding computational complexity" and "making reasonable order-of-magnitude estimates". It is difficult for me to imagine how someone could work as a software developer without using these skills regularly.
Then again, I never took any "algo class" or any other CS classes, so what do I know. I just think people fret way too much about getting all the details right in coding interviews, because in my experience (both interviewing and being interviewed) that just isn't the point.
Problem with take home projects is that there is no commitment from the interviewers side. Candidate wastes time coding up the solution but interviewers does not have any obligation to even look at the solution.
I wasted a week coding up a ML paper but never heard back from the interviewer. At least in white board interview they spend time with you with real time feedback.
What paper? I'm always interested to see implementations and this is something I expect to be doing in the near future. Maybe I could read it and ask you some questions about your process?
My life is separated. I have a computer filled with dev tools at work but my home computer is a media center plugged on the TV. I do have a laptop for emergency work but that computer is provided by my employer.
I wouldn't do the job interview on that computer.
Whenever some company tells me to do homework as part of their hiring process I simply run away. What this tells me is that they expect their employees to do unpaid work at home.
If you're interviewing with 10 companies, that would mean a solid month of work. I'd much rather spend 15-30 minutes on the whiteboard per company.
But the whiteboard test should only be a smoke test for can/can-not program (ie. test for impostors and frauds), not an assessment of aptitude or in-depth knowledge about algorithms. 10-15 lines of code or so, "Guess the number", "FizzBuzz", "Count zero bits in 32 bit int", etc.
> Working at my pace with my tools in my space? Yes please.
I've seen varying schools of thought where a 3 day time limit implies you're expected to spend full workdays (3*8 = 24 hours) on it, which is less at your own pace.
I'm still annoyed at a 2-day assignment I was given which could not be done in less than 16 man hours.
I don't want to name names but one YC company I interviewed at left me with the distinct impression that they talked a big game, just to get me to work on one of their (apparently) "strictly-hiring-related toy projects" as homework. Homework, "sure", except it: 1) integrated with a real system, 2) mirrored something they actually did, and 3) they would only agree to proceed if I signed an IP/work contract and accepted their (sorry, too small) payment of USD450 for the (not insignificant) project.
I pushed back at the contract, and asked them to ask their lawyer if that was really necessary. When they insisted on a contract for the homework project I pulled out of the process. It just felt so wrong.
The worst part, is not that they would seek some way to freelance a small part of their system, nor that they would actually use ideas from hiring interviews in their product -- but that (it seems) they would so blatantly lie to someone they professed to want to hire that this was homework, when in fact it was not. Even worse was the feeling of disrespect that they wanted to freelance part of their system for USD450, trying to abuse the hiring process/ talking a big game about offers, to unfairly lowball the price. Like they are a well funded, established, largish YC alum, at least they can afford to pay market. Anyway, everytime I see "XXXX is hiring a YYYY in ZZZZ" on here from them, I'm reminded. Feels good to vent.
Just sad. Anyway, I'm not going to name names because this is unwise to do that in a public forum.
Anyway, I only added the above anecdote after writing the following comment.
This is why Triplebyte is so good. You do a single technical interview ( after a quick online quiz ), then get fast tracked to final interviews at various YC / silicon valley startups.
Sure, maybe not everyone wants to work for a smallish startup. But some of their clients are 200+ people. So...it's not all the same.
I don't want to pass judgement on your coding exercise, and it certainly seems fishy with the contract, however...
I would point out that on the hiring side, evaluating a submitted project in a way that can differentiate between good and great candidates, and that the reviewers can understand well enough to get the signal they need, is very difficult.
Because of this, I know several times we have asked candidates to build a simplified version of something that we have already built internally, where we understand the problem space deeply. We only ever do something we have already built, as that's the whole point, and we typically simplify it, although that might not be obvious from the outside. We also cap the time at 1-3 hours depending on the task as we care about evaluating the candidate, not getting a complete/usable/production ready solution, and so that it's not dependent on how much time a particular candidate can spend on the problem, which would introduce biases.
This isn't perfect, and we try not to do this, and to use problems that are obviously toy problems, but sometimes it's necessary.
That is fine, just be upfront about it. Still require a contract for it is reeking of trying to get work for "free". If you have it running in production, and use it because you understand it thoroughly, it shouldn't matter what the candidate comes up with. Any new insight he might bring you can always incorporate with your own code. In any case none of the code is expected to be used in production, and if it turns out you want to, it's probably the candidate you want to hire anyway.
Just a minor bit of friendly feedback on your comment, I hope you don't mind...
You said "Any new insight he might bring". To refer to a hypothetical developer as "he" contributes to the stereotype of developers being male, and that can make those who don't conform to that feel less welcome in the community.
I'm sure this wasn't a conscious decision, and I know it's something I slip up with frequently, but I'm trying not to do it, and I think writing that is more inclusive is more persuasive to more people and generally better.
If you want to use the candidates code, then you have to have a signed contract. Nothing prevents you from looking at the code, and write your own version of it.
this, just have them do a small subset of something you already do. It tells you so much. I also tell the candidate that if they complete it and it works they are guaranteed to be hired at some level. At the end of the day if you can't show me your work on a open source project, or demo me something you made, there is no way for me to know you can actually build something until you do! Especially true for junior level positions!
I have noticed a trend of not naming companies here. Is it against the rules? I fell that we should name and shame companies that are disrespectful of peoples time.
I've had multiple comments detached (as you can see by my comment history) for calling out companies that suck to apply for in the monthly hiring threads. Either the people who posted must have complained so they removed it, or HN doesn't want people to know some companies suck to deal with when it comes to hiring process.
Would Reddit be more open to something like this as in could you create a subreddit and remind HN readers to check?
I see a few people seem interested in the idea of naming such companies. I have wasted a few weekends worth of my time (which at freelance rates is a decent amount of money) and have been ignored when I asked for feedback (sometimes just ignored completely). Its disrespectful and companies should be called out for it. I could name at least one one company that regularly posts in the hiring threads here.
I made a place that is not moderated by HN, where you can share your experiences of applying. It's called hirepwned.xyz and that's the domain name. Just wanted to let you know because you were passionate about this.
I'm still undecided about the best option for these situations, but I made this ( simple ) site as an experimental database of people's hiring experiences, and posted to HN. I'm letting you know because you cared about this: https://news.ycombinator.com/item?id=16895043
It’s a chilling effect caused by overzealous moderation.
When Coinbase first launched, someone was having issues getting funds out of there. That person made a thread on HN, and the response has always been “HN is not a support forum for YC companies” by the moderation team.
If not here, where? There's no other forum where it will be easier to hear both sides of the story, naming a name on HN is probably the best place to name names because it's pretty likely to actually get hashed out in the comments.
There perhaps now is another forum because I made it ( actually reused some old code ), after reading these comments. But it's not a forum. More like a very simple database with elastic search. I'm replying you because you seemed interested in this. Hope this is useful somehow, the HN post is here: https://news.ycombinator.com/item?id=16895043
> Anyway, I'm not going to name names because this is unwise to do that in a public forum.
Why not? create a throwaway if its a matter of coming back to you. Otherwise we really need to name and shame these companies if we have any hope of getting the practices to improve. Sunlight is a great disinfectant!
As part of an interview I did recently I was asked to build a web app that mimicked a part of their service and deploy it somewhere for them to view. I probably spent about 15-20 hours on it in total and I believe I did more than what the brief asked for only to be told I did not get the job with no explanation, even after I asked if they had any feedback.
I thing is I don't think anybody even looked at what I made because to view everything in the app required the user to login (part of their brief) but no new users showed up on the site at any point. Pretty frustrating experience overall.
Despite some of the comments here, I'm actually a fan of the 'homework' format. I'm not always great at hiring, but I know how to gauge the 'homework' interviews, and I've usually been right about the interviewee's abilities afterwards, WAY moreso than whiteboarding or riddles or any of the other alternatives.
That being said, I would add some constraints:
1. The 'task' shouldn't take more than 30 min to understand, and no more than 1-2 hours to do. If it takes more than that, you should make some obfuscated dummy-code endpoints that do everything you're not directly evaluating on.
2. You should let the interviewee know that it should only take 2 hours as well. If it takes more, then there was either a misunderstanding in the task, or the interviewee doesn't have the right knowledge/experience/qualifications etc.
3. You should do your homework as well, meaning you've thoroughly reviewed the code submitted and have detailed questions ready to go for the interviewee. This should also be the 'control' to make sure the interviewee really did write the code (you'll know really quickly if they didn't)
4. No other coding tasks in the interview process. You can have them review code or suggest approaches to solve problems (things that people do on the job with others)
5. The interview should be shorter than if they were asked to do whiteboarding or live coding or anything like that.
Most employers have (on paper) "trial periods" of around 3 months. A 1-2 hour coding assignment followed by 1 hour of well-crafted questions about the assignment (plus the usual interview stuff) should be enough to know whether or not you want to give the employee a chance for 3 months.
(I realize most employers don't use the evaluation period for evaluative tasks, they just have the person start and hope for the best, but if done right, it's actually a pretty good system)
I'm going to disagree with you on a number of reasons.
1. A lot of studies out there show that the most important factor for a hiring success is culture, not skill.
2. Are you even measuring skill with your test? It doesn't matter if it should only take 2h for a candidate to complete your test, if they're nervous or want to impress they're going to spend 12h on it. How can you tell the difference?
3. What about the really good candidate that doesn't want to spend 2h on your test, because he also went to 2 other interviews in the same day where no tests were demanded of him. You're again not testing for skill, but willingness (and against other employer's testing regimes).
4. What is your test about? As a random example implement Web Service X that does Y. Is it important for you that candidates know how to implement X from the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively answering 2 questions, where you could have interviewed him in 2 hours covering his whole history of computing experience, what he loves about it, why, what code he's written, what he's into, what he thinks about X,Y,Z etc.
5. Your job is not to impress him with your knowledge (not that I'm saying this is what you're doing, just a general remark on interviewers out there), but to determine what his is, where he's strong and where not, and how the person will fit into your structure. Making your test the big determining factor is a failure.
6. There's a reason fizzbuzz for example makes a great starter test. It immediately weeds out people that can't complete it, and allows you to see the interviewee reason through it. He goes for example, well, I take this and that, then I have to do this; oh yeah and so so etc. It allows you to see a lot more than just the end answer. After that you can apply more interesting/harder questions, but focus rather on how they do it than if they can complete it in the allotted 5 minutes.
I find code review-style discussion of a simple coding challenge to be great for assessing both culture fit and skill.
What we administer is like a web app equivalent to FizzBuzz in terms of difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get different results from the kid who just successfully finished a CS degree and the grizzled veteran who's built and maintained complex systems for decades.
The right challenge can be completed in an hour or so by a mid-level developer in a brute-force manner. It will weed out those who simply can't code (or don't grok how the web works or how apps are constructed); and also highlight candidates who are capable of more sophisticated work. If being up to speed on a specific technology from Day 1 isn't important, you can give the developer their choice of framework or language or whatever.
Then when you do a code review in person you can get a feel for both how the developer approaches code (a skill) -- but more importantly, how they take critique. Do they get defensive? Do they riff off your ideas? Do they suggest improvements/refactorings they would have made if they had more time?
You can also explore other cultural factors - for instance, how did they approach constraint trade-offs in terms of time vs. completeness vs. sophistication? How clearly are they able to communicate about technical concepts?
This actually sounds really interesting, putting some code up for one interview or take home assignment, and then asking them to walk through a code review. You could also throw refactoring into the mix, ask them to actually refactor it if needed.
It also got me thinking: what if you just had some generic code, which was low quality, or a prototype front end app, and then asked them to refactor or code review it? That could spark some interesting discussion and be less stressful, but may be easily bsed I think.
Regarding #2, I believe your observations and concerns are correct and valid. However, in these situations where the candidate spills his technical heart out, you really start to see the difference between good candidates and bad candidates. Good candidates just have a deep sense of the problem space, can enumerate issues, document such things, etc.
Well when done this way with all the constraints you specified, then yes the homework interview works very well. Unfortunately, the vast majority of hiring managers do not come anywhere close to any of the points you mentioned.
Just last week, I was interviewing for an opportunity as a VP of Engineering for a post-early-stage startup. The CTO gave me a take home assignment involving building not one but three complete applications (to be delivered in a docker container so it could be easily run). When I told him that I didn't have the kind of time it would take to perform this, he told me all the other candidates for the same role had no issues with it.
I ended up withdrawing from the process.
If senior roles itself are being hired like this, then I can only imagine how much worse it would be for individual contributor engineering roles.
There's a trade-off for companies. This kind of hiring process will filter out 99% of low-quality candidates, while also filtering out <99% of quality candidates. So it increases their odds of a not selecting a poor choice.
This is great for companies who are willing to sacrifice getting the best candidate for assurance they won't get a bad one. This makes sense for a small startup, where a poor candidate hurts growth more than a great candidate helps it, but less so for a Google-class company where a single great candidate is like a billion dollar lottery ticket and the bad candidates cost almost nothing.
Senior roles should have more demanding technical interviews than junior roles. The company had a bullshit-preventing immune system. It's not a culture fit for you. Win-win.
When hiring for a senior role, you should also consider that the person, who is already in a similar role, would not have the time to spend on your humongous homework app. When literally the only free time I have for myself and my family is a few hours everyday and then the weekends, I am definitely not going to spend the majority of that on an interview, not would I expect my candidates to do so if I were the one doing the hiring.
So yes, I suppose in a way it was not a culture fit for me, and that was indeed why I ended up withdrawing. I would also urge anybody interviewing for any role to evaluate whether a company which requires you to spend a substantial portion of your personal time, before you're even an employee, has the kind of culture you're okay with.
I start with a phone screen. Then onto a (relatively short 45 min to an hour and a half) face-to-face. This is a 'let's get to know each other and our expectations' low-stress interview just to make sure we understand each other and what it is we're actually getting into.
Then, a technical discussion. Approaches. Experience. Past projects. Some 'did you lie on your resume' questions but nothing that would require a whiteboard or anything. This culminates in getting a paragraph of requirements that will lead to the take-home test. The requirements are intentionally vague and missing critical information to see if the candidate recognizes this and asks the right questions (I'm convinced that 60% of any job is asking the right questions). Then, when I feel like they've hit a few of the important gaps I pull out the detailed specs that tries to answer all of the questions they might have, serve as a reference sheet, and make it clear what not to do (like turning a two hour task into an over-engineered weekend project just to try to impress -- I've got to review all of this code after all!).
This is not a real world test but it is a somewhat simplified version of a real-world thing they will be expected to do. It shouldn't take more than 2 hours.
I point out that if they have any code they can show me from a real, working project it can act as a substitute. What I am looking for is knowledge, approach to the problem, and code style.
Afterward, I am looking for how they handle constructive criticism.
This has saved me and I think applicants so much time with much higher quality information than a long interview or a nerve-wracking whiteboard session.
I give them five days to a week to finish. Most do it within a day or two. If a busy person can't find the time that they'd normally spend in an interview session essentially any time they can fit it in over the next week or so...well, too bad.
Oh, I also give them explicit permission to post their solution on github so that if we don't end up hiring at least they'll have some code to show the next potential hiring manager.
I believe the big problem people have with take-home work is trust. Nobody wants to do a take-home assignment that doesn't really impact their prospects of getting a job, or, worse, that has only downside for them. Hiring teams give assignments and then run a standard interview gauntlet. On both sides, most people believe it's the interviewers making the real decisions, without much consultation to the tests.
The candidate thinks, what's the point? It's a seller's market. I'll work somewhere that doesn't haze candidates.
We run work-sample challenges now, and (loudly) did so for about a decade at Matasano. We had essentially no complaints once we locked our process in. Part of the reason why, I think, is that we tried hard to be very clear about our process. But more importantly, the work-sample challenges made most of the decision. When we told candidates the time they spent on the challenge offset time they'd have to spend in person performing for an interviewer, we weren't lying. A 4-5 out of 5 average on our challenges more or less totally technically qualified a candidate; the odds were overwhelming that such a candidate would get an offer from us, and we told them so before they came in for interviews.
What I think I see now is hiring teams who want to have things both ways. They don't want to take a leap of faith with a new process, so they're still abdicating their responsibility to random subjective interviewers. But they want to look on-trend and competent, so they assign homework.
When you get on the phone with one of these teams and want to check them out, ask about the rubric for the challenge, and ask if they're going to do any on-site technical qualification after you do them (if they're giving challenges after the interviews instead of before, hang up the phone on them.) I'd be surprised if even half the companies that assign homework even have a rubric.
> Nobody wants to do a take-home assignment that doesn't really impact their prospects of getting a job
I spent a few days on one, sent it in and was told that day that they had made an offer to someone who accepted the offer. Then I did another one over the course of a few days which was looked at, but a new VP came in and eliminated the opening.
It's similar to asking me to come down for an interview for a few hours during the work day (and then to come back, because one person I had to talk to wasn't there). Or before I've even talked to anyone, they hand me a form asking for three references, preferably (or exclusively) former managers, and their current phone numbers.
I don't mind them asking or me giving these things if they are near a decision about me, but almost every short phone screen I have nowadays is followed by a request for a work sample that in reality would take a couple of days to do (between my schedule, and wanting to send it in near-perfect).
Something I think is preferable is for them to ask me for code I wrote. Then I can write something once and be done with it.
As others on the thread have said, in a tight market, companies put up huge new barriers to getting a job with them like several days worth of free work, then bemoan the fact they can't hire qualified (young, SF local) people who "care about their mission" of selling advertising junk from one corporation to another corporation.
I provided some sample code recently, it was experimental, rolled-at-home 'research' code that was, I think, in line with the mission of the team I had applied to, complete with test data and instructions. Needless to say, the interviewer didn't want to talk about what the code did but instead grilled me on various technical language details. In some ways it is a great litmus test, because the people I have enjoyed working with in the past would have had endless questions for me about what the code was actually doing...
We have a suggested amount of time to take, but we don't put people on an actual clock or take into account how long it takes them to solve the challenges.
One big issue that a take home assignment is that it removes the ability for the candidate to ask questions. Interviews are two-way streets.
Nowadays, I reject more companies than I interview for based on an initial phone conversation. I have requirements in what I look for in a company and position, and some simple questions help me resolve many of the primary requirements. Given a take home exam, I will have invested hours of my time, before given a chance to screen the company.
Just say no. End the madness.
EDIT: I should add that when I mention a phone conversation, I meant with someone on the dev team, not HR. A take home exam after only an HR phone screen is not acceptable.
At my current job (where i've interviewed candidates) we give out a take home assignment, but we explicitly enforce a 2 hour time limit to do it. We felt this was reasonable because combined with an onsite interview and a phone screen, the total amount of time a candidate would spend interviewing for a position is about 8 hours (a typical workday). I'm not against take home assignments per se - but 3 day assignments seems crazy. I can't imagine myself doing that as long as there are other decent jobs out there that don't require it.
I think this is the best approach. You get the opportunity to do exactly what the job expects, without the pressure of having to manage communication with someone breathing down your neck. A savvy employer can extrapolate what you're able to produce over 2 hours to whatever project they'd have you work on.
We've had pretty good luck with it thus far. Generally, it is a good way to chop off the long tail of "not good enough" and bring the better ones onsite.
As you said, we calibrated our general expectation to what we think is reasonable in 2 hours - so we expect "good enough" code, but not perfect code.
I've done homework as a candidate twice, and given it once as part of the hiring team. Every time, it was after an initial phone screen.
When we gave homework, it was to a slate of junior candidates, primarily just out of college. When I did homework, I was unemployed and looking for my first job in tech.
If homework is a good idea ever, those are the circumstances. You have candidates who are hard to evaluate, will get a large benefit from a job offer, and are less likely to have severe time constraints. I wouldn't make it a requirement for all potential hires.
Also a good interviewer WANTS questions from a candidate. They show that you've actually thought about what it would be like to perform the job and understand what sort of environment you thrive in and thus can filter yourself out if you think it's a bad fit. Because you don't want to waste you time starting and then quitting or being fired a few months down the road.
One of the biggest mistakes I see strong technical candidates make in interviews is not asking any questions at all.
work environment, fluidity of assignment, career opportunities, acceptance of suggestion, involvement of developers into the design process, key tools and applications, workflow automation expectations...
there are lot little details that one can ask and will give you back a picture of how the working environment is going to be. here's a good starter list, even if I find it too formulaic and question you need to ask heavily depends on what your role will be: https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
also, as an interviewer, if a candidate doesn't ask about the work environment and daily operations I'll take it as a potential sign of the candidate having never reflected on its own craft.
includes dev-ops, pace of development, how and when does code end up in _production_? It is important to ask at least two different people separately because you are likely to get different answers.
A while back our company switched to the "homework" format for evaluating software engineering proficiency because we felt like it took a lot of stress off of candidates to write code in an unfamiliar environment (or worse, on a whiteboard). And I feel like it's worked out really well. But we have pretty strict criteria for any homework problem: it needs to be a simplified/toy version of something we'd actually ask the person to do if hired, and it needs to be something that a qualified candidate could easily complete in an hour.
The code itself is, of course, valuable for helping us to evaluate, but honestly some of the best signals we get come from the open-ended discussion that results from us talking with the candidate about their code when they are done: "imagine this file you're processing is 100TB instead of 100kb? Would you change anything?" or "What if this clean data you've been given to work with had this or that issue, what might you do?"
My only point of disagreement with this article is that it seems to assume that a homework problem has to be really time-consuming or burdensome, but there's no reason that has to be the case. To me employers who are asking candidates to spend hours or days on homework are of a piece with ones who ask people to solve tricky, unrepresentative computer science problems on a whiteboard: they reveal that they don't really know how to interview people and they don't have much respect for their candidates.
Interesting, I actually prefer take home challenges to the previous trend of whiteboard interviews. And within the realm of take home challenges I prefer ones without time limits which usually implies they are full or multi day assignments.
Even with rehearsal I am not a great public speaker and impromptu formats certainly don't make me better. I also prefer take home assignments since it is more closely related to how I actually do work: Get a problem, think about the requirements, do some research and design a solution, implement.
Since I personally prefer this style of interviews, I have given assignments like this. Usually a simple microservice that is given AWS t2.micro size constraints and no time limit for a response. When I get a submission I am looking at code style, tests, and documentation as well as solution quality to get a better picture of how a candidate approaches tasks and gauge how they work independently. It's amazing how much you can tell about a single person from a challenge like this.
I also think it is more flexible since there are no strict pass/fail criteria. A solution in C/C++ that optimizes heavily but skimps on documentation could be just as impressive as someone who uses Python to stitch together multiple tools with more features and more thorough documentation.
Interviewing is difficult...for both interviewee and interviewer. Finding the right compromise between depth for the interviewer and work required of the interviewee is a challenge. I am sure there are take home challenges that are ridiculous, but I wouldn't dismiss all take home assignments.
With no time limit, aren't you just optimizing for the candidate with the most free time? (Granted, some people are more efficient than others, but you probably only want to hire from the top band of efficiency anyways. Meaning that "time spent" becomes the primary deciding factor.)
This is definitely something to keep in mind, but we keep the requirements for the microservice extremely light for this reason (2 API endpoints). A solution submitted in a week with many extra features can be just as appealing as a solution submitted in a day with the bare minimum. It's not primarily about how much work is put in or how fast it is done; rather it is about the quality of what is included.
If the software development industry was the home building industry...
I'm a general contractor (Unicorn Homes Inc.) and I'm currently building a number of spec homes. We're going to need electrical work done in all the homes and I'm in the process of sub-contracting out that work. What I need from you, Joe Electrician, is for you to go wire the kitchen of one of these homes. Once you're done I will inspect your work and let you know if I'm interested in hiring you for more work.
I've never seen a take home coding challenge that had even the possibility of being used as production code. For me it's always been a standardized, self-contained example problem. Maybe there're scummy companies that try to push job applicants' work to production, but I'd be shocked if that was anywhere close to the modal case.
So it's more like "What I need from you, Joe Electrician, is for you to go wire up this demo kitchen frame that I've set up. Once you're done I will inspect your work and let you know if I'm interested in hiring you to wire up the actual houses I'm building.".
I've definitely seen some prompts that looked suspiciously production-ready.
It's usually not "build out our new feature", because spinup time is a thing, but some companies do seem to throw around "build our new one-off microservice". Stuff like "given this incoming data, set up intake, a storage layer, and a backend layer that applies formula X". If the data and the formula look business-relevant, it's at least enough to raise an eyebrow.
That said, I'm not sure how much I care, provided the interview is sincere and not some kind of bizarre scam to farm out work. My main concern is much more with scale. I totally see the merit of replacing 5 hours of whiteboard interviews with 5 hours of "build a toy project at home". I don't so much appreciate "take a week of your life and build a substantial project so we don't have to do any legwork".
I should maybe have gone into more detail on that example. If BigCompany or even AcceleratorCompany asks for product-relevant work, I'm not going to assume they're having random people off the street write production code.
What I've seen is something like "this is a startup in market X, whose product doesn't do Y but could, and they're asking candidates to write code that does Y, and sign over all rights to everything provided during the interview process".
Optimistically, it means that if whoever they hire will be told "hey, first assignment is Y, you've already made progress!" Pessimistically, it goes in the same category as "Whartonite seeks codemonkey" and "sign this NDA to interview" - it's a matter of ego and ambition exceeding skill.
> We've designed this project to take about 2 hours to complete.
Then you've got it right.
It's good to see people admit whiteboard interviews are a crummy system, I think you can learn way more from X hours of "do a task on a real computer with the internet" than you can from the same X hours of "do algorithms work on a whiteboard".
The objection, for me at least, is to "do 10*X hours of work on a real computer". It's bizarrely common to massively scale up the time investment when moving to this sort of task. And the answer is so easy - just don't do that!
Which makes perfect sense and isn't controversial. You're simply hiring them for a small job to evaluate their performance before you hire them on for a much bigger job. It happens all the time and is normal.
You are welcome to go work in an industry where all the projects are essentially identical so once you've done it once you can do it for anyone anywhere. Enjoy your pay cut and ease of getting hired, unless there is an oversupply .
I don't understand this development. In Germany, we've basically got full employment in the IT sector. Why would anyone waste their time with this? Just apply for the Job next door.
As an employer, hiring someone new is always a leap of faith. Might work out, might not. If it doesn't, dont feel bad to fire people again.
In Denmark we put a job offer out on the various job banks, receive the applicants, go through them in the hiring team, starring them 1-5 and we call the best 5 candidates in for a 30-60 minute interview.
Mainly it’s just a talk to see what personality is the best fit. Sometimes we do a personality test, but only for people who will do any form of project management. We’ve never done a technical test or had anyone program for us, I know some companies do it for juniors that are fresh out of school, but otherwise it’s faily uncommon.
That’s it really.
I’ve hired around 20 developers like this over the years, none have failed me. One junior wasn’t really worth his salt at first, but he learned quickly.
When I read about these American hiring practices I wonder why anyone would want to work there. I wonder if it’s because a lot of American developers are autodidact? I mean, I’ve never interviewed someone who wasn’t educated in some form of CS, academy level and up.
"In Germany, we've basically got full employment in the IT sector."
I can only confirm this. I used to be a freelancer and I got almost daily offers per email and phone. In order to stop I ask them a very hi price per hour ( 90-100 euros ) and to my very surprise yesterday the guy said, "that's fine, when do we start?". I backed up a bit and after some discussion he told me that they really could not find anyone. There is really no IT people in Germany without job, so the only way to get work do is to turn to freelancers and offer a good rate.
You have a bit of the opposite problem in America. The market is flooded with applicants. The real issue is that many of them are woefully underqualified (see the fizzbuzz debate). It's a constant struggle to find a system that filters these people out without insulting the people you are trying to hire in the first place.
Contrary to what you seem to expect, that price is a steal. A more realistic rate for contractors doing basic IT work (like configuring DB backups) is 400-800 euros per hour.
Please tell me where I can find these jobs that will pay six figures a year for two hours a week of basic IT work. This is more than an order of magnitude higher than anyone I know makes.
They made me do coding problems on the spot right there. Given an integer n, write a function to print ever digit backwards. It took me a minute to think about it, but i solved it, and evan ran it at home to make sure.
Several months later they ask me to do an online 3 hour interview. I did it, there was no human interaction whatsoever. I had to record video responses and solve code problems.
More than a month later with no follow up, i got an automated response to take another 3 hour interview with no human interaction.
I did not take the interview and will never re apply to IBM
Reminds me of an "interview" I had last year with a secure email company that gets mentioned around here a lot. I was given a coding task to do. Think it took around about a full work day spread over a few days. Wrote some code. Submitted it. Got an email a few weeks later to say they'd hired somebody else. No conversation about the work I'd done or the position it's self whatsoever. My only communication with a human was basically somebody telling me they were going to be sending me some work.
They originally sent me the work over Skype. I don't generally log into it Skype but I did in order to accomodate the way they wanted to chat (send the work). A few weeks after they told me they'd hired somebody else I logged into Skype again to find a message asking how to access the work I'd done from several weeks earlier. Even though I'd provided a link to it earlier on in the conversation. To this day I don't know if anyone even looked at the work that I spent a full work day doing.
Perhaps they just didn't like the work that I did. I wouldn't blame them, I wasn't particularly proud of it. But I had a lot to say about the work, the problems I had and the things I would do differently if I were to start it again. I wasn't given the courtesy of expressing those thoughts though.
The funny thing is, they approached me about the position in the first place.
I wont be participating in any "interviews" structured anything like that ever again.
I actually prefer take home. I’m not great at white board solutions as they’re basically asking you to solve something immediately with a gun to your head in real time. It’s very unnatural. Take home gives me time to think about something more like I would job.
That said there needs to be a balance where the tasks are enough of a test to give indication of quality of work without taking 3 full time days to finish.
Whiteboard is cruel for actual coding. It's too far removed from your normal tools. Whiteboarding can be good for architectural design or stepping through something.
A sixty minute screen share is the sweet spot for me. They get to use whatever environment they are familiar with. Seeing someone work in their environment also tells you potentially a lot about the candidate.
Staff engineer that can't navigate their debugger?
Senior engineer that codes by copying every line off of stack overflow and mashing the keyboard until the lines somehow work?
Bootcamp graduate that builds test cases in Excel?
Junior engineer that writes code in notepad.txt and pastes it into a Python shell?
All things that I've seen happen that tell a different story than a whiteboard or a 3 hour submitted homework test.
It was a combinatorial sum-of-n problem. Given a sequence of integers, can you produce a target by summing N of the values. He used Excel to put the expected return value in column A, the target in B, and C+ were values. Saved as a CSV which his program ingested. It was neat, actually: the rationale he gave was that it made it easy for anyone to add test cases.
(The candidate was previously in the accounting biz, so his navigation of Excel to do this was wizardlike to produce a few dozen test cases in about a minute.)
Not incompetent. Doing it via screen share lets you see their approach to problems — sometimes those approaches are poor. Had it been a take home test, I would have never seen him do that, and never had the opportunity to discuss the approach.
I've noticed an interesting trend in my career: the more grueling the interview, the less-satisfying the job. The jobs I enjoyed the most, had the most interesting challenges, etc, were the ones where it was little more than a handshake. The ones where they put me through hoop after hoop turned out to be the ones that were the least interesting and had the least upside.
On the flip-side I've worked with engineers that were so entitled it completely undermined their ability to work with others in the company.
Work is going to involve some bullshit. That's life. If you can't jump through a few hoops for a job then you don't want the job very badly or have an ego problem.
Don't get me wrong, expecting someone to spend three days on a technical solution is nonsense. But the "special snowflake rockstar" mentality of some developers is not helpful and important to check for during an interview process.
It's not new - I did my first assignment around 18 years ago - and it's not necessarily a bad thing if done right.
One problem is that assignments take too long and have over-optimistic estimates, which apart from taking up time can lead to feelings of self-doubt. Feedback on the assignments is lacking to no-existent, and this is an unforgivable sin in my book. Also, anything that looks more like a contribution to their own software than an academic test is a red flag.
The other options noted in the article are also pretty bad. Whiteboards and fibonacci numbers, forget it. "Pair programming" in an interview is deceptive, nothing like normal pairing; it's a test (fair enough, but don't call it pairing). It's also problematic for me because I have a problem with my eyesight and any "accommodation" requested up front tends to be inadequate, leaving me at a disadvantage.
For people with busy lives or family commitments, a short "interview prep homework" of 30-90 minutes would help the person to prepare, in their own time, in a relatively less pressured environment. The interviewer could then expand on this during the interview.
I've done assignments like this where it proved a useful opportunity to brush up on a particular language (if cross-training or coming back from management), sometimes
even enjoyed it. Also, it means recruiting is less of a numbers game - you might interview with a handful of companies instead of a dozen, choosing the ones you really care about.
IMHO what we're seeing is a terrible implementation of a reasonable idea.
I went for a job interview with a company years ago just after I graduated. This was for a graduate position so I kinda made a few assumptive expectations on what they would want from me and my skills.
Went in there, did their pointless psychometric tests (which by the way litterally have no relevance to my skills) went through their interview and went out their door passing it with flying colours.
I got a call from the agent leasing with them. She was shocked to say that neither me or another they interviewed made the cut.
I graduated with a first class degree with honours in Software Engineering.
The company said I wasn't qualified enough for the graduate position!
Told a good friend and lecturer at my university about this and he said, "pass me the name of the company, i'll get them barred from exposing internships and job oportunities to other students. Your time was utterly wasted and that was a completely pathetic reason to not accept you".
The agent said to me that she would fight to testify my qualifications. I told her that there was no point. This company is not only wasting my time but your time as well. They dont deserve your time as an agent nor my skill set.
Performing well at a job takes a lot more than just technical skills.
Social skills and having a compatible personality with others on the team is very important to team success too.
May have been something else but you saying "did their pointless psychometric tests (which by the way litterally have no relevance to my skills)" makes me suspect you failed for non-technical reasons.
Ive been grinding away at these for months... coming out of freelancing, latest contract with a startup that hit its burn rate and failed, and having beeen unemployed for almost 6 months (except not really unemployed, since contractors dont actually collect unemployment...).
So many companies have these assignments. Theyre infinitely better than whiteboarding, or hacker rank tests, or "pair programming" on some crazy abstraction for only 30 minutes. Some of them are so much work - I am working on my 20th hour of a really involved full stack project, and still think I have about 6-8 more hours before its "pretty".
Then I have 2 more to do, and I told the HR people at those companies Id do them 3 days ago.
One weekend I spent 12 hours doing a SOA example project, and on Monday, the owner simply deleted the upstream - and my fork along with it - and said nothing back to me. I hadnt even turned it in yet!
Hours, weeks... its been months of this. Im working for free and its just energy that goes into thin air.
Not once has anyone actually done a real code review, I suppose they just get buried in hundreds of these little projects to review and select someone and just never get back to you :-/
Next week I have about 6 interviews lined up. Cant wait to see how many unpaid hours of work I get to do!
...oh, and I have > 12 years of industry experience, have built multiple applications that garner 50mm requests per month (I can almost guarantee you, that youve been on at least 2 of my websites before), have worked with IoT, HIPPA medical, ecommerce, greenfielded, been on big teams, worked internationally, have buttloads of experience, and every person Ive ever worked for praised and lauded me plenty. Yet I cant get a fucking job for the life of me because of these paint-by-number processes.
Man, thank you for writing this. I am totally in a similar boat - but at it for 3x as long. It looks like you are far more agreeable than me when it comes to these assignments.
Consider that for most jobs, there is naturally a 98-99% rejection rate. Only one position and 300 candidates. So assume that most coding exercises you do will go nowhere, will not be reviewed, and will leave you feeling disappointed.
You are a thought-worker, and an obviously talented one. The most important thing you can do is to stay nimble and don't get discouraged or bitter. Part of that means saying "no" to unreasonable requests. You don't have to full out say no - you can also just negotiate down the scope to something digestible.
Identify your limit and push back should anyone attempt to cross it. For me, if it's a good company and the test is reasonable, i'll throw in 2 hours. If it's a BS test, I'll push back or skip entirely.
These companies hiring with absurd processes do not know what they're doing - and will walk all over you/us if given the opportunity.
I had to do homework assignments twice. Once was for TopTal, which kind of made it worth at the end since I've got a few years of work out if it already and they're supposed to check their candidates thorough.
The other one was for Automattic that seemed to be intentionally vague to check if I would spot some pitfalls and smells they left behind. Never heard back. Mailed the guy that I had contact with twice. Not a fucking word. Thank you for wasting my time.
I don't think I'll ever do it for free again unless it's a position for Godmaster of Cooltech at AwesomeCo.
I've only been asked for this when I applied to a "digital design agency" (i.e. web development) during a moment of madness.
They asked me to write a GAE based web app for something or the other. I told them (politely!) that I'd do it, if they provided me with their diversity policy, specifically with an explanation of how this sort of exercise didn't discriminate against anyone (as noted in the article.) I got a "sure, we can do that" and then never heard from them again. I didn't consider that a loss.
I spent days almost 10 hours, completing a massive interview project for one company. I submitted a fully working solution, had unit test, was pixel perfect, and they didn't even give me an interview.
Not only that they gave me a vague reason, only after pressing for feedback.
Left a very bad taste in my mouth, don't think I'll be doing code assignments again, what a waste of time.
There's no way I'd spend 10 hours on a take home for these reasons alone. I'd spend maybe 2 and call it good. If they expect a "pixel perfect" solution, then I'd better be able to do that in 2 hours max. Otherwise, they get what I submit and it will give us plenty to talk about during the interview.
I think the best way to hire is to hire optimistically, without forcing ridiculous trivia questions or memorizing algorithms.
Then give up to 3 months of mentorship. If the person doesn’t look like they meet expectations, then fire her quickly. Give them the remaining time up to 3 months as severance.
This is the best way to do it because you're not subjecting people to bad interviews, and you're giving them up to 3 months severance if it doesn't work out.
Focusing on the “what to do instead”... There's an interesting suggestion at the end:
> Holiday’s company [CallRail, Atlanta, GA] now uses a structured “live-coding interview” format that involves reviewing code together and talking about it. The process typically takes between 30 minutes to an hour.
I wish there were more details, like whose hands are on the keyboard, or whether the interviewer has a rubric to structure their evaluation.
As a developer and a manger, there are many candidates who fake or bolster a resume, especially in tech. Additionally, there are some developers who are incredibly smart, but take 2-3x the time as someone who is not as senior to do the same task.
The only way I've found to reliably hire is a coding test. I think it's ethically irresponsible to choose actual client work for the task. But it is important to create a test that is similar and with a time constraint, especially if you work on any projects with deadlines so you can simulate how they work under a time crunch.
Not just ethically irresponsible, it is probably illegal. Unpaid work that a company has someone do must be of no value to the company.
It would be legal (though the ethics are still questionable) to give someone work after the client has decided not to give you a contract, as then there is no value in doing that work.
This is why most big companies prefer to to hire contractors. You get someone for 6 months, if you like them you hire them, if you don't you let the contract expire.
What about paying someone to do client work as a test? That is what I've found to be the best approach for me. It gives the candidate a very concrete understanding of the kind of work they'll be doing as opposed to a more contrived example. It comes at the cost of paying for poor work but usually some portion of it is salvageable.
I wrote a couple of these home works and finally gave up. In fact, I won't interview for any dev positions that require me to do a recruiter phone screen, another technical phone screen, a homework and then an onsite coding test.
Expecting production level code in a homework is just asinine.
I'd much rather do a timed coding test for 30 to 60 minutes and be done with it.
I've had to do a lot of telephone interviews and then face to face interviews. I'm regularly surprised how many developers can't do basic stuff. I'm not talking code that requires specific knowledge of an obscure area of an API or language, but things like picking the smallest values from an array (without using a language's built in array min function).
At my current company we get them to write code (or something like pseudo code) down on paper. We give them a print out that clearly explains the task, which requires no domain knowledge and then leave them to it for an amount of time to write stuff down on paper. We then go back in the room and get them to talk us through their solution. But so many developers just can't do this.
Then some can, and they can't explain the edge cases, what if the array is empty? What if it's null? What if it's all the same value? They'll just freeze. They would have past the phone screen first to get this far.
We're not looking for more advanced stuff with red–black trees or an exotic data structure. Just a demonstration of what you've written and simple understanding.
In reality, this is an invitation to a conversation. Why did you do it that way? Is there another way to do it? Which way do you prefer? Why? All if this is basic first principle stuff.
Generally when you get to the good candidates you know. They can answer all these questions and you go in different directions with the questions, they're comfortable doing it too. You can get a technical conversation going. They won't lie and I say they can do X, the good candidates will quickly admit they haven't done X or Y, but their overall impression means not knowing X doesn't matter. The trouble is I've had so many candidates who are on the wrong end of the Dunning–Kruger effect.
Not going to name the company but I topped 38 hours over a weekend to land a job at a tech company. It was brutal.
But I still prefer that to those stupid algo questions.
At a different company, I literally had an entire dev team sit on the opposite side of a table in a board room and play stump the programmer. After the interview I left hating the company regardless of the job offer.
Testing coding abilities should only be done in the context of finding out if I can trust the resume (which should be relatively minimal), or if the candidate is looking for their first job (again, minimal as they won't have experience).
The rest should be possible to analyse TALKING to the candidate or doing some join design session...
Good luck making people spending more than 4 hours in a selection process if you're not Google (and even so)... Seems like a great way of massively discarding anyone that's busy...
I suspect any kind of process will disadvantage somebody. For example the on site pair coding lauded in the article my disadvantage people with social anxiety or who can better focus alone, and traveling also takes a lot of time.
That said, I would also reject homework assignments, as long as I can afford it. If they want it, companies should pay for it.
Multistage interviews are also a big turn off. Imagine going on a date, and your date saying "I just need to try yet another date to see if I like you" multiple times.
I mean we can't have it both ways. You either hire with heavy credentialism or cronyism or you hire based on whatever trivial markers you can come up with for instant validation. Companies are trying tons of different validation markers, from white boarding, take home, stuff like triplebyte.
The bottom line is you can't instantly validate how someone will perform over the course of a year or more. It takes time. But nobody wants to spend time so...
I think assignments are a really good way to test people, but they really do have to be short. The second-last job I got took around 4 hours which I think is fine. Helps if the task is a bit fun too.
Also it shouldn't be the first stage of the interview process. That's just taking a piss. For this job it was after a phone interview, and with a scheduled face-to-face interview (where we discussed the code I'd written).
Getting a (new) job in IT is an exercise in futility if you don't know someone inside, so essentially there's not an interview but an invitation.
It's lower-status, more degrading and has worser odds than cold-call telemarketers trying to sell you some new credit card.
By the way, when a telemarketer tricks you into answering his call, you don't subject him to a 3-days intensive rectal exam ("homework") just for the remote possibility of getting a new credit card, which at the end you still don't buy anyways on some made-up pretense.
So developers agree to way worse level of abuse and degradation than the lowliest sales guy.
Also there's one and only one defense with respect to abuse either at interviews or after you get the job: having a large stash of money, which allow you to say NO to abuse.
At least 5 years runway, entirely on your own "payroll". If you're actually competent, that's more than enough time to pick any skill (possibly starting your own business rather than look for a job using it), if you can do 12 hours per day by your own choosing on whatever you see fit.
Don't have those 5 years but only 1-2 or, god forbid, have nothing more than a few months on top of a large amount of credit you have to pay back.... and you're in the 99% percent of desperate and destitute crowd who will submit to anything in exchange for a bowl of soup (guarantee they never break out of slavery).
Problem with the solution is that it's a vicious circle: if you don't already have those 5 years of savings, chances are you're never going to save them anyways.
I've been experience this more and more lately. Some employers make their own tests, some rip off those from big players like Facebook/IBM/Google and some are just blatant attempts to get some work done for free.
There is a lot of talk on HN about the best way to do IT recruitment, but whatever method is close it really shouldn't take more than a few hours of face-to-face contact and some legitimate tests. With HR doing due diligence by checking references etc after an offer is made.
The first time I encountered this I was told that I would have to do a 8-12 hour coding challenger over the weekend. They wanted me to come up with a way of proving someone has watched a training video without skipping or faking an API call - something they admitted in their interview that their system was lacking i.e. free work for them. I told the recruiter that I would not be doing any such thing. I have young children and my weekend time is precious. If you can't validate my skills from my a few interviews, my detailed CV, plus HR doing due-diligence, then it makes me wonder about the organisational culture of your company.
There is such a low barrier to entry for our industry that virtually anyone can can call themselves a software developer after a week of coding exercises, compared to other engineering disciplines which have a very high barrier (mostly in requiring a degree and often difficult certification like the PE). What's worse is that even computer science majors aren't guaranteed to be an effective hire. This is NOT a new problem[1].
Ultimately I would argue that the risk is higher, for a company, of hiring an incapable engineer, than it is for an applicant being hired (and paid) as an incapable engineer.
Requiring an applicant to demonstrate capability is smart. Where we draw the line for how much needs to be demonstrated is what is being questioned here.
I actually gave a link to my Stripe account and told the company I was applying to to pay me hourly as a contractor for doing this work. In the end, they took it offensively and denied me.
I wish companies would figure out a way to have their best engineers do blind-shopper-style interviews where they regularly see if their best engineers would be able to make it through their current interview process. This can be tricky to implement but if done right, would provide lots of insight into the types of candidates their interview process is geared to pass.
I asked a friend who worked for years at a large public company if he would have passed the company's interview process at the time that he left. His answer: "most certainly not!". I think his answer is astonishing. Here is a guy who is considered a top notch engineer at his company, assigned to help the company screen for similar top notch engineers and yet, the established process would specifically fail a good percentage of high-quality engineers of the type that already work at the company.
That reminds me. At Amazon there was a concept called "raising the bar" that was applied during interviews and promotions. The idea was that you should only hire or promote people with the capacity to do the job better than 50% of the people already doing it.
I often wondered how sustainable that was. Will be interesting to see if they keep that strategy long-term.
What you want from a hiring process is to be as objective as possible. For fresh engineers, the industry has settled on basic coding right from fizz-buzz to reverse a linked list and beyond.
For more senior roles, majority still rely on pedigree (previous employer, resume etc), and those who try to be more objective hand out a take-home which ends up frustrating both sides. Candidates have to spend a lot of time and effort and the hiring manager/devs have to setup/run/review.
This is where the industry needs to make the process easy, and operationally objective and efficient for the initial screen: An hour or 2 spent by candidate on a take home problem, which can be reviewed in under 5-10 minutes by the hiring manager. (This disparity in time is because the ratio of interviewed to screened is roughly that.)
(Disclosure: I work at hackerrank, and build the take-home equivalent of a coding test.)
An interesting thing happened to me when I applied for an internship with your employer. I got a call, saying that I would have an interview few days days later even gave me the exact time. Come the day of interview, I got no call. And no response to follow up emails/calls. So, when somebody from hackerrank comments on the hiring process, it is a bit vexing.
As a senior level engineer who's currently job searching, this trend is really burning me out right now. I'm at the point in my career where I don't want to work for just any random company (I've already worked for NASA and done client work for Young Money, Carrie Underwood, UFC and more). All of the positions I'm applying for are highly competitive. My resume performs well; I get interviews at 70-80% of the companies I apply to. And I usually have 3-4 interviews that go well. Then I get to the final stage, unpaid take home projects that last anywhere from a few hours to a week. Many of those go well too; I get feedback from companies saying they're really impressed by my work and that they'll talk to their team and get back to me with an offer. And then a few days later I get an email saying that they think I'm a really talented candidate but that they're moving forward with someone else. Over the last month, I've been working 50 hour weeks, unpaid, on code that's looked at once and then never seen again. It's really demotivating and exhausting.
I think there are a couple of solutions to this;
1) Instead of unpaid sample projects, companies could offer paid trial periods where you work on real problems and get paid, with the option to continue as a full time employee after the trial.
2) Instead of doing tens of sample projects or exercises, one for each company you apply to, you could be evaluated one time by a third party and that evaluation could substitute for technical interviews done by the companies themselves. It would save companies the cost of evaluating candidates projects, and it would save candidates the cost of proving themselves time and time again. The evaluation could potentially be handled by different entities; tech recruiters, certification bodies, or standardized tests. This may be what companies like HackerRank are attempting, but they don't really seem to follow through with job offers.
As it is, I'm trying to find a solution to my own job search. If anyone has any recommendations on how to find a job without burning out, I'd love to hear them.
Hiring manager: Hey recruitment team, I've read articles saying <interview type A> doesn't work very well, we should try <interview type B>. <Interview type B> solves <concern X>
Recruitment team: Sure thing, we'll start with a parallel run so we can see how <interview type A> results compare with <interview type B>
Hiring manager: We've run both for a few months now, should we stop doing <interview type A> ?
Recruitment team: Some hiring managers still like it, as it addresses <concern Z>. We don't see it as our job to second-guess them. You can spend your time and political capital convincing every other hiring manager, if you like?
Hiring manager: I'm awfully busy with things that are a better use of my time.
It provides a diversity of opinion. You might be surprised at how much disagreement there is about most candidates.
Its probably one of the few parts of the process that actually makes sense, but the fact that people rarely agree belies the deeper problems with tech interviews.
The selection criteria is way too subjective and the bar at so many companies set artificially high.
In my experience your best bet is still to know someone on the inside.
Though even that doesn't always work. A former intern of mine was rejected for a full time job at the company because he was "shy". Mind you this was someone who built something still running in production today.
I do 3 or more interviews a week and the process seems like complete nonsense. (but its my job so I do what Im asked)
Did they do something to correct the intern not being hired or they just wanted extroverts/loud people?
I've erred on the wrong side going the other way, recall one interview on a team where I thought something was off with the candidate because they were so abrasive and then they got into a shouting match with one of my teammates (we thought they were fighting in the conference room), but when it came down to it I didn't want to be the only no and reject them. That was my fault because they turned out to really be a sociopath after a couple of months.
I don't necessarily buy fully into all of these (I am in favor of homeworks, but they should be paid and later staged [this usually means more phone screens. can't win em all]). But since you asked:
1) because meaningfully large swaths of applicants are too stupid to fizzbuzz
2) because passing fizzbuzz is not the same as writing software
3) because teams are comprised of many people in many roles and just because team/person X likes you, doesn't mean you'll be a good aggregate fit.
1) I don't get fizzbuzz in first coding rounds anymore. It's usually graphs
2) Ok. I write software. Your whole team can go take a look at my software. Done coding?
3) Ok, so the last on-site round should be meet and greet then? Because I already submitted my code before. You guys all should have looked at the code and agreed to bring me on-site and just meet me. Why ask me to traverse trees again?
1. If applicants can't pass fizzbuzz then you shouldn't hire them and you can end the interview process right there.
2. Asking them to write fizzbuzz is exactly the same as asking them to write a method that does anything else, unless if they're aware of it and already knew how to do it, but of course you're asking them other questions as well?
3. Agreed, but it's impossible to match a new candidate with everyone in the company. If they're going to be working closely with person X then have that person sit in the interview for 15-30 minutes and see if they can work together on a problem.
Think of these interviews as a means by which interviewers assure themselves they're working on deeply intellectual CS problems and working with people who are incredibly intellectually gifted mathematicians practing computer science (like themselves), or that they're of the same caliber as what they believe of developers at FAANG, and it begins to make more sense.
I was recently on the job market and decided to apply to TripleByte as a way to hopefully skip some of the annoyances. The process involves a short online quiz, and then a 2-hour live-coding session with one of their engineers. It's supposed to demonstrate to potential employers that this candidate is actually skilled at coding.
The problem were the employers, who just treated TripleByte approval as an add-on, instead of a substitute.
One employer said I'd also have to do a take-home assignment (which was, I kid you not, recreate React) and then a 4-5 hour onsite. A few weeks later, that employer contacted me to see if I had completed the assignment. I told him I accepted another job. (Of course I didn't even bother starting the assignment)
The main reason 'take home' assignments are BS:
It will take the qualified person that needs to evaluate the submitted result more time to go through the code you produced and make a basic judgement, than it would take that same person sitting in a room together with the candidate to produce a far better judgement of the skills by just talking with the candidate if front of an IDE.
Truth is many of these 'assignments' will never be looked at by a person that can actually derive good insights from the submitted work. Most of the superficial treatment they will receive could have been established in a five minute phone interview.
Sounds like it might be the same company the author is referring to. I'll throw them under the bus. Hello Fresh interviewed me and that interview went fine. They then wanted me to deliver an application using no framework. I was allowed to use libraries, but no framework.
After reviewing the requirements I quickly determined that it would take me days to build an application that I felt would effectively WOW them. I told that to the recruiter and bowed out. Being already employed I looked at the opportunity cost and it was too high. Sure they paid more and are a bigger company, but that is A LOT of work. I rather do almost anything than unpaid programming on an application that will be thrown away in a few days.
If I were hiring, the "lab" portion of the interview would be simple. For mid to senior level developer:
1. Write a small inefficient application that requires some indexes, SQL optimization, and some code optimization. Very, very small. Very, very easy for a reasonable developer to fix. Afterwards I'd have a conversation about other general optimization things like Redis, Beanstalkd etc... because we'll optimization is important, especially to me.
2. A small class with very poor cyclomatic complexity. Clean it up. Because clean code is important.
3. A small insecure application. Things like not using bind parameters in queries, not cleaning inputs, XSS etc.. Secure it.
4. Show me something you've done. Personal or professional. I would then ask questions and it would be an open conversation. Because I want to see how much you enjoy programming and how we'll you can speak about it.
I think these 3 items, combined with the other interview questions, their experience, and resume would give me enough to go off. Ideally, 1, 2 & 3 can be done in under an hour. I think that is reasonable for pre-screaning an applicant. I certainly wouldn't send me running, but I am also writing the test so who knows...
Why would I want to look through 10 or so applicants full fledged application? God no. I have better things to do. I guess for some companies its a way of weeding people out, but these large tests are akin to using DDT to remove weeds.
I worked for a company that did a 3-day, challenging take-home problem. It was a big ask, but it was amazing for getting a sense of people's problem solving and coding abilities. (The final result would only be 1000 lines of code or so, but 3 days provided time to think and experiment with different approaches.)
Maybe paying people for the time if you end up saying "no" is a nice compromise?
If you are potentially serious about hiring the candidate and they are serious about potentially working for you, doing some "homework" does not seem like a major amount of effort in the grand scheme of things to me.
I've met folks, recruiters and (tech) leadership, at companies that do these type of long, arduous interview processes.
They make it a point of ego / pride that they have such a long, arduous process. They've even mentioned the occasional candidate who asks about billable hours for an 8+ hour code problem, saying "Who the hell does that guy think he is?" Of course, they were shocked and appalled when I asked who they thought they were to ask someone to spend 8+ hours of their time on an code test before even having so much as a phone screen to determine compatibility.
As it were, this was a company in the contracting space so I assume this is an intentional filter to weed out all but the most compliant individuals. If you're not willing to bend over backwards, they're not interested.
If it's going to take more than a few hours, companies should definitely pay.
I know that games developer ThreeRings did that. Once you got far enough along, they'd give you their games toolkit and ask you to build a simple, original game in, I think, 20 hours, for which they'd pay you a decent rate.
I'm not sure I'd do that today, as it can exacerbate existing systemic bias issues, and cause you to miss out on good candidates who have lives where it's hard to free up 20 hours on short notice.
Plus, I think there are cheaper ways to get that data. I now structure my interviews as joint work sessions. One round is a couple hours of pair programming; the other is a couple of hours of joint design and discussion with the team. The goal of both is to see people doing something pretty close to everyday work. I think I can get a much better feel for somebody's skills in a few hours of collaboration than 3 days of remote work with a big-bang delivery at the end.
This heavily biases your interview process to only take candidates with time to complete your homework: unmarried, childless new graduates. If you tried to give me a 3-day interview I would walk away.
It also weeds out, disproportionately, those who have major obligations to family and other things outside of work I have no business asking about, regardless of marital status or children.
Guess which socioeconomic groups it tends to disadvantage most?
As a hiring manager (Director), I've long maintained the entire hiring process in tech is broken, and this movement toward requiring an arduous amount of unpaid personal time on "homework" is tantamount to abuse. Respect is a two-way street in the hiring process, and it starts with me showing them that respect for their time and obligations.
I've ended up having a bit of a row in an interview over this kind of thing because people just bias their tests to expectations they don't even have of their current staff. 1000 lines of decent code in 3 days is a rare thing at the best of times.
"we'd like you to just whiteboard some pseudo code for this problem that one of our senior engineers solved recently"
"was he stood in front of multiple people, timed and drawing on a whiteboard when he solved it?"
"no, but we're just wanting to see how you'd approach the problem"
"well, I'd probably do what he did which is Google some stuff, read some stuff, have time to process it, then write some code and then probably restart as I figure out the direction I want, probably bounce an idea off a colleague or two to make sure I'm not just in crazy tunnel vision land and then I might have the makings of a solution. Shall I whiteboard that flow diagram for you?"
It didn't seem to work out that way at the time; plenty of older people with kids were there.
Probably it helped that we were in a smallish college town w/o a ton of other tech, & this was a while ago when the tech job market wasn't as crazy hot as it is right now.
(Also, I want to emphasize, it was a challenging problem, but it wasn't a lot of code at the end of the day. If you knew what you wanted to do, you could knock it out in a couple of hours. But you had 3 days to mull it over or try different approaches. We were not asking you to build an app or something similar involving a ton of work.)
I'm sympathetic to the fact that it's a big ask, but I really truly feel like it gave me a much better sense of candidates than just an interview & a resume (what we do at the company I work at now).
It was also a fun problem to solve, and people would tend to continue to refine their solutions (to be better and faster) competitively even after they'd joined.
If you currently have a job then you likely can't accept payment without violating your employment contract, so you wind up having to do all this extra work for free anyway.
In the UK I see a lot of tests that say "don't spend more than 2 hours on this", and that I'd generally expect to take 3-5 hours tops. Americans seem to have a strange employee / employer dynamic
I refuse outright to do this kind of thing... mostly, it's because I fail badly ! - I do fine in real jobs, but when it comes to tests and homework, I tend to not get the job.
Also... I could be applying for other jobs in that time or doing.. you know, anything else - lying in bed doing nothing for instance.
Seriously, this trend can go fuck itself... the last time an recruiter told me I had to write 100 words about 12 different skills I told her no, and hung up - only to send her a ranty email about how stupid the idea is.
Latest trend? Haven't we all gone through this before? Back in 2012 I did a "homework assignment" too, just to get an interview with a game studio. The task? Build a SOAP client and server that do some silly little thing, I don't even remember what. Basically, just something to prove competency. It took about an hour, mostly time I spent reading the manual for the programming language they requested. Trivial shit... WTF are people being assigned that takes multiple days? Ridiculous...
I had this happen recently. Not only did they want me to do unpaid work as part of the interview, but that work was directly related to their business. They requested that I do this before they had even spoken to me. I responded that I was unwilling to do it in exchange for a chance at an interview. I was an excellent candidate, and I’d done years of work on similar systems that I could point to. This sort of thing is a huge red flag that the employer doesn’t respect the time of those who work for them.
This practice is a massive warning sign. I've only seen it applied by companies who lacked the technical competency to actually interview a candidate in depth. If the tech lead / architect is competent then he/she can just a candidate from the CV and an hour of chatting.
The two worst developers I've ever had in my team both aced those interviews. How? Their colleagues did most of the work for them. It worked out great for me, it cleared out the dead wood and punished some faker at another company :)
I did a take home assignment a couple years ago where it was time constrained to 2 hours. At first, the idea sounded cool: nobody looking over your shoulder, I can listen to my music, use my tools, and work in my most productive environment. Selfishly, it can be nice (assuming that you didn't just have a long day of work and aren't dirt tired).
I'm very quickly realizing it wasn't that great and in fact, comes off as lazy from the interviewer perspective. There are several "coding interview websites" where you can ask the question, give the candidate a terminal to code, provide a list of available languages, and even have unit tests from the interviewer side to make sure the solution is correct. We've completely abstracted out the interaction and questioning of candidates and it has become a binary "did they get it right or not" type of process. Sure, there's follow up interviews and such but what have you gained from this process that couldn't be asked over the phone?
Worse yet, if you aren't using these tools and simply letting the candidate email you a project (or however they choose to deliver it to you), it can quickly become a time sink as you spend it trying to understand the solution (i.e. why did they interpret the meaning of "API" as a REST service? What is this obscure library they're using? etc.)
I say this every time this sort of article comes up here, but seeing how ridiculous this field has become, I wish I had chosen a different career path.
I don’t know if I’m going to be able to go through all this bullshit the next time I’m looking for a job.
I have a friend who is a pharmacist and I asked her what a typical interview is like for her. She said interviewers will typically ask something like “tell me about a difficult situation you had to deal with at work?”.
No homework, whiteboards, group interviews, or stupid puzzles. No bullshit.
She said interviewers will typically ask something like “tell me about a difficult situation you had to deal with at work?”.
That's just another form of bullshit. I'd much rather write a for loop than try to figure out if the interviewer wants an honest answer or a humblebrag.
Obviously if you don't have a job, do whatever you need to in order to get one. But code challenges are making it increasingly difficult to move forward in your career without subjecting yourself to inordinate risk.
I've been going back and forth on how best to handle this newest trend. I don't think it's a good approach to just refuse to do them. But accepting the challenge and then not following through is even worse. So a mutual understanding of the deliverables and level of effort is required, otherwise job hunting approaches second job status.
My current stance is to first negotiate the code challenge until it's fully in your wheelhouse. If you are unable to do this, politely decline to do the code challenge, citing time constraints. One job wanted me to do a basic web app wholly in vanilla Javascript with no libraries. That's a challenge I should have just declined.
Second, refuse to accept time limitations. No company that's not paying you should be allowed to monopolize your time. One job had a hard 20 day limit and the task was both outside my wheelhouse and expected to make a measurable delta in their production application. I definitely should have declined that one.
But if a company is willing to work with you rather than subject you to an inhumanly-inflexible meat grinder, that's the one I want to work for.
Given that interviews now a days are not one way street.
Candidate is also evaluating whether company is worth working for.
if you give a take home assignment and don't give me a feedback on stuff I could do better, i would consider the whole thing a waste of time. So if you don't offer that's ok. But let the guy learn a thing or two.
Many people will agree that yc interview process is rigorous but it's also satisfying as you get to learn a lot through out.
The whiteboard interview is coding's version of being asked to do a freestyle rap. It's largely orthogonal to creating good code, but it's also a trainable skill that arguably does show something, even if that something is just a willingness to play the game. Whether it's worth seriously training for is another matter.
Cue the music (sung to the tune of Eminem's 8 Mile Road, see https://bit.ly/2iI8tG2) and the scene of a nerd sweating in front of the whiteboard as the interviewers nod their heads to the beat:
Sometimes I just feel like, quittin', I still might
Why do I put up this fight, why do I still write (code)?
Sometimes I just hate life, something ain't right
Hit the brake lights, case of the stage fright
Drawing a blank like
It ain't my fault, great big eyeballs
My insides crawl, FizzBuzz I scrawl
And I clam up, I just slam shut
Gonna travel this road
Gonna show 'em my code
My Big-O is so dope
the universe will implode (as our hero proudly writes = O(1/n^2) on the board)
etc. Someone can make a viral video; I'm too busy practicing code problems.
Might make it more expensive to hire, favoring insiders who are employed at the expense of outsiders. Also makes it harder to switch jobs if you’re unhappy/deserve better.
Some places, yes. But surprisingly many places do a really bad job at encouraging internal recruitment, even when it would often let them identify talent they are unaware of and let them shift people into positions where they might be a better fit or be stay for longer. It's especially bad as it can often help shift more of their external hiring to lower risk entry level hires where a bad hiring choice has a lot less impact.
I tend to regard interviews as an opportunity to determine if the company doing the interviewing is worth working for.
People like me are highly employable/contractable. So, the way an interview works is very simple: you got lucky enough for me to take note of the fact that you are looking for somebody with my skill-set and I've determined that you are worth talking to. Fantastic, because I ignore all HR & recruiter spam so you probably found me through some mutual contact that I trust. Great, now we meet and my expectation is for you to decide to hire me on the spot and be extremely enthusiastic about it and get in a mode where you are selling the job to me as something that I want. We both know that nobody else similarly qualified is just going to walk in and take the job. If so anyway, my advice is to hire them on the spot. No hard feelings. So, very simple choice. Either we like each other or we move on. You don't get to ponder your decision for very long. The words "home work exercise" typically don't feature in such a conversation.
How many of these homework assignments become actual code that companies use? If you ran a business and your programmers were all busy but you wanted some new widget, what is preventing you from posting a job and tricking 100+ programmers into bootstrap your project for free?
A friend recently spent a week building a chat server with a very specific telnet/ssh interface as a "code test" for a video game company. I helped him test it and it was really solid. He walked the interviewer through the setup and how everything worked and then never heard back from them. I suspect that they simply used his code in their next project. What's stopping them from handing over his code to the next round of applicants with the assignment of adding all the features they need? They could post a high salary and offer great benefits, but never actually hire anyone.
We, as developers, simply can't be giving away free work. We're all aware of how opportunistic business folks can be. I refuse to do "homework assignments" (unpaid work) and so should you.
I've been on both sides of this, having completed and reviewed interview tasks, and I have to say it's absolutely crucial.
As an interviewee, it gives you a glimpse of what work will be like once you come on board. Are the requirements solid or loose? Is testing required or even mentioned. These and other things can give you insight into how the business interacts with engineering.
As an interviewer it's hard to take resumes and unfortunately even GitHub profiles at face value. I've personally had code taken from work put up on a colleagues personal GitHub. The coding task is also not a sure-fire way, as I've seen those taken from online tutorials. The real magic is in the tasks code review.
I'm not saying this is how it works everywhere, but given the task matches the deliverables asked and we make it to the code review stage, it becomes less of a tech interview and more of a glimpse into the social skills of the developer. PR criticism is a daily occurrence and getting an opportunity to test these waters prior to fully committing to onboarding a developer.
While we're on the subject, I am tired of taking coding tests or homework, then getting ZERO FEEDBACK on the coding "test" I spent 2 hours (sometimes much more) on.
If I am going to get rejected, then have the professional courtesy to give me constructive feedback on the code. In the future, I will ask for this up front before taking any more tests.
It looked like it might be a devops position with a smallish family video site. After an NDA (?!!) and some emails, no interview, they set me up with an obscure chat client with a guy for a tryout. He had me set up some services on a new instance, apache, php, firewall, etc, following a checklist they gave. He wouldn't answer any questions about the company. I did this quickly and that was the end of it. In the end, they said no thanks, but did not give any feedback. They paid me for a day of work on Elance :-/ $50 for a couple hours of work.
The bad news is the whole process was super shrouded and I to this day do not know why. The NDA forbade revealing any contacts, who the site was, or anything else. Nobody was willing to give me their name or answer any questions. I guess the vanilla family site could have been a sister/front for a gambling or pr0n or offshore hedgefund company, or ... what? Why bother?
This absolute trend of garbage behavior has happened to me for a while now. That mixed with this so called absolute inability for employers to provide feedback because their so called policies do not let them. It is absolutely ridiculous how someone in a position of power is treating others who merely are trying to be part of the ingroup and get a job. This ingroup/outgroup "do these steps, and you will join us" mentality is complete shit. Especially on the part of these so called HR departments which claim they are having such a hard time finding right candidates. Before this jumping through all these hoops crap was invented, companies were hiring fine, people were working fine.
Then these geniuses decided to profit and "flatten the world" as they call it. Just so they can make a buck.
https://www.hackerrank.com/aboutus
The only thing that is going to get you is a flat brain. Unable to think creatively, unable to respond to uncertainty but very very good at memorizing solutions to problems.
And all this is based on this ignorant viewpoint that you can observe something or a product for a short period of time and somehow ascertain anything which you can use to form a useful opinion of a candidates future performance. Bullshit!!!
No HR department should be allowed to ask any candidate to do any work without compensation. It is work since the HR department is using this information to make their decision therefore protecting or enhancing their companies performance. I don't know how this is not related work to a companies success.
I am sick of it. I have started to send templated emails in response to them which explain i do not work for free. I can share personal code from past projects or my github.
A few months ago a PR representative even called me once to make sure i was not going to sue when part of the application process was a take home assignment which was asking me to implement a new feature for their actual product. What a sham.
When i said to her "You mean to tell me that over the past years when you claimed that this process has been the same, no interviewer has derived any benefit from free work of past candidates." She didn't even know how to respond.
The more candidates that do not put up with this behavior, the faster it will stop and hopefully move onto something more agreeable by both parties.
Work sample tests have been researched extensively in applied psychology [1]. There is a healthy debate on how effective they are.
For me personally, I am not so much for or against 'homework', as would rather see it used correctly. For example, I want to know how it will be evaluated, no tricks. I also think there is a quickly diminishing return on the length of the assignment. Up to ~10 hours may be OK, beyond that seems like a waste.
On the other hand, it's also a way to see who's serious. If you are serious about getting a job, it may take some investment. Also, the most glamorous companies can always demand a higher investment from applicants.
Its not just devs that get these assignments. I've seen startups use BD folks for interviews to research and structure potential deals as a "homework" assignment. They dangle a fancy job in hopes of generating leads, which they do, and then they never end up filling the position. Candidates put in weeks of work setting up an actual deal for a fictitious job.
I think these tactics are a lazy way to interview. If you truly want to build a great company, you need to spend significant time with someone to understand 1) can they do this job, and 2) are they a fit with the culture, and 3) are they passionate about the mission. There are no shortcuts, yet so many companies try to hack the process.
Everyone is busy and looking for ways to steal time but this seems to be one of the most short-sighted. The best companies don't look at labor as just an input.
My response - and even how I got jobs early in my career as a developer was this:
"let me know show you some of the software I've developed on my own - I'd me happy to tell you how it works, the tech involved and the patterns used ..yada yada"
Corollary : if you apply for a job or walk into a tech interview and can't talk about these things, its a red flag for the interviewer.
PS. Now well on in my career, and having started my own software business after leaving a bigco. I'm back to a very similar thing: if I go to a meetup / conference / sales call I better have some code I'm will to show and discuss ..and highlight what I consider my talents. (fwiw : it feels great to be back in this mode)
edit: also : asking an candidate to do the kind of homework/coding is total BS. I wouldn't do it. But I would give them the line above :)
Hmmm.. Agree they can't show it, but they can most def. talk about parts of it, their role and show contrived examples, discuss language skills, architectural elements, improvements they wished they could have made, how they worked with others on the larger team, presentations they made... the list goes on.
This is especially possible given that a lot of work includes open source code, publicly available APIs, SDK services etc. In interviews I've conducted, when a candidate mentions these its an opportunity to dive into more detail on their expertise ..and again discuss contrived examples.
In addition, probationary periods are a way to guard against complete mismatches.
I have been developing software for 12 years. I can't show you anything that I've worked on because it has all been internal tools. I can talk about what I've built, but I can't show it to you or show you any code.
Unfortunately that puts you at a disadvantage and is a small downside of your previous job. Many professions these days require a portfolio to show, and programming is quickly becoming one of them (and it's better for your career if you can accumulate it while working). I guess your options are either making a portfolio weekend project or submitting to interview homework or other coding test.
yep -- see my response below on same subject. Key is to walk through the "what" and discuss contrived examples ..as an interviewer I might ask you to show me pseudo code ..but really the more you can describe a contrived system and detail how you would implement, the less my BS detectors will be going off.
I have had the 'normal' 3 hour assignments, but recently I had one that was timed - they gave you 1 day to do it. I got at 10 at night, I thought I would have time but the kids had tired me out and I could not stay up 1-2 hours getting environment set up that I needed.
So then actually I could not do anything about it until 5 in the afternoon the next day, I had problems with environment setup (hadn't needed this particular stack on my new machine so I hadn't set up and things had changed) so it took 3 hours. then I had to do dinner with the family, get kids to sleep. I didn't have the time available in my weekend to do a 3-4 hour assignment that also required 3 hours to get ready to work.
I sent off an email to the recruiter and said what happened and I wasn't really interested in anything similar in the future.
What do folks here think about an interview assignment where an applicant is asked to address an existing issue from an open source project, presented as a variety of cherry picked issues from popular projects but selected by the employer? The code would then be discussed during interview.
I just got one of these for a mobile position.
"Please complete this coding interview. It should take about 3 hours."
Me: "I've been writing mobile apps since the app store launched in 2008. Can we skip this step?"
Corp: "Sorry, good luck on your search!"
> I've been writing mobile apps since the app store launched in 2008
That's your main mistake, admitting that. I'm sure they were looking for someone with five more years of experience than that.
You should have been writing apps in a text editor waiting for the app store to come into existence, you know. I'm sure they picked someone with at least that much experience.
Is it just me thinking it is weird to get hackerrank tests where they use a lot of Java stdin stdout on the test for a mobile position? It might be because that's the only way to grade tests on the platform but I have never used that in an Android app.
I work as a contractor which means I change projects fairly frequently.
In the past, I would sometimes accept the homework assignment. Now I refuse to do those (as opposed timed assignments where you get set amount of time, for example 2h, in which you are supposed to complete the assignment).
If the homework deadline is few days you can practically be sure that there will be candidates that will spend all that time perfecting their solution. What is supposed to take "2-4 hours of effort" in the words of the interviewer is actually the entire duration from the moment your competitors got their assignments until the deadline.
There are other reasons to refuse:
- you can spend that time working to find other offers,
- the interview is supposed to be both ways, they want to learn who you are but you are also supposed to learn who they are. Home assignments give practically nil information about your prospective employer.
- there are other ways to test the candidate with probably same or better results. Choosing a way which minimizes the effort on the part of employer but maximizes the effort of candidate shows lack of respect for the candidate.
- Home assignments are invitation for cheating. I can assume there are candidates that will take advice from others on how to solve the problems they are given so this is hardly proof of their ability. I refuse to take a test where I will be punished for my honesty.
- Even though the assignments tend to be time consuming they are typically hardly challenging. This is typically some mundane application and not really a test of your ability. My ability is to solve complex problems, organize complex logic, produce extremely reliable or efficient applications, draw from wealth of my experience to propose and discuss solutions. Yet another Pet shop written in technology of employer's choice is hardly any test. If you want to know if I can program at all you can learn it in few seconds of conversation with me, you don't need to force me to do 2 days of programming for this.
I worked for a company where I devised a 3 hour assignment given to candidates (full-stack developers) during on-site round. They were free to bring their own machine and work on their choice of stack. They were free to use any web resources available. I even broke down the assignment into small steps to follow.
Assignment was to develop an application which tested the candidate's skills to access web api, browse through documentation, do proper authentication, parse api results and finally the most important skill to design an algorithm to compute desired output depending on the question asked. It was doable within 3 hrs for experienced engineer. Not many succeeded in this task but those who were able to do it were real good developers.
But then I realized that the occasional two days of unpaid homework is still way better than being required to have a college degree or professional certification - unlike nearly every other profession that pays over $100k.
My dad was first concertmaster of the VSO, and my wife is a pianist. What's interesting is to look at the parallels between auditioning for a position in a prime orchestra and "auditioning" for the position of software engineer/developer.
When you get called to audition, you spend a ton of time preparing at in your "free time".
If you want to apply as a (senior) sw engineer, it makes sense to have an idea about the depth of knowledge and/or the engineering approach of the candidate.
The other side of that coin is many orchestras now do auditions blind - nobody evaluating the artist can see the artist, they can only hear them. The numbers indicate that this eliminates most of the bias orchestras traditionally had for visual attractiveness/friends of the orchestra/white males.
For me, I think that unpaid homework is an acceptable, if not important, part of an interview. However, it must pass some qualifications:
* It can't be taken as real work at all. Something like building a game is clearly not "real" for a SaaS company and still conveys lots of value
* It should be discussed in person as the in-person technical interview (don't come in an whiteboard in addition to the outside work)
* It is provided only after a phone screen and it is discussed on the screen.
* There are clear objectives and "bonuses"
* The language of choice can be used
With this, I think that it provides value to the interview process and helps the interviewee as well by simplifying the in-person technical components of the interview.
I've had this happen once. I applied for a standard Rails dev position. After the first interview they wanted me build some sort of standard CRUD app in Rails on my own time at home with specific requirements. I already had a full time job, as well as a family, so even though there was nothing difficult about it, I told them I probably wouldn't have time. They gave me extra time, I said okay. A few days before it was "due" I told them I wouldn't have time. They said they understood, and asked when I could come in for the next interview. I guess it was just optional homework?
For my previous job I spent around a week to build a simple microservice from scratch, with login, jwt, email notification etc. - it's still not a full app of course, but it was very very time consuming (it was a bit out of my field), considering that I did not spend that time with family, hobby et cetera.
I know noone forces me to do it, but back then I really had high hopes for that job. Nowdays the maximum I'm willing to spend with an entry app is two to four days but deep in my heart I still find it a bit too much, but I still prefer this over live coding sessions in front of a bunch of random devs.
These are unreasonable asks for just an interview. A read of their CV and a 30-minute phone screen, maybe two with different people, is all that is needed to determine if a candidate is worth bringing onsite, unless the interviewer and their company is utterly incompetent.
I share your opinion Gaius, but the recruiters seem to think otherwise. In the last year I've been to seven interviews (by all standards I am considered a senior dev with an appealing pedigree) and this is not something unexpected at all.
The worst is that when they ask you to take three or five days off and work with the future team to see how you fit in (not for free though). I'm not too enthusiastic about it, but I already did it twice.
Can I ask which geography you are in? I am in England and that would be excessive here. I think the most I have experienced is 3 phone screens and two full but non-contiguous day’s of on-sites.
Hungary, but I had two non-hungarian interview processes as well, one for Berlin and another for a multinational company (for the Hungarian office though, but six interviews out of seven was held in English), same experience.
The latter one had a fairly complex entry task which took me three full days and I had six interviews after that - during the seventh I gave up and told them we are not looking for each other here.
My favorite interview anti-pattern: a position that stresses analytical decision-making, and then an interviewer that pushes you to give a quick, off-the-cuff answer to “do you think we should build X or not?”.
- select your candidates that maintain a public profile. You should get a broad idea about the skills and interest of that person
- have an informal interview to see if they fit your company culture and values
- then have them work on a small real project for less money
I'm a dev and happy to work on such a "taster" project. It's a win-win: I'm not working for free, the company gets a realistic picture of how I work and how I fit into the company.
This works naturally best for hiring contractors, but even then I don't understand why not more companies do this.
As a potential employee, I am just not going to do this. Nor am I going to complete puzzles (and I LIKE puzzles) or dance around in front of a whiteboard.
What I would be willing to do is work as a well paid contractor for 1-3 months to make sure that we are a good fit for each other. I'm not sure how anything other than actually working together could suffice.
I am fortunate enough not be concerned about immediate benefits like medical due to my wife's employment--I recognize that this might not be a good option for everyone.
I'm waiting for recruiting doctors like that. Oh, you are a surgeon with 17 years of experience? Cool. How about you make a surgery at home, record it, and send us a video?
More pathetic are only companies which have only one question at the beginning (without any further contact) "how much do you want to earn" and after that there is no negotiation. Sometimes I feel like I'm drowning in an endless sea of candidates. Oh, wait, no, the same companies publish the same job ads over and over for years. So maybe not.
Would you want to work in a world where all programmers have to be trained, licensed and regulated with the same rigor as surgeons?
Surely the fact that programming is one of the few 'white collar' jobs where someone with no education and no relevant work experience can get hired based purely on the fact that they've taught themselves the necessary skills and can do the job is a net good thing.
Yes exactly that. The problem is that anyone can stitch a leg back on. We need people who can stitch the leg back on and the patient not die 3 months later or the leg fall off again.
If you look at other engineering disciplines there are formal structured approaches to education and testing leading to chartered engineers etc. We could really do with the same.
At the moment we have certifications which merely prove that the individual can pass an exam. I can routinely get near 100% on these sorts of test without knowing the subject or having any experience.
The ability to build a top tier career from self-teaching is remarkable. It would be such a tragedy if this was stripped and replaced with a signalling based equilibrium run and/or structured by the government :(
Very true. These people literally don't know what they're talking about, or what they're asking for. They've thought their plan through about five seconds into the future.
Either that, or they're blatant featherbedders who aren't bothering to disclose their interests.
So you either use no commercial software at all, or you're happy paying a hundred times what you paid for it, and doing so in return for the privilege of running 1980s-era applications in 2018.
I want to live in a world when we have normal recruitment process for programmers. I have about 17 years of experience. And really my university grades are not important (yes, some companies want to see that). In all places where I worked I knew databases much more than all other people together, I organized training for them etc. Despite my database-oriented CV, I'm always asked basic database questions usually asked by someone who doesn't understand them, and then doesn't understand my answers. I also got some stupid IQ tests with multiple good answers (however exactly one was in the key).
I can talk about bad recruitment processes for hours. I just want to meet competent recruiters, and managers - just like they want to find a competent candidate.
Anybody know the names of the companies that do homework/project type interviews? I am looking around as an Android dev, and I would prefer those over whiteboard/coderpad.
Yes, I had that offered too, a 6hr Xp assigment with a selected employee.
But intentionally blew off the 2nd quiz (thus the interview) before starting the assignment due to a family medical emergency (I thought, WTF am I doing, there's more to life!)
Nowadays I find it's either:
a. multiple quizzes or HW assigment
b. a presentation (that takes 4-8hrs to make anyway).
c. and that github was a source of code eval, not anymore! ...cause everyone forks everyone's stuff (that's ok, by design).
I once interviewed with a startup that disguised their technical-onboarding as the last step of the interview process.
Learn three different frameworks, unlisted in the job description, and extend a github repo, suspiciously similar to their production codebase.
That they insisted what was easily a 10-hour assignment could be done in 30 minutes "if you're already familiar", made it seem like they were offloading the learning-on-the-job to before-getting-an-offer.
I dislike tech interviews as much as the next guy, but let me pitch my two cents from the startup world.
The company I currently work at has very few employees, and each new hire is a big cost. If a new employee does not work out, that can be a major dent in the budget. We still haven't figured out a great way to interview, and we are open to new ideas, as we have had hires who looked great on resume/interview who did not end up working out.
3 days of work is certainly an excessive commitment. But limiting the verbal interview to one hour then leaving the candidate with a a computer and a well defined problem that takes half a day to solve it's not unreasonable, in my opinion.
They have already committed that day to this interview, there is no abuse - show us you can solve problems and not give up after a couple of Google searches, like a good half of programmer-aspirants do.
Yea I think live exercises are the way to go. Or hire person as contractor for a day or two. One small startup did that with me and I thought it was brilliant.
We had someone come in and spend 45 minutes on interview problems e.g. produce an array of 100 to 1. You had free access to Google and were encouraged to use them for reference.
After each attempt we said no that's not right, and then at the end we were asked if we were going to use his code in our platform and if he should be paid.
Felt kinda bad for him, clearly didn't pass the interview test.
”..format that involves reviewing code together and talking about it. The process typically takes between 30 minutes to an hour.”
This sounds like a good idea to try. Much less time consuming than having to create meaningfully things from scratch. Also more relevant as you don't usually need to write code in front of others, but you might need to review existing code.
I did this for a data scientist position. It was obnoxious. Compounded by the interviewer believing his dissertation work was representative of the entire machine learning field. I didn't get that job. I'm more interested in the people I'll work with in hiring than what they know. to a certain degree anyways.
It really seems like the best way to avoid the need for interview coding is to have a professional license for software engineers. Prove to a licensing board that you are competent in your desired specialties.
I don't ever hear of structural engineers or medical doctors having to solve puzzle to get past their interview.
Why do you want to work for a company that gives you days of unpaid homework? If you're given such a task, simply state that you don't think you'll be a good fit for them, and move on. It is an employees job market at the moment, and every tech company has several unoccupied openings.
In my last job search I solved a dozen or take home assignment. For the one I finally got, I... refused to solve the assignemnt... and it is a very good one. Do everyone a favor and refuse, fixing the situation is in hands of anyone looking for a job.
I find it best to be asked that kind of homework after I have done my first interview. It usually means we both got a chance to pass and that the effort I will put will be seriously considered.
It's probably not for most people, but this is how I got my job as a C++ programmer in CAD software industry (Computer Aided Design) that lasted over 10 years.
i think its utterly insane for companies that are well funded to ask skilled labor to work for free. if you want the best talent, attract it. i have refused every interview asking me to work for free, and ive never had problems finding gainful employment. these companies are just losing out on great talent by being shortsighted. only desperate people will work for free, and they are usually desperate for a reason. my $0.02.
I would love to have the chance of doing some work for a company being paid a low rate or nothing at all to prove myself. That's how I incidentally got my best jobs. In the last interview the RH recruiter labeled me "arrogant" - God knows where she took that one from. Most probably concealed ageism.
Here’s how we did interviews at the startup I used to work for.
Phase 1: 30min non-technical phone screen. Usually we would have three people on our side of the call, but only one of us would do most of the talking. The goals were simply to describe the company and role to the candidate, gauge the candidate’s interest and ability to communicate, and allow the candidate to ask basic questions. We would discuss and decide whether to advance the candidate immediately afterward and usually notify the candidate on the same day.
Phase 2: in-person code review, typically 1hr. We’d give the candidate the option of providing us with a (working) code sample from something the candidate had written previously, or of coding up a solution to a problem faced by our actual product. While I was there we had a good mix of candidates picking each option. The “homework” problems were always drawn from the product, so they were real, but we also kept them small, so that an average candidate could have a working solution in an hour or two. And we’d give the candidate a week with no other constraints. Candidates could use any language, could research solutions however they liked, and could email us any clarifying questions they had. Whichever option the candidate picked, the goal was just to get our eyes on a dozen sample the candidate had written and could be expected to understand. At the on-site, we’d have the candidate explain the problem in the candidate’s own words, then walk us through the solution. The goals were to determine how well the candidate understood the problem and how comfortable the candidate was with explaining technical decisions. It didn't matter to us if the code ran or even attempted an optimal solution; we advanced some candidates whose code samples did neither, but who were able to talk us through what they had tried to do and explain why it didn't work.
Phase 3: working interview. We hired candidates for a day, paying $1,200–$1,600 for the day, depending on the role. The candidate would pair with a member of the team and work on some aspect of the product. The task was selected to be representative of the work the role required, but also simple enough that could be completed in a single day. At the end of the interview, the candidate got to ship the code to production (for which purpose we had a big, important-looking, physical red button). The goals were to determine how well the candidate functions in a working environment, how readily the candidate asks for help or conducts research, how the candidate responds to frustration, and how interesting and engaging the candidate finds the work.
It was quite a time consuming process on our end, but it was extremely effective at finding excellent candidates. We never made a bad hire the whole time I was there.
I prefer homework to timed, monitored programming sessions. But the task should never be a big task.
When i interviewed for a backend development position, the task was to set up flask with two routes, connect to a dB and retrieve a couple of rows when user visited the routes.
It was not hard even though I never worked with flask before, and it wasn't supposed to be hard either. It was a test if I had basic programming skills and that was it.
Exactly - a take-home test that will take more than ~2 (maybe 3) hours to complete is unreasonable. And it should be administered only after a preliminary phone screen.
I'm willing to give up an evening for a job that appeals to me and that I have a decent chance at landing. I'm not willing to give up all my weekday free time for a week or stay up until 3am. And I won't do either if you're not willing to first invest 15 minutes to make sure there's sufficient mutual interest to be worth my investment of hours.
If you don't respect my time before hiring, I assume you won't respect it after either.
The problem with any take-home test, ESPECIALLY one that you only expect to take 2 hours, is that you hand people an advantage who do have more time to spend on it.
That was what stopped us doing it, and started looking at reasonable tests that you can't prepare for, and can't spend more than the allotted time on. So that means we weave the testing into a 1-2 hours interview, and everyone has the same chance.
(we also document our process so you can see what you're getting into before you start)
Time pressure always make me perform a lot worse, on programming tests as well as iq tests.
I did an iq test that was not timed and got 10 points higher, which is quite significant and also much more how real life problem solving works as a programmer. I'm never under any time pressure in my job.
I don't love that approach. There are lots of people who can do the work well in a reasonable amount of time but don't test well under intense time pressure.
We do require candidates to use version control and (though we don't tell them so) we review individual commits to understand the candidate's approach and how they use version control, and commit timestamps to see how long implementation took them.
And the volume of candidates who pass both resume review and phone screen in our process is low enough that we're not usually comparing candidates to one another; it's "can you do the job or not". So relative advantages aren't such a big deal.
This is a very reasonable task, but most of the take-home projects I've gotten (for both frontend and backend) have been significantly larger.
The worst one was for Optimizely, who asked me to write an entire website (front-end using React, backend in Python) which will display a list of products from a database, including full tests. I determined that boilerplate and configuration would take me about 5 hours, with maybe another 5-6 hours of actual coding. They even offered to pay me something like $15/hour (literally nothing) to do this. I said no.
Setting up a web server is also a different skill than backend programming -- many experienced back-end programmers have never needed to spin up a new server for their job.
It's not insurmountable, but it's absolutely time consuming, especially when it's not a regular aspect of your job. I recently had to do exactly this for a prototype that I built and it took up a good 10-20% of my total project time.
Another way to look at this is, are you more concerned with testing a person's end-to-end or development skills? There's no right answer, but you have to realize that testing one comes at the expense of the other. If your company often uses brand new development stacks, it might be important to test a candidate's ability to do exactly that, but if your company has a framework in place that everyone uses, then it's probably best to focus on a candidates pure dev skills.
There are plenty of surmountable obstacles that are a waste of interview time. Why give a jr admin task that any teenager can do as a test for a professional programmer?
It does test for having a wider skill set that just memorising a text book.
Ill give you an example of a task that I got in my 2nd/3rd year of employment.
Came into work and was presented with an exotic (for the time) piece of hardware (A0 digitizer) that cost 3x my salary to be asked to.
1 Connect it up and get it talking to a PDP
2 Research and Write the code to allow the kit to interrupt the os and transfer the data to a second program (this is on RT11 which only allowed 1 process normally).
3 Work how to initialize and calibrate the kit
4 Was asked to go and chat with one of the senior engineers and work out how to use the kit to digitize the results from his experiment to produce a 3d representation of the droplet cloud - z axis was done by using a camera with a very tight focal plane
With flask it's just `flask run` which will cover all your development server needs. I agree if it's about setting up a production environment, but I don't see them implying that.
Homework is still timed. It's unavoidable. You have 3 days to spend "any convenient 4hrs, more if you need it"? Your competition spent 60 hrs on the homework, adding bells and whistles that imlddss the interviewer.
> adding bells and whistles that imlddss the interviewer.
Bells and whistles are likely to have the opposite effect on me FWIW. We've got enough work to do around here without inventing more work for yourself. 'Keep It Simple Stupid' as it were.
When I found myself out of work last year, I interviewed for a big auto company - their interview process involved an online coding session. They gave me a spec for an assignment and asked me to complete it in 20 minutes - I could pick Java or Python. I picked Java, and got the spec - it was ridiculously simple. I can't remember the exact wording, but it was something like, "write a function that will print out the text value of a number in the range 0-4, given the numeric value (so if you got an argument of 3, you'd return 'THREE')". Well, in my haste to complete the assignment in the generous 20 minutes they gave me, I... didn't read the specification closely enough. I started writing a program that took in a _list_ of numbers on the command line and output their text values. It took about 10 minutes to code it and sanity test it - then I thought to double-check the description and, "shit, I misread it". So now, with 10 minutes remaining, I'm thinking, "if I change this to fit the description, they're going to see that it took me 15 minutes to come up with a pathetic function that should have taken me 2 minutes." So, I started refactoring the whole program to _contain_ the function that they asked for, but call it from a main routine that would parse the input, call the function, and write out the results. With 19 minutes on the clock, I submitted my abomination. The recruiter was kind enough to call me back and tell me that they had decided they had "better qualified candidates in the pipeline". Oh, well...
Probably the ones that read the spec and delivered what was required. If you're going to gloss over something trivial, what does that say about your attention to detail?
Alternatively, it's someone that wrote a solution, re-validated it against the spec, realized it was not fully correct, and was able to correct it within the deadline given. Hiring isn't yet a science.
That's an entirely reasonable assignment that I would accept doing. Unfortunately, there are definitely way more time consuming projects being handed out regularly.
It's lazyness / entitlemant because we want competent hiring practices that are respectful of our time? Because when you have to work 40 hours in a week doing nothing but take home tests & quiz projects for companies you are interviewing for - for the 38th week in a row - and still not taking in any money for all this - it's more about affording food and paying rent.
The free work is the issue. Pay me for a day to do a project to validate my abilities and we're golden. I've done that at a few places, and it's a great way to go.
Yet, we do. Ok, then what about open source contributions (if they have them)? Well, sure, but we don't really know, for sure, if that code is their own or if they indeed truly wrote it, can we? So we can't count on it. It helps but it can't be an automatic "this person passes the technical" signaler.
So maybe the person shares actual code and walks us through it that they wrote, even if not open source? Again, we can't know for certain it is their own code and not some friend who wrote it for them 2 years ago that they claim is their own. Or was a teammate's, or what-have-you. So we can't count on that either.
Apart from getting a candidate to actually pump out some code, even trivially, we don't really have a way to de-facto verify that the person says they can do what they say they can do, short of a personal reference from someone, say, in the company already that knows and has worked with the person in the past.
I absolutely hate this practice of having to constantly "re-prove" to the next employer that someone knows how to write software. It's extremely redundant and yet we keep on doing it, with no real hope of actually making it better.