Most startups today (those you've actually heard of) generally do 2-4 phone screens which encompass a technical interview. You can utilize Google docs and watch them code.

Take the feedback of everyone who interviewed (your team) into consideration and either offer the position because you feel they are motivated and technically sound or you can fly them in for a 3-4 hour person-to-person after a phone screen or two.

This is what Slide, Facebook, Google, etc. do. None of them mandate puzzles, seems like they use them as a way to "Wow" them.

In the past, I've seen companies give "quizzes" they can do in a limited amount of time. Generally 10-15 question quizzes as a quick way to judge, without wasting your own time, if they are competent enough but hopefully a resume is enough.

Disclaimer: This isn't necessarily my opinion of how it should be done, just how companies that have been around the block do it.

Etherpad is a nice alternative for google docs fwiw (especially if the code sample is in javascript).

This is great advice.

Incidentally, depending on the position, during the interview Google has questions that are similar to puzzles, but relate more directly with the position. e.g. instead of estimating how many gas stations there are in the US, it will be estimate something to do with a Google product.

Any thoughts on video chat interviews instead of flying people out (ah the startup wallet calls again)?

And... what is your opinion of how it should be done?

You should only fly people out if you are strongly considering them. You're going to pay this person $20-$35/hour all summer, what's $300-400 just to make sure they are exactly what you want?

You do not have to fly people out, Google/hi5/Ning don't seem to do this but Slide/Facebook still do. It's clearly how convinced you and your team are of this person. You can try and make video conferencing part of the phone screen process of course.

I am no expert on how interviews should be done and it depends on the kind of intern you want: Someone who can learn fast? Someone who is already skilled with your stack? Someone purely motivated?

Ideally all, I'd pick one and base your questions accordingly. Companies I mentioned above let potential hires code in any language and simply check that they know the fundamentals: recursion, data structures. Gauging their passion/motivation/drive is done through their resumes and side projects like others have mentioned.

My personal opinion, you want to pick someone who is capable of learning most things without asking you otherwise it's going to be a struggle of he/she constantly asking you questions.

Here's my personal advice. While there are plenty of companies hiring strategies to look at to get ideas on how to accomplish getting the "best person" I would first take a step back and evaluate on what's important to you and your team and then filter based off that - be creative if you have to in order to get what you want.

While Frocer touched upon a majority of what we did, I'll throw in my interviewing experiences. We are partly in the video game industry so my #1 requirement was the applicant MUST have video game playing experience (of the 'core' variety) and the more the better. I just couldn't see anyone who only played Wii Sports or Bejeweled on the iPhone could comprehend what we do and therefore unable to help the team in any sort of discussion beyond the technical. So that was what was most important to me and that was what I filtered on and not surprisingly, I had a good feeling immediately after each interview on whether or not I wanted them.

My interview consisted of no stupid puzzles or whatever cause it was of no use to me. Show me you are technically proficient (of which a lot were, otherwise why waste time applying for the job) and then show me you understand video games by carrying an intelligent conversation with me about let's say your favorite xbox 360 game, which smash bros was the best, or why world of warcraft is both the greatest and worst game ever, etc. You'd be surprised on how many people couldn't do that.

Also I'm seconding the reference/portfolio thing. The more you can find about their past, the better the idea you will have about their future.

I'm curious. Did you ever try hiring someone who was an occasional gamer (the Wii and Bejeweled gamer) and they didn't work out or did you just assume that they wouldn't be a valuable addition to your team and automatically exclude them? The reason I ask is because I wonder if having some diversity in the level of gameplay ability in your staff might actually produce better games (i.e. ones that mere mortals can play). Of course, I have no idea what your target market is, but its pretty clear to me that occasional gamers are a largely untapped market that the large game companies are mostly ignoring. There's no doubt that Halo, Call of Duty, WoW, etc. sell a lot of units, but there are also a lot of people (I am one of them) who have absolutely no interest in playing those kinds of games for the very simple reason that we have other things to do with our lives. We still play games, but we avoid games that require a significant time investment in order to achieve competency at gameplay. For example, I do not play first-person shooter games at all - the user interfaces are too complex to learn when you're the sort of person who only plays games for a couple of hours on Saturdays and may go for a month or more between sessions of playing any particular game. Battle for Middle Earth II is at the upper limit of interface complexity that I am willing to tolerate before I give up on a game because it seems like I'm spending more time trying to remember the mechanics of the controls than I am just having fun playing. I tend to gravitate towards simple turn-based or mouse-click slash & hack RPGs (Pool of Radiance, Dungeon Siege, Diablo II), sim games (Sid Meier's Civilization, Masters of Orion II, The Sims), and simple arcade or puzzle games (Dig Dug, Arkanoid, Minesweeper). If it takes more than a minute to relearn the controls of a game when a month has passed since the last time I played it, odds are that I'll get frustrated and stop playing that game and avoid others like it. Similarly, if the default game AI is tuned to perform at the level of a hardcore gamer and there's no way to turn it down to a level that a perpetual n00b like me can compete with, I'll ditch the game and never buy another one like it (many real-time strategy games are guilty of this offense).

Sorry if there was some misunderstanding - what I meant by a 'core' gamer (and I hate this term) is one who doesn't play hardcore, but someone who is familiar with and plays in the more traditional gaming industry (which is our target market). Whether they be a casual gamer or not, I don't care, but if they can discuss all those things you just did in that huge paragraph and carry a conversation with me about them then that's what I'm looking for.

The problem is, trying to talk to someone who's only knowledge of gaming is Wii and iPhone about traditional gaming tends to go nowhere in my experience, whereas talking to a traditional gamer about the Wii and iPhone still leads to interesting conversation and ideas as those "casual" games are pretty much small extensions of traditional gaming.

Here's my advice as a two-time former intern at two different places.

As a small startup looking to hire hackers, you obviously want the best. You want experienced hackers who already know how to code and perhaps have experience on a project before (e.g. open-source). Of course, you want the brightest.

But this is always a challenge because the smarter, more experienced, and more valuable a hacker is, the rarer they are and--odds are--the less likely they are willing to work for you because they're less desperate for an internship; they can afford to shop around, or may even just decide to do their own projects for a summer.

So here's how you can try to attract those people.

1. Pay decently. A lot of companies like to pay interns nearly nothing these days (or in some particularly obnoxious cases, actually pay them nothing). This of course doesn't mean pay them as much as a full employee, but it does mean they should not feel insulted by the amount of money you're offering. If they could earn more than 5x your pay as an independent contractor, you're doing it wrong. While not all hackers do internships for the money, it's definitely a huge plus. And if you don't think you're going to get enough out of your intern to justify the pay, don't hire them; you should not hire an intern unless you have no doubts about their ability to produce significantly more value through their work than you're paying them.

2. Deal with the interns' practical issues with regards to working for you. For example, if they're coming from out of town, deal with their housing and flights. Hackers dislike dealing with bureaucracy--you'll save your intern's time and effort and sanity by dealing with these issues.

3. If your interns' time is so much less valuable than your employees' time that you cannot spare significant employee time to helping the intern with what he is doing (in the same way they would help another employee), don't hire an intern.

In terms of picking an intern, often you have to actively seek them out to be successful, especially if you're looking for the kind of hacker that doesn't go around applying to every internship he sees. For example, if your company makes very significant use of open source software, you can look through those open source projects for potential interns.

That's great advice. What about the type of responsibilities? It seems that what we've heard is just being able to make an impact and being able to show exactly what they've done to future employers.

I think a good intern should be able to take real responsibilities as well. They shouldn't be exposed to the worst of the firefighting if it happens (they should still take part, but don't put them under the worst of the pressure; that's what your employees get paid for).

This means a number of things:

1. Give them at least one project to work on with specific things that need to be done. Don't necessarily have a hard deadline unless there actually is one for the project; as long as your intern has work to do he clearly can't justify goofing off, so always make sure he has work to do.

2. Have at least one project available for him to brainstorm on. Any good hacker is going to have loads of good ideas; the best way to waste these is to throw the hacker onto a project that doesn't need them. Your hackerf is not just a codemonkey. When you have your hacker dropping by your office or pinging you on office IM about a really cool idea he had for Project X, listen to him; it just might be a good one.

3. Ideally, hire an intern who has a skill set or knowledge base that nobody in your company has. I don't need to describe the value of this.

Best places to post- Aside from the HN startups, so far, we've found the biggest pickup to be from Craigslist, Snaptalent, Startuply, Startupers, and JobScore. Others have said we're likely going to work off referrals (our personal networks).

I feel the economic situation didn't make the hiring process easier, but actually much more difficult. We thought it would be easy since there are tons of talents being laid off from top-notch tech companies, and what happened was the exact opposite -- there are THAT many more resumes out on the market, and it takes THAT much more time to find the perfect fit. Keep in mind because we were still a small team, every hire must has the right balance of cultural fit, technical abilities, and entrepreneurial spirits. Finding good guys who can get the job done is easy, but finding the great guy who can take our product to the next level is much more difficult.

After going through thousands of resumes, and conducted probably 100 interviews, we finally brought on-board our first couple of team members. Some tips to share from my experience:

1. Don't bother with job boards, such as Craig's list, Dice, etc. In this economy, there are too many resumes floating around, and reviewing them is an incredibly time consuming process when you can spend the time building the product. Negotiate a low rate with a recruiter and outsoruce that process. We wasted over a month using job boards, and got tons and tons of junk resumes. Probably only 1% of the candidates we got from job boards were interview-able. We got fed up, so we engaged about 5 recruiters simultaneously, and within the first week we got the perfect guy. If you really want to use job boards to try your luck, I had the best experience with Startuply though it still didn't result in a hire.

2. Be extremely selective, if there's something that doesn't seem right, it's probably your gut telling you he/she isn't the right person. For hackers -- my favorite question to ask is "when did you first started hacking?". I have found most great hackers started hacking when they were very young (middle school to high school time frame). I found most people who started hacking in college is because they were forced to, not because they are interested in it. Of course this doesn't apply to every case.

3. Do reference checks. In fact, ask the references for additional references. A candidate can master the art of interview, but reference check may tell a totally different story.

Also, I don't recommend technical puzzles. I recommend you to just ask a technical challenge you are facing right now. e.g. How would you architect JamLegend? How do you solve so-and-so problem? etc. While puzzles can test brain power, I think it's much more practical to have them solve a real problem that you face on a daily basis specific to your product.

Hope that helps, by the way, if you get referrals from personal network you can probably skip a tons of things I mentioned above, and they are usually the best early hires. Good luck with your hiring process!

Just want to add another quick note -- this is actually something I learned from Mike Cassidy.

We like to do all of our interviews in 1 day, and when the candidate is in his/her last interview, we will all gather and make a decision right on the spot. If we like the candidate, we will make an offer when the candidate is on the way out. You can say this offer stands pending all reference checks went through. But this serves two purposes:

1. You show you have strong interest and is willing to make decisions quickly.

2. You lower the risk of the candidate going to another interview and receive another on-the-spot offer, and take that offer on the spot.

Great candidates are very hard to come by, and they are in very high demand. So realize your offer won't be the only one they receive. If you are unable to make a decision on the spot, he/she is probably not the right person.

I thin he said to "Hire Fast, Fire Fast" something someone just said to me.

What do you think is wrong with technical puzzles?

Nothing, I just think it's more practical to ask the candidate to solve a problem you are currently facing, rather than trying to solve an arbitrary puzzle. You can also gauge how well the candidate would work in your team when you are bouncing solutions/ideas off each other on a real problem you are facing. It's just a personal preference.

I generally agree with you. You're right that the economic environment can make hiring harder. There are more people looking for work, but the average quality of applicants is probably lower on the whole. The "extra" people are those laid off, who will be weaker on average than the general pool, and a lot of the talented people (who have a choice) are staying put until the hiring environment improves.

This: In fact, ask the references for additional references.

... is actually pretty damn scummy. The candidate has no way of knowing who his shadow references are, and could get unfairly burned and left with no idea why. It'd only be morally acceptable if you asked the candidate for permission to call these second-degree references, but that would defeat the purpose of snooping.

You are absolutely right, and I apologize I didn't clarify. When you ask for second degree references -- always ask your candidate for permission before you contact them. In fact, when you ask the first degree references for additional references, you should make it clear you will get permission from the candidates.

First degree references usually have nothing but good things to say about the candidate, but you will get a much much more accurate picture from talking to someone who isn't on your candidate's top list.

Selling It-

Joel Spolsky's Field Guide to Developers (or what a developer wants) - http://www.joelonsoftware.com/printerFriendly/articles/Field...

Joel also has a book on the subject: http://tinyurl.com/dx4gyf

Interviews- two resources we found here:

http://news.ycombinator.com/item?id=89615

Joel Spolsky's Guerilla Guide to Interviewing Developers: - Version 3.0: http://www.joelonsoftware.com/articles/GuerrillaInterviewing...

We honestly don't know what were great interview questions or were terrible ones. For example, google asks "can you teach us something?" while James Hong of HotorNot said once that he would have loved to asked the question "If you had to rob a bank, how would you do it?"

For Fun-

Anti-rockstar thread: http://news.ycombinator.com/item?id=255587

The ultimate secret to hiring: http://www.netjeff.com/humor/item.cgi?file=hire.txt

Puzzles:

We've been told to give simple puzzles during the interview (intro CS) and to not make puzzles a requirement to apply. Nevertheless, here were some prominent ones we found

- Justin.tv puzzles: http://jtvproblems.weebly.com/ (problems for flash developers too...) - FB Puzzles: http://www.facebook.com/careers/puzzles.php - Meebo puzzles: http://www.meebo.com/jobs/#web - ITA Puzzles: http://www.itasoftware.com/careers/puzzles07.html - ICFP contests

Those links are all to companies who have the puzzle sent in with the application, not given in the interview. Also, on that note, Joel Spolsky advises (http://www.joelonsoftware.com/articles/SortingResumes.html) that this will reduce the number of applicants you get, but not increase the quality. The reasoning is that, since great programmers will have options, including the problems will be equally likely to dissuade good candidates and bad candidates from applying and what you're left with will be desperate candidates that are willing to jump through any hoop. Perhaps just one take. Anybody that has used the with-application puzzles, how did they do for you?

As for in-interview questions, Joel also had some good advice (http://www.joelonsoftware.com/articles/GuerrillaInterviewing...), use two types of questions: (1) softballs, and (2) complex questions requiring a grasp of multiple layers of abstraction, like C pointers or recursion. With type 1, he recommends caring more about how quickly they do it than if they eventually get it right. For type 2, the goal is more to see how they think, and you're free to help them along with the trivia that is easy to forget but easy to find on Google in 15 seconds.

If you are at the point where you are communicating with people (via phone screen or full interviews) make sure to ask about any side projects the candidate might have. I cannot stress enough how many strong candidates I have seen with this being their leading indicator. Do not directly ask "Are you working on any technical side projects?" because they will know immediately what you are asking for and they will give you something that you want to hear. Phrase it more from the standpoint of what they do in their downtime or even what they enjoy doing other than writing code. That is where you should be able to see what kind of drive this candidate potentially has.

My old article, still somewhat useful, imho:

http://inter-sections.net/2007/11/13/how-to-recognise-a-good...

I like your article a lot. I think passion is what really separates the 9-to-5er from someone who genuinely cares about programming.

Dealing with Resumes:

Joel Spolsky's Sorting Resumes - http://www.joelonsoftware.com/articles/SortingResumes.html

I have hired hundreds of people during the course of my career.  In my experience, the most effective way to make solid hires consistently is to a) conduct an analysis of the candidates history of accomplishment and b)evaluate brains.

I understand you are currently seeking summer interns, no problem. Today, young people that are proven achievers have much to examine:

1)  Projects they've built or been a part of--just because they were curious if they could do it.

2)  What are they passionate about:  who have they read/studied in the area?

3)  Are they aware of market leaders in your sector and can they discuss what they believe is right/wrong in their model.

4)  Extra-Curricular's (speaks to buy-in of culture)

5)  How do they define providing value/work ethic?

The above represent top 5.  Be certain that when you find a candidate you'd like to hire that you can articulate the value proposition you offer to them. Be prepared to demonstrate how being a part of your team will benefit them.  Don't bullshit, anybody smart enough that you want to hire can smell it a mile away.  Be Good.  Absolute best of luck.