They once used the program to generate a top 10 list of factors predicting the probability of acceptance. ”I don’t want to share it, but it was fascinating,” Graham said.
This is understandable, but a pity nonetheless. I'm sure HNers would love to read about the factors. PG, can you tell how well these factors sync with the characteristic of a good founder you've written about in your essays?
It's at the level of spam filtering: individual words in application fields. So the insights were the sort I noticed when I first tried statistical spam filtering-- e.g. that ff0000 was highly correlated with spamminess (because it's html red).
Incidentally, the past tense in the article is misleading. We're using that software now. It's the product of a company in the w2012 batch. I don't think I can mention the name because they're not launched yet, but we're very impressed by its performance. SV Angel is using it too.
I'm wondering if it says more about the companies or the people judging them :)
Kidding aside, I'de be surprised if specific words correlated with the probability of success. So there's a good chance this is measuring only if there are key-words that please the reviewers.
I'm sure you've looked at prediction models for individual partners and compared them (or at least considered it, what with days only having 24 hours :)
It'd be fantastic to see you write about the results of that - maybe once the company has launched.
The reason this has me so curious? If the predictive models are not partner-specific, this could indicate that there are very specific things that successful companies/founders say and/or do. Sharing that might or might not mess with the models, but it has a good chance to increase the likelihood of success.
People have done simple word frequency analysis in various domains, and it's been effective -- from (housing listings, resumes, car listings) the takeaway I have is specific, quantifiable things seem to raise value, vague or weasel terms tend to lower value. I think the #1 word for housing listing value is "granite" (as in granite countertops), and various hedge/weasel words like "cozy, comfortable, ..." lower it.
Naively, I'd assume the same might be true of something like a YC app. Harj has said in many places that he turns to the "impressive achievements" section first (http://askolo.com/harj#4f74bc2be6c38a8e50000077). I'd assume that's fairly similar to a resume and would have the same statistical properties.
Bayesian, or other algorithmic approaches (what many call AI), on signals like word frequency can help with separating wheat from chaff. Rarely any better when it comes to such multivariate decisions. Given the uncertainty in startup success, perhaps a rough solution at scale would perform. Maybe that's the goal with YC, but at least on the outside YC has always seemed more like an elite private school that delivers on personal treatment and alumni network. I wonder whoa is building the equivalent of the public university.
for those interested, a nice explanation of the correlation of words to value / weaseliness in real estate is covered in a chapter of the book Freakonomics.
Chapter 2. (I googled "freakonomics real estate" and got a helpful google book search except. I love living in the future; Book Search and Google News are the best-executed Google products of the post-gmail period.)
I'm surprised nobody mentioned this (http://www.google.com/m?q=bullshit+bingo ) yet, because what I hear, sounds like advanced version of it. Consider selling online YC-Branded cards.
I can see correlation to
1. connected-ness to existing YC alumni via linkedin and facebook (why else would this be on the applications anyways, except to crawl)
2. site growth (via compete) of company URL
3. keywords: users,traction,growth,
4. age
5. karma score on here
any others?
I've wondered if this is a skewed statistic due to the "resubmit" feature. How many of these submissions were resubmissions, versus "first-time" submissions?
It seems more reasonable to do a last minute check, and update the app with any most recent progress (as I did) on the last day.
Is there a total published anywhere on yc with total applicant teams vs accepted during last two batches? I remember someone saying yc was more exclusive than HBS now. Is that true? (harvard accepts roughly 7% of applicants)
I've seen the 3% number mentioned a few times, and seem to remember hearing that this has stayed consistent for some time (so more applications but still 3% -> bigger batch size) but I can't find the original source just now.
As a current applicant, I'm glad to see that YC is thinking hard about scaling. With rising application numbers, I'm sure the noise to signal ratio gets worse and considering that funding a bad startup is probably better than missing a good one, I can only applaud any measures that help in that direction.
Actually, the cost of funding a bad startup are relatively low, aren't they? It's more the time of the partners and the resources of the alumni network, I'd personally be concerned about.
As another current applicant patiently awaiting the yay/nay email I wish you luck! I know it's got to be difficult for them to sort through all of the applications with so many great teams/companies. I tried to condense mine to make it easier on them but if you do the math (Number of questions * 180 Words) they still come out 3-4k words each! Now I know not all of the questions will fill 180 limit however it's still a lot of reading.
The valuations of both AirBnB and Dropbox are >5x that of the exit value of Heroku, and are still in play.
It's entirely possible either (or both) could be $10b companies (yes, this seems crazy to me at some level too, but it's consistent with economic reality as it currently exists). If AirBnB totally disrupts the hotel industry, it could be bigger still. If Dropbox totally takes over as how documents are stored, edited, and managed, of course it would be bigger.
These valuations can go just as fast as they come though. I know for me, and a lot of people, the concrete value that was determined because someone actually paid it holds a higher weight than the valuations speculative investors have given the lastest hot startups.
Pro: I think selling the whole company for $x is harder than selling 10% for $x/10.
Con: Big companies can also destroy value effectively (doesn't appear to be the case at all with Salesforce/Heroku, but...look at anything Yahoo has bought, and a bunch of things Google, Microsoft, etc. have bought. Skype, etc.
There's also some bias in the press here -- people reading mass-market publications actually might use (or at least understand) dropbox and airbnb; they have no idea what Heroku does. And, in tech/venture press, independent companies are more interesting because they might get bought; it's unlikely anyone will buy Heroku from Salesforce, or buy Salesforce, so it's less interesting to the press.
The value destruction is a good point, it just depends on where we are looking. Whether people consider a good sale price that was driven into the ground as more of a success than a lower sale price which goes on to continue to thrive and be a good investment for the acquirer.
I disagree entirely. A professional investor at the early stage is no less a valuation expert than an investor after IPO. This holds true even more so for companies who acquire to integrate or to acquire talent.
I think it is about the cashout event, if a YC company went to IPO, we could talk about what kind of value YC/ others cashed out at. The company paying to hire talent is also cashing YC/ others out. It doesn't matter how badly they over/ undervalue it they have paid that amount.
My disagreement came from how to think about valuations and investing:
I know for me, and a lot of people, the concrete value that was determined because someone actually paid it holds a higher weight than the valuations speculative investors have given the lastest hot startups.
When an early stage professional investor, invests in a startup at a particular valuation, they are just as qualified as someone in a later round. Yes, for the entrepreneur they do not have an "exit", but you do not know the other terms of the deal. In many cases entrepreneurs cash out at many rounds so a big valuation in an early round can in fact make them wealthy.
The line of thinking that professional investors at early stages are "speculative" is true, but not as much once you consider they are making more than 1 investment.
-I work at a post IPO investment fund, but I respect those that make tough decisions earlier in the lifecycle of a business and respect entrepreneurs that get that far.
When it comes to the applications and the fear of missing good startups, I think the biggest bottleneck is how many groups they can interview.
If I'm not mistaken, for W2012 they interviewed 180 groups and accepted 65. That's 36% acceptance rate, but it shouldn't distribute evenly. If the applications were properly graded and the acceptance rate didn't get very low for the lowest graded applicants who were interviewed, then they need to increase the number of groups they interview. I guess that's the case, since pg previously commented that for S2012 they'd probably interview 270 groups. The "problem" is that for every new batch they probably get more applications (and most likely of higher quality), so this might be just playing catch up.
Maybe one day an Anybot robot will perform a preliminary interview so they can scale :)
Alas we are in fact going to be playing catch up. We got about 50% more applications than last time, so interviewing 50% more startups will put us exactly where we were before.
Maybe this time we'll do a perfect job at the reading stage. Not likely, but we can always hope.
I saw your tweet that applications this cycle seem unusually good. Hence with 50% more applications and 50% more interviews, it seems that YC didn't scale when it comes to mitigating the risk of missing good startups. Unless of course the applications are graded perfectly - I'm sure you've been working to improve that. I'll let you know how you did on April 16th :)
3000 applications this time?! do you remember by name every founder from last batch? ' cause that's the line that I would draw in the sand re:scaling. Of course it is just me, as 500 startups prove that this criteria is arbitrary.
I'm waiting for "StartupAdvisor", a program written in Arc and trained on paulgraham.com/articles.html
Office Hours will consist of chatting with the program and (if necessary) pushing the "Request a Bag of Mostly Water" button if it reaches its (temporary) limits.
It seems like the academic publishing model would be useful here (and in fact that might be what they are doing with alums reading applications.) Each paper gets a few reviewers, which quickly determines the definite accepts and rejects, and the whole committee need only discuss the ones in the middle. The way partners and alums are structured in YC, it would probably be ideal to weigh reviews accordingly.
There will definitely be some noise, but one would hope things will shake out as they should most of the time. At least that's what I am hoping.
The academic research community has certainly solved this problem to a large extent, eg. reviewing and program committee meetings at elite conferences such as SigComm and SOSP, or to demonstrate scalability, conferences such as Infocom which get thousands of submissions and still maintain high quality in their accepted papers.
One big difference is in the incentives for a PC member at a top conference vs for a YC partner. The former needs to maintain his and the conference's status/reputation by ensuring the papers are of top quality. He doesn't care that much if some great paper is left out. Whereas in the case of a YC partner, the biggest concern is to ensure not missing out on spotting the next Facebook. This is why in the former case, past reputation and being part of the clique is very important, and in the latter case (YC) it is not so.
Ugh, completely disagreed. Do we really want startup funding run by Statler and Waldorf? Academic reviewers tend to be looking for ways to reject your paper, not even trying to give an honest evaluation.
A surprising amount of peer review is actually done by lowly graduate students who are assigned papers by advisors. I really try to give an honest evaluation :) [and my experience is most people generally do]. However, like it was mentioned above, the penalty for rejecting a good paper is really low.
The reason we want to grow YC is (a) to see how big an effect we can have and (b) as PB says in the article, because it's better for the startups.
As for not being able to support 1000 startups, maybe we couldn't. 1000 is a lot. But if you'd asked me in 2005, I would have been skeptical about the idea of supporting 66 startups at a time, and yet the last batch turned out better than ever. That's the thing about scaling. You can always do better than you expect. I'm not saying there's no upper limit, just that experience has taught me to reserve judgment about where it is.
I don't think there is any shortage of problems to solve / companies to start, so I say, keep scaling.
Higher education is in need of reinvention. YC may be the most successful experiment to date towards that goal. I would really love to see a quantified ranking of top business schools by the total value of companies created per student-month in the last five years.
But don't you think that YC is a business school, by any definition of the word? And wouldn't it be interesting to do a comparative ranking alongside traditional business schools?
There was a time I wished I had gone to HBS or Stanford GSB. While those remain fine institutions, wouldn't any young nerd today prefer YC?
> But don't you think that YC is a business school, by any definition of the word?
No; it's not a school, and it's for entrepreneurs, not businessmen, which is the heart of the distinction I made previously.
> And wouldn't it be interesting to do a comparative ranking alongside traditional business schools?
No, because there's no basis of comparison. You can be a successful MBA without ever in your life working for a company with less than 10,000 people or having more than a token amount of equity compensation. Some MBAs prefer it that way.
> There was a time I wished I had gone to HBS or Stanford GSB. While those remain fine institutions, wouldn't any young nerd today prefer YC?
Any "young nerd", as far as I understand the term, would prefer a succession of stable job titles with "engineer" on the end from their first job out of college to retirement to an MBA. Now it turns out most of these engineers can be more valuable as founders, which supports the mission of YC. If it wasn't for the bubble and the social proof surrounding YC it would have never attracted the business-school types and you'd never have this confused association in your head.
I suspect our differences are mostly semantic. I consider all entrepreneurs to be businessmen (although not the reverse, of course). You may have a different definition. That's ok!
Here's the way I view it:
- Look at YC as a transfer function. In go individuals who want to learn how to successfully start a business. Out go many individuals who do indeed succeed starting a business.
- Look at HBS at a transfer function. Difference is that the individuals spend a lot more time inside the black box, do a lot more "makework", usually end up with debt not equity, and fewer of them succeed at starting a business. Still a great institution! And clearly many HBS students want to go work for Goldman Sachs, and go on to have great careers doing so.
I'm just saying that higher ed needs to be reinvented, and that reinvention will necessarily happen in ways that dont look like conventional education. Like YC.
To whatever extent business schools are replaced by something like YC, it'll be to the same extent that large businesses are replaced by smaller and more granular businesses generating massively more revenue per employee. That was more or less what I was hinting at with the battleship/dive bomber analogy. It's not that YC is taking the place of business school, or that the types of people who went to and thrived in business school will be attracted to YC. It's that creative people like hackers and designers will learn just enough business to handle it themselves and the types of people who get MBA's will have to learn to work for a living.
If they can scale, they can create more value (for themselves, for the alumni network, for founders in general). If they successfully can scale it, there'd be minimal dilution (and, as they mention in the article, some additional network benefits to the companies/founders).
What if the number of quantity startups has grown though? If last batch they had 66 startups that were judged as good enough to be in the program and this time they see 100 that make that grade, they can take 100 and quality shouldn't drop off.
It sounds like they have room to grow on the quality scale. They miss some good ones, as a recent HN post on a successful reject (who just missed the cutoff) showed.
This is understandable, but a pity nonetheless. I'm sure HNers would love to read about the factors. PG, can you tell how well these factors sync with the characteristic of a good founder you've written about in your essays?