I agree to a point. If they end up being reasonably successful in predicting this information, and they can convince enough people of this, then someone's willing to pay for it. The latter problem is probably the tougher one. If people believe it works, you could sell to, say, business travelers--or more obviously someone like Fedex.
Business people who fly a lot would be willing to pay quite a bit for a service that they could use to save time and travel headaches. That seems to me to be the niche that they should focus on near term.
One question that's not clear to me, which they could offer in retrospective analysis, how often can they make a recommendation that actually improves your arrival time. For example, if an airport is blocked by snow or thunderstorms then switching to a different flight that goes to the same airport may have no effect. What I didn't understand were the kinds of problems that they could not only detect, but offer an alternative that had a higher probability of arrival within a desired window. I don't see that explained on their website either.
Nothing, which I guess is kind of the point. I'm still wondering how they plan to monetize or get acquired. I would've thought they'd've been one of the best acquisitions targets out there.
OP provides a nice description of FlightCaster's business issues and technical approach, but exactly where is this "prediction of the future of development"?
Great Article! I didn't understand how flightcaster worked until a couple minutes ago. I thought they just scraped airline sites faster. :)
I wonder in what other areas this tech could apply? Financials comes to mind first, but perhaps something less saturated. Groceries? Gas prices? Perhaps something with less supply/demand pressure.
Decisions in airlines are made based on a set or rules and conditions. You will be able to predict flight schedules based on published flight times, weather, airport traffic and similar variables. Not different from stock prediction or insurance risk assessment, and similar industries heavy on mining data, both historical and real-time.
Their platform runs on Amazon's S3 which has an slightly different usage pattern than their application needs. So they overcame this limitation by implementing in-house measures to augment the platform (instead of, say, scrapping S3 and building their own storage stack and hammering scalability and distribution on top of this later; in another words, a quick fix saved them both time and money.)
They also layered a higher-level API, Cascading, on top of Hadoop and S3. Cascading allows them to interact with underlying storage and network at the function call interface. Essentially, their code doesn't look any different than a straightforward serial desktop application. This is bottom up programming; instead of writing a lot of code in a big language over a weak platform, you build up the platform, make it beefier, and program it in a simple domain-specific language (though it's not unusual for big chunks of software to be generalized and moved away from the application core; most large applications have a huge library dependence, both 3rd party and custom built. So without even a clean API and specialized syntax, software can be reused in this bottom up fashion [See: Greenspun's 10th Rule])
The best analogy I can think of for bottom-up programming is spoken language. Experts in a field, say doctors or programmers, are able to speak with each other in technical terms. They have enriched their mutual language and they can communicate with ease. However, if a doctor or a programmer would speak to a non-specialist, a lot of the vocabulary would have to be "expanded in place"; acronyms would have to be spelled out, concepts would have to be explained, and analogies and metaphors would take the place of direct communication. So even though the gist of the conversation might be understood by the non-specialist, he we would not have a complete picture until he learns the subject and builds up his own vocabulary.
Programming on top of a weak platform is like having a novice for a peer; you will be doing more educating than collaboration.
[Edit: I really thought the parent was either a non-english speaker or a non-programmer; checked his history and he is both :-/ You owe me 20 minutes of my time, jorgem.]
