Hacker News new | past | comments | ask | show | jobs | submit login
People who have seen their work destroyed in a failed rocket launch (theatlantic.com)
158 points by yarapavan on March 23, 2017 | hide | past | favorite | 63 comments



Interesting article - I was always curious about how those people feel. Though sadly, "life's work" in the headline is pure clickbait - two years of preparations is not life, and the explosion left enough of that "work" on the ground that they could reconstruct the experiment in two weeks.


The people who work on the James Webb telescope or on some planetary missions will definitely have lost their life's work if something goes wrong. I am not sure I could take the pressure.


I knew a guy that worked for a huge computer company that for 10 years never saw a product ship. The projects were always cancelled before it got that far. Instead of feeling pressure, he pretty much stopped caring.


Been there. Done that. For much shorter period than 10 years, but still -- as long as I get paid and interesting stuff to do/research, why should I care if it goes to production or not?


I cared and cared enough to quit. Worked my last (and first real software) job for about 5 years and quit a month ago because of this kind of thing, at least in part. I was getting good at making the v1, but never learning how to really do anything beyond that. And figuring out how to ship an MVP is the least important part, so if that's all you're learning you're missing out on what will make you successful at a place that's actually successful.

Plus, I'll just generally disagree with you, sitting there making stuff that falls right into the garbage can is incredibly depressing. What's the point?


> sitting there making stuff that falls right into the garbage can is incredibly depressing. What's the point?

A very journey vs destination thought. If the journey is your goal, then you already have the "point" you're looking for. If, however, the destination is the goal, then you are forever reaching out for your next "point", and the moment you reach one, you'll be reaching for another. Always looking for your next point that will never satisfy you.

The preeminent philosopher Jake the Dog had an interesting comment about this sort of thing:

https://www.youtube.com/watch?v=fFsQprx5pQM


I get where you're coming from and hadn't really thought on that aspect explicitly yet, but my first thought is that rather I'm missing half of the journey that I want to have.

To torture a metaphor, I like hiking, my wife and I have a goal of hiking the whole SHT [1] in bits and pieces over the course of 5 years. We've found parts of the trail that are absolutely amazing to hike, we could just hike those sames bits over and over again because we know that they're amazing and not risk having to hike the bits that are just a mowed bit of grassland (which is uninteresting to hike), but then we'd be missing out on the whole experiance and not only that we might be missing out on even better sections that we haven't encountered yet.

I feel the same way about how I'm progressing in my career. I could keep doing the same bit over and over again and I'd probably be pretty damn good at it, but I just can't let myself. I'd rather keep exploring, keep experiencing not only what's new, but what's old-hat to others but new to me, and see if there's something out there that I'm even better at that I enjoy even more than what I already know.

[1] https://shta.org


It depends on the perspective. If you spend your time learning, doing new things... That is incredibly enriching for the individual. However, those experiences are usually made possible by others. If you venture to do these things, create new stuff, only to have it trashed or never see the light of day, should it ever have been created at all? Things happen and projects fail that is true. But my goals are to ensure that the things I learn and apply can give back to the community and lessen the suffering of others. If I learn things and keep them to myself, they may as well not be learned at all.


"What's the point?"

The point is that I get still acquire new technical skills and parts of (failed) projects might be challenging and interesting, even though complete project don't result in success (in business meaning). For other points see my response to a sibling comment.


Fair enough. I guess it also depends on how far the projects actually get before they get canned. I assume part of my problem is that I was always working on my projects alone or nearly alone (from a technical contributor perspective), so customer feedback was the only way I was going to get any input on the code I'd produced.

Coincidentally, I was also an embedded dev.


> And figuring out how to ship an MVP is the least important part, so if that's all you're learning you're missing out on what will make you successful at a place that's actually successful.

That's actually a very important part and many projects don't even make it that far. This is an extremely valuable skill to have, the actual running of a company beyond that point is a lot simpler and interesting. Don't underestimate this capability you have.


> the actual running of a company beyond that point is a lot simpler

I respectfully disagree. I think that the problem of maintaining a project in the field, providing support, taking feedback and making improvements, and managing a pipeline of new developments alongside sustaining efforts is just as difficult of a problem as getting to an MVP. Both sides take vastly different skillsets, and require people with very different backgrounds to compromise and shift in leadership.

That said, I completely agree with your point that the skills of getting an MVP out the door is significant and worthwhile. I know I am on that side of things myself - I enjoy the initial development, brainstorming, hard work, and hard choices that come along with that piece of a project. I jus think that the other side of the coin deserves equal representation.


> Plus, I'll just generally disagree with you, sitting there making stuff that falls right into the garbage can is incredibly depressing. What's the point?

The sad thing is when you get old enough you start seeing even the successful projects you worked on show up in the 25-cent bin at Goodwill. (Maybe not so literally anymore, since software isn't bought in boxes, but you get the picture.)


> What's the point?

Earning a paycheck. Getting passionate about coding at a company is setting yourself up for disappointment. Save the passion for your personal stuff, or start your own company.


What is the point? It is fun. You know there are monks who create and paintings, and then immediately destroy them... What is the point?


I think most people have an innate desire to have an impact on the world, however large or small it may be. If you work on something for ten years and nobody ever uses it you've made no impact on humanity other than taking. I get the most satisfaction out of life by creating things and having other people use them.


Why are people down voting this? I understand that there are people out there where their satisfaction with life happens to come from things outside of work. But, I'd hope that those people also would understand that there are people that happen to have their passions line up with their careers, especially on a place like HN.


I've never had such feeling, but yes, I agree that for most people that's probably the case. Work is just work for me, in spite of I like it a lot. I realize myself through different activities.


>why should I care if it goes to production or not?

Having other people actually use the stuff is the best part for me.


Well, it probably depends on your area. I work with embedded systems, so I rarely see "people actually use" my products anyway, even if the project is successful and goes to the production as these are mostly niche products and/or part of a bigger system. If I was webdev, that might be different though..


Let me tell you what's "depressing" about web development:

You finish your part, the web site is done (other team members get their stuff done), you ship to production, it goes live, the client loves it and uses it. Users love it and use it. All is great.

Over a year (or more) updates and additions are done to the site and code. Some mods here, a few CSS tweaks there, a whole new module or some new tool for users or the back end. All is going well.

Then one day, your team isn't needed any longer - either the customer is completely satisfied with the work, and no longer needs or wants any further updates (and what updates they do need, your back-end you developed can handle it for them). You or your employer never hear from them again, or if you call, they just say "we're all good - the site is running great".

Eventually, you note it in your CV and your portfolio and move on.

Then, one day, you turn to look for another position elsewhere (maybe years later and an employer or two in the future), and decide to go back thru your old portfolio, both to catch up on what you did, and what it looks like, and also so you can show a future employer or recruiter what you worked on (at least from a user perspective - because you likely don't have back-end access any longer).

...and it's gone. All gone. The client decided to do a complete system update, using a completely different design firm, etc - and the site looks nothing like what it used to. Your work might as well not exist at all, and you no longer work at the old employer, and you don't have a backup of the code or site (because it was a work for hire or such - legally you can't have a copy). Gone.

As far as anyone is concerned, you have little to no proof you actually worked on anything.

It's a very frustrating thing to see, which is why it is so important to have a portfolio of personal projects and other things to fall back on (and best in a github or other repo).

Still - that only softens the blow somewhat; I know that out there have been a couple of small things I worked on that I would love to be able to show in the future, but which are completely lost (my old employer went out of business not long after I left - then again, we didn't part on the greatest of terms, either).

At one employer (whom I still use as a reference - they were good to me when I worked for them, but let me go in a downsizing prior to the sale of the company) my software is still in use; don't ask me how - it was written in VB6 and used an Access 2003 MDB backend to serve about 50 users on a daily basis; when they let me go they said they were looking into other options, but they never found one - it has had no updates since 2004, yet still runs and works fine, according to my former supervisor (who's now a VP there). I can't do much more than say what it is to a future employer, though, because it was an internal-use only "product" that was never sold outside of the company. It was basically a custom CRM/issue tracker/billing/reporting/file-packaging and distribution solution that grew in an ad-hoc manner - tightly integrated with the company's business rules; when I left, I was working on transitioning the back-end to a PostgreSQL database (via an ODBC connector), reporting used the Actuate reporting suite (the company was a VAR for them), and the file management stuff used Visual Studio as the back-end. On top of that, I built a small scripting language interpreter (today, I would do something different) to handle online updates and installation (for a time we handled updates by going around with a CD to each user, with the new system, we could target updates per user, per update - we could also roll back things - all from within the application). There was also a similar install system for the file updates sent to clients (their main product was written in DB/C - a COBOL variant - and the file updates were packaged based on the repo in VS - turned into a ZIP file that could be installed using a custom installer I wrote that was similar to InstallShield of the time).

Anyhow - I would love to be able to demo it to potential employers, but since it was an internal project, I have nothing to point to in order to even prove I worked on such a system (all I can tell them is to call my former supervisor as a reference - fortunately he still works there, and they still use the software). I was let go in 2004 from that position - so I have one product out there that is still running for 13 years now (fairly amazing to me - I keep expecting a call from them to port it to another platform - but since Windows 10 still supports the VB6 runtime, probably not anytime soon).


This isn't just true for web development thought. If you worked on building some piece of hardware, a few years later you'd be hard pressed to see any of that hardware still around. I guess you can keep one on your shelf as a memento, but it's not like it would work any better as a demo.


I did some hardware design for a product in my first job out of college 10 years ago. They are still shipping it. I'm sure they've updated the boards as parts go EOL (end of life), but they definitely didn't restart from scratch. I would like to get one, but it costs $30k+ [1]

[1] http://www.keysight.com/en/pdx-x202266-pn-N9020A/mxa-signal-...


I always keep screenshots + video walk throughs of apps I build, for this very reason. You can't expect any software to live forever.


> Having other people actually use the stuff is the best part for me.

That's right. It's why I rarely invest the time writing a program just for my own amusement. To me that's like building a model train layout in the basement, that nobody but me will see, and when I sell the house the next owner will rip it all out.


One should care, because knowing that one's creation might reach millions of people, will prompt careful consideration to usability aspects.

I guess that's why we have so many poorly designed products, because their creators don't truly care.


because then the entire idea of the american economy being based on a meritocratic system where good companies do well and bad companies that make no products fail, and the same goes for workers, is somehow an elaborate lie and a shell game?


I'm a web developer and very often end up building apps or websites that never even make it live. I've learned not to mind. The fun is in the build :)


One of the reasons I started my own company was then nobody but me was going to cancel a project I poured my efforts into.


Life is long. If someone work in this field for 30-40 years, and launch a satellite every 5 years, then they will make 6-8 satellites in their lifetime. (The article says that they spend 2 years building the satellite, but I guess there is some additional time, so I prefer to use 5 years in my estimation.)

There is a 2%-5% catastrophic failure rate in unmanned rocket, so they will have a ~50% chance that one of their satellites just explode. As the article says, "it sucks", but no one launch only one satellite.

To somewhat confirm my estimation, yesterday someone posted the photos of two of the members of the teams of the JPL that build the Pathfinder, Spirit+Opportunity, and Curiosity. This is a 20 years timespan, I'm not sure that everyone worked in all of them, but my estimation is not too wrong.

http://i.imgur.com/3Hg9O.jpg

https://marsmobile.jpl.nasa.gov/images/Evans_Mars_Yard.jpg


What researcher launches a new project every 5 years? NASA does, but each mission is unique to some new academic or research group. The team from U of C isn't launching in 4 years or probably ever. Its heartbreaking for them. They fought for this mission, somehow pulled it off, and lost it. NASA has other missions to launch. There's no going back.

This is also why critical missions like spy satellites are often built in pairs. One to launch, and if blown up, the mission gets a second chance with another rocket at a later date. Science missions don't often have this luxury. You get one shot and if it fails, too bad.

edit: note I wrote 'often' not always. You don't know if your mission will get a second chance and that's the stressful part. If this was a mission to study Earth's climate change would it have gotten a second chance under the current administration? Probably not.


At the end of the article:

> Lindbergh and her team rebuilt their experiment—again—and put it on another rocket last April, a Falcon 9 launching from Cape Canaveral. She watched as the engines ignited, the smoke billowed out, and the rocket rose.


I agree with what you're saying. I can give some examples. When the CO2 monitoring satellite OCO failed to reach orbit in 2009, a replacement was built, partly from flight spares, called OCO-2. It was launched about 4 years later and continues to operate. Most of the team remained intact throughout the interim period.

Due to the same launch vehicle problem, in a separate launch attempt, the Glory satellite failed to reach orbit. But I don't believe it was replaced in the same way.

Some missions are highly successful, and they tend to have successor missions. I suppose most mission teams aspire to this status. The JPL Mars program is an example. A succession of more-capable solar monitoring satellites has been launched by a team including Stanford and Lockheed, which has remained stable over 2 decades now.


I briefly, and well down on the food chain, worked on the Spitzer Space Telescope. IIRC, it was originally supposed to launch in the laste 80s/early 90s but finally launched in 2003. Probably a good thing because the IR detector quality was significantly better.


There are also Olympic athletes who sacrifice 20 years to compete. They can easily be sidelined due to injury, someone else being slightly better, their nation deciding to boycott the Olympics that year, you just were sick that day, etc.

It's not like there's any backup plan, either. There's no market for Olympic skills.


Failed Olympians are a real tragedy. Most of them have been in their sport since they were as young as 5 or 6. To lose out on a medal after doing something for your ENTIRE cognizant life must be devastating and identity-crushing.


In most sports you easily can go to 2-3 Olympiads in your career though.


This happened in "Contact".

But they had constructed that spare time-machine in secret using the fact that most of the cost was sunk in R&D.

There's a Hubble Telescope hanging on the wall in the Smithsonian in DC, fwiw..


They could budget to build two of everything as cold spares for less than 2x the cost.

But outside the military and certain high-risk areas such as offshore oil that appears to have declined as a contingency sometime in the 1970s.


Even though the Ariane 5 has a decent record I am so nervous about this launch! Makes you wish they'd built a backup just in case...


The nerves that I will have (and I have nothing to do with building it) when that thing takes off will be nail biting.


I watched a video about the unfolding sequence and if I remember correctly the whole process takes half a year. That's a lot of nail biting!


Plumber comes to my house and fixes an issue in 20 minutes and charges me $100.

"Why so much," I ask. "It only took you 20 minutes."

"Well," replies the plumber, "It actually took me two decades to fix this, you just saw the last 20 minutes."

In other words, there's a lot of talent at work here even if it was the "last two years." Worse, how often do these people get a space mission. Its competitive. Just because your mission blew up doesn't mean you get favored status next time you work on a proposal or team trying to get into space. You spent the past few years working, not writing grants, playing politics, publishing papers, etc and all the stuff it takes to get a space mission. You're not just years behind, but perhaps a decade (or more) and most likely you'll never work on another space mission again.

Its a lot of pressure and heartbreaking. I don't think the tone here is overly dramatic. It think its pretty accurate. Luckily, this team got a chance to launch again. That's not always in the cards. Imagine if this was a planetary climate mission during this administration. It wouldn't launch again.


If you were in the program management side of a big interplanetary probe and got on the project in the planing stages, you really could lose decades worth of work, though probably not if it failed on the launchpad. The worst case would be getting it all the way to its destination and having something go wrong there.

But if you spent a couple years as a grad student putting an experiment together... well, I hope that's not your life's work.


I had a year's worth of my life crater next to the launch pad. It's a terrible feeling.

It was ever thus. Before first light of the 100-inch telescope on Mt. Wilson in 1917 (at the time the world's largest), George Ellery Hale was so nervous that he only invited a poet, Alfred Noyes--no members of the general press.

Noyes later recounted the drama as part of a poem, "Watchers of the Skies":

  "To-morrow night! For more than twenty years,
  They had thought and planned and worked. Ten years had gone,
  One-fourth, or more, of man's brief working life,
  Before they made those solid tons of glass,
  Their hundred-inch reflector, the clear pool,
  The polished flawless pool that it must be
  To hold the perfect image of a star.
  And, even now, some secret flaw—none knew
  Until to-morrow's test—might waste it all.
  Where was the gambler that would stake so much,—
  Time, patience, treasure, on a single throw?
  The cost of it,—they'd not find that again,
  Either in gold or life-stuff! All their youth
  Was fuel to the flame of this one work.
  Once in a lifetime to the man of science,
  Despite what fools believe his ice-cooled blood,
  There comes this drama.
                          If he fails, he fails
  Utterly. He at least will have no time
  For fresh beginnings. Other men, no doubt,
  Years hence, will use the footholes that he cut
  In those precipitous cliffs, and reach the height,
  But he will never see it."
[http://www.gutenberg.org/ebooks/6574]


It's bad to have your hard work blow up in a rocket. But to then re-make the entire thing, put it on another rocket, and have that one blow up too?

Hats off to Rachel Lindbergh for being part of a very exclusive club in that regard.


When in university, I worked on a cubesat project; the first satellite apparently got stuck in its launch container, never getting out to deploy its antennas and say 'Hi, mom!'. Good thing we had built two; now all we had to do was wrestle ourselves in on a later launch.

The second, duplicate one? Oh, that wound up in a Kazakh field while still attached to the launch vehicle. I like to think that second one made some poor farmer need a new pair of pants, if nothing else.

Then again, as far as impact goes, it would have been a lot cooler to receive one's telemetry from space rather than just having a pair of soiled pants to show for it...


Reminds me of the people who survived both atomic bomb attacks.

https://en.wikipedia.org/wiki/Hibakusha#Double_survivors


Here's my thoughts as someone who has worked on rockets, and even seen a few of them blow up (but not my fault). It is amazingly depressing. It happens so fast, and there's a feeling of sheer helplessness. You can't do anything, and it's just gone. I imagine it's exactly how you would feel it you were watching your house burn down, or get ripped up by a natural disaster.

That being said, being reproducible is key. Both in terms of production of vehicles, and scientific experiments and equipment. Some of these things are very expensive, but it's always a good idea to have at least two copies going around, and probably more. You'll want to do testing and comparison experiments if the space flown one comes back.

Another thing not mentioned in this article is the concept of launch insurance. It's the homeowners insurance for getting stuff to space. It is also very tricky in terms of wording, and what is covered when (for example, when AMOS blew up on the pad that's different for the insurance than blowing up en route).

Some of the biggest troubles come when you have a limited launch window, and might not see another one for years, or tens of years. You just can't get that time back.


The consolation is that it is always much quicker to build the second of something than it is to build the first one, because you know exactly how to do it.

I like to say that I could have done my PhD thesis in 3 months rather than 5 years if I had to do it all over again... Of course it's actually figuring out what to do that is the real work, not the doing. ;-)


...then there's this scientist who apparently worked on scientific instruments on not one, not two, but three failed satellites.

http://www.nature.com/news/software-error-doomed-japanese-hi...

> Software error doomed Japanese Hitomi spacecraft

> Dan McCammon, an astronomer at the University of Wisconsin–Madison, helped to design and build Hitomi’s premiere scientific instrument, an X-ray calorimeter that measures the energy of X-ray photons with exquisite precision. He has been working on the technology for more than three decades, flying versions of it on the ASTRO-E mission, which failed on launch in 2000, and the Suzaku spacecraft, in which a helium leak rendered the instrument useless weeks after its 2005 launch.

> McCammon says that it would take about US$50 million from NASA, and another 3–5 years, to build a replacement calorimeter. A version of it is slated to fly on the European Space Agency’s Athena mission, but that is not due to launch until 2028.


Dr. McCammon has endured even more failures than this article shows. He has a smaller calorimeter that he launches on NASA's sounding rockets. I was a telemetry engineer on another mission while both of our payloads were at White Sands in launch preparation. Dr. McCammon told me that he had been trying to have a successful launch since 1998, but launch vehicle issues (guidance failure cost one launch, command uplink cost another) and experiment issues ( the calorimeter is relatively fragile) had resulted in four consecutive failures.

For all that though, he is an accomplished scientist and still had a positive attitude, which made me admire him greatly.


I thought this would be about the Antares team (who worked on that rocket significantly more than 2 year).

I heard however about the story of the experiment that blew up both on Antares and CRS-7. Glad to see an article about it. Space is hard.


Is it just me or did the whole turn to the general idea that "space is hard" feel like an odd turn for the article. It felt like, specifics, specifics, generalities that have no meaningful tie-in.


"space is hard" is a fucking lie.

We knew why the Challenger blew up, we could have prevented it, we chose not to.

We knew why Columbia disintegrated, we could have prevented it, we chose not to.

None of this shit is hard. Engineers know how to prevent this stuff.

The hard part is choosing to care whether something is reliable or not, then doing what is necessary to make sure it is reliable.

Every engineer can tell you about some management fuckhole choosing to do the wrong thing. The hard part about our civilization is how many fuckholes there are pushing people to do the wrong things for stupid reasons.

"Space is Hard" is a lie we tell ourselves so that we can ignore the real problem. The real problem is our personalities, our laziness and our stupidity. These scare us because they are truly hard and we truly have no answers for them.


This is why I always tell people: If all we had were technical problems to solve, it would be easy. People problems are much more difficult to solve. As the scale of the project increases, with more people, more personalities, more money -- so does the difficulty of the project. "Space is hard" because the scale of the projects are huge. It is hubris to believe that we are the ones that don't contribute to the problem. It is naive to think that some manager isn't looking at us and thinking, "That asshole programmer is going to sink this project for sure". If we can see the problems, why can we not fix them? Because of assholes? Or because we lack the skills to explain the issue and motive people to fix it? Or because we are wrong? All of those things come into play and if you want to be successful you need to navigate that minefield.

If only it were rocket science, it would be sooooo easy. Alas, it is not.


You are wrong. We only ever got to space thanks to our personalities, our persistence, and ingenuity.

Edit: Just to be clear, space is extremely hard.


On the plus side: It's "built to print", we've already done all of the R&D work so it's just a matter of remanufacturing all of the hardware. The software's already tested and ready to go (an will get further testing when the satellite is rebuilt).

On the minus side: Long lead times, and payloads/instruments where the right things happened at the right time to make them possible. A failed launch on some projects could mean a 3-5 year delay - and then the whole point of the mission could be moot (technology moves fast).

And that's where you find yourself in the gray area: Ok, let's take our lessons learned and improve it!


It usually results in a lot more work: things like failure review boards, fault tree analyses, corrective action recommendations, and endless status meetings from management wanting to know when they can launch the next one.


I've known people who've lost huge amounts of work due to a failed hard drive. Stuff happens.


respect to those people


>is a member of a an exclusive group no one actually wants to be in: people who have seen their work destroyed in a failed rocket launch.

If I got to join this group, I kinda feel like it would be my lifes pinnacle.

Clickbait to the max here guys.


At first I thought this might have been about the poor North Koreans forced to work on their rockets.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: