Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
IBM to Acquire the Weather Company (nytimes.com)
306 points by ISL on Oct 28, 2015 | hide | past | favorite | 175 comments


Hopefully IBM can fix the weather data industry. It seems ridiculously broken right now that if I want to get a "prediction" of how much it's going to rain next week there are a million free sources, but if I want to see how much it actually rained in the past week then I need to pay thousands of dollars or else figure out how to parse a bunch of obscure and undocumented formats.


Hey buddy, I use this resource[1] all the time (and they're actually housed into the building next to me here at work). It's the long-term climate records from monitoring stations across the country, the United States Historical Climatology Network, and you can get daily data going back over a hundred years in some cases.

For example, I've used it to crank out random temperature histograms in the past[2], and even plotted some long-term averages to decide on temperature and precipitation ranges when going on camping trips. It's really useful if you're willing to work with the data a tiny bit. You're getting csv files, so it's not terrible to work with in any way via Python or R, or whichever data-friendly language/tool you want to use.

[1] http://cdiac.ornl.gov/epubs/ndp/ushcn/ushcn_map_interface.ht...

[2] http://btus.us/decadal-histograms-of-monthly-yosemite-temper...


So apparently in order to use this I need to enter a NESDIS ID. When I Google that, the first result I get informs me that a NESDIS ID is "an 8-character identifier assigned by NESDIS that becomes the RAWS identification number for use with the GOES satellite." Clicking on 'glossary' in the breadcrumbs then fails to take me to an actual glossary.

I realize that it's theoretically possible to get this working. But it's not easy. What I want is a way to enter a postal code and get a JSON response with the precipitation for the last week. Without having to convert to scientology, learn GeoDjango, license GIS software, read an 800 page manual on parsing a bytestream from a satellite, etc.


Um, maybe to use it in an commercial setting? I've literally no idea what you're talking about as I've never come across that before. I've always just pulled down the station stuff I need by hand (since you can get all the station's data at once), or by some lightly mechanized webscraping. Your comment sounded significantly more chill than what it seems like you might need.

It wouldn't be terribly difficult to mechanize this and get all the data at once, but I thought you wanted to occasionally see how much it rained, not build a service out of this.

Does this page[1] not contain what you need?

[1] http://cdiac.ornl.gov/ftp/ushcn_daily/


*.txt.gz is not exactly a modern data format.

I share the parent's sentiment; finding easy access to good weather data is hard.

Do you want your side project to be an experience in learning how to parse obscure formats and coordinating obsolete dependencies, or HTTP GET some JSON and immediately do something with the data?


You're shitting me, right?

They're literally giving you all of the data in a plaintext format with documentation, and you're still complaining?

Sure, having everything handed to you on a silver platter is nice, but why should they do your project for you?


Imagine what is going to happen when they discover the entire financial system is built on copying CSV files around by FTP.


Don't forget the emails back and forth and manual fixes and resubmissions from either end when there's a problem causing the CSV to not parse!


> *.txt.gz is not exactly a modern data format.

You know what you get with a format that's not "modern"? A huge, mature set of tooling that works. For instance, without trying because just responding to this whinging is barely worth the wear on my keyboard, I'd be disappointed if OS X wouldn't cheerily open the file, and unarchive it into the original directory structure simply by typing "open weather_data.txt.gz" from a terminal window. In other words, one could script it. I'd show you how, but holding your hand leaves me with only one hand for typing.

> HTTP GET some JSON and immediately do something with the data

Well, that's the magic of JSON: any code I write just wondrously knows what to do it without me telling it how to parse it. How can poor ol' ASCII text possibly compete?


You don't even need the terminal, just double click the file.


Can't script a double-click in Finder. :-) (Well, there's AppleScript...). But, yeah, point taken.


Not modern doesn't have to mean difficult to parse, it depends on your personal outlook. You can also download individually customized csv files if you want, which my original link led to. csv and txt files seem eminently reasonable to me when dealing with a huge historical cache of data that's being put out by a public institution for free.


I could write a node server that converts all that fixed length ASCII data into JSON faster than you guys can complain.


  wget -O - http://cdiac.ornl.gov/ftp/ushcn_daily/state01_AL.txt.gz | gunzip -c |  in2csv -s schema.txt | csvjson
Fixed that for you. And in less space than it took for you to complain.


What entitled nonsense. *.txt.gz is not a problem for anyone of even minimum competence.


Wow, you guys really missed the point.

Why can't we ask for nice things, even if they are "free"?

How are you going to get a kid interested in programming historical weather analysis when he's got to figure out how to configure an entire stack just to query your dataset. If you really want my opinion, this should be an AI problem, not yet another tedious MVC application.

Furthermore, when I jump into projects, I am most definitely appreciative that there is a responsive system on the other end, no matter how much I'm getting paid. I'm not some noob and yes I have imported gigabytes of minimally-structured data into my own applications, but I don't want to have to do that every time. Especially just to tinker.

If you're not thinking about how your software development experience can be made quicker, more direct, and less painful, I pity you. Shit on JSON all you want, but I'll take a populated native object and a blinking cursor over FTP'd text files any day.


"Shit on JSON all you want, but I'll take a populated native object and a blinking cursor over FTP'd text files any day."

I'm not sure this particular hill is worth dying for. Remember, text was the universal transport long before JSON was (and probably will be long after).


Try this http://www.forecastwatch.com/

I once bought data from them for couple of hundred dollars.


Wunderground has lots of data available. They may have actually been part of this deal as they were owned by The Weather Channel.

http://www.wunderground.com/history/airport/KSFO/2015/10/28/...


International Business Machines on Wednesday announced an agreement to acquire The Weather Co.'s B2B, mobile, and cloud-based web properties, including WSI, weather.com, Weather Underground and The Weather Company brand.

Source: http://www.cnbc.com/2015/10/28/ibm-plans-to-acquire-the-weat...


I use weather underground for my personal server (developer accounts are free up to 500 calls a day...). We used them for work to get weather data for lots of locations hourly ("drizzle" plan). Very reliable.

api: http://www.wunderground.com/weather/api/d/docs

National weather service, http://weather.gov is my goto weather forecaster, but at the time I wrote the code, getting hourly data based on lat/long for several thousand locations (some outside the US) was problematic. and XML..



Aww I love this app except their ads. I'm not sure what IBM's plans for this is.


That's what I love about wunderground. I actually have an Ambient Weather ws-1001-wifi weather station at my house, and I'm part of wunderground's weather station network. Whenever I pop open the weather app on my iPhone, it shows the weather in my backyard which I love.


That doesn't look free, unless you want to scrape.

http://www.wunderground.com/weather/api/d/pricing.html


Part of IBM now as well.


Have you tried using http://forecast.io/? You can get historical forecasts via the JSON API[0], which is, from what I can tell, free for low-volume use.

[0]: https://developer.forecast.io/docs/v2#time_call


I think he's trying to get historical data of actual weather, not forecasts.


> Please note that we only store the best data we have for a given location and time: in the past, this will usually be observations from weather stations (though we may fall back to forecasted data if we don't have any observations)


That looks really nice.


For the most part, the Weather Company is merely redistributing data collected by the federal government, specifically the National Oceanic and Atmospheric Administration. You can easily access historical rainfall data here: http://www.nws.noaa.gov/oh/hads/


The National Weather Service makes historical weather data available for free in a standardized format.[0] I'm sure there's already a parser written for any language you might be using.

[0]http://w2.weather.gov/climate/index.php?wfo=mtr


That's true, but that data is for only a few locations.


Weather Underground makes nearly all climatological data available in easy-to-consume formats (csv mostly). Using your rainfall example, here's how I'd get a listing of rainfall by day for the month of September, from my local National Weather Service monitoring site:

  $ curl -s "http://www.wunderground.com/history/airport/KMSP/2015/09/31/MonthlyHistory.html?format=1" | grep -v Precip | cut -d "," -f 20


You can actually get all that for free from Norwegian Meteorological Institute, both forecasts and observations. The only caveat is that it's all in Norwegian. Looks like that is a little intentional[1].

[1] http://om.yr.no/verdata/free-weather-data/


Free quality controlled historical data available from NOAA as well:

https://www.ncdc.noaa.gov/data-access/land-based-station-dat...


I once found a link that showed useful charts for historical weather data including averages and highs and lows. I can't find it right now, but Googling for it brings up a similar site: https://weatherspark.com


Sorry for non value adding cynicism, but when has a big corp fixed any industry? Especially IBM? Sure, more competition is welcome, but I would not get my hopes up.


I scraped http://en.tutiempo.net/climate when I was writing the following article: https://blog.wearewizards.io/comparing-the-weather-of-places...

There is also a link to the python script I made to scrape it


I wouldn't be too hopeful; this situation is the entire reason IBM made this purchase.

IBM is in the business of providing data about a whole lot of things. I would expect the cost for access to this data to go up, not down -- the majority people who care about how much it rained in the last week only care because they can make more money by knowing that.


I couldn't disagree more. I bet we actually see a service exposed via Bluemix in the not distant future.


Those formats are very well documented and standardised world wide. Commercial use is another thing though...


Check out products like ForecastWatch from Intellovations. Not sure, but might be of help. Not affiliated, just had conversations with the founder a while ago.

http://www.intellovations.com/products/


There is Noaa http://w2.weather.gov/climate/ you can get past results there.


Wolfram|Alpha has the data you are looking for.


...and operates like North Korea when it comes to openness.


  The company also has a popular mobile app, which Mr. Kenny said 
  collected barometric pressure from its users’ phone sensors. 
  Millions of locations are turning in information 96 times a day.
Well, this is news to me.


Yes, I think that news is about as big as the news of the acquisition itself. Very exciting news! I have been trying to get the leadership of WU and TWC to do this for the last 3-4 years, and I'm very happy they are finally doing it. In the beginning[1], they made comments suggesting that they thought it was a dumb idea or a waste of time, and that the barometric pressure data from smartphones could never be used in a weather forecast model. I was flabbergasted and could not understand why they wouldn't even consider trying it.

Well, four years later here we are! Finally. We are at the beginning of a revolution in how we understand our atmosphere. There are now billions of internet-connected barometers that can be used in weather models - this should provide a revolution in our ability to forecast short-term mesoscale events like thunderstorms and tornadoes, among other things [2].

[1]: http://gizmodo.com/5851288/why-the-barometer-is-androids-new...

[2]: http://cliffmass.blogspot.com/2015/10/apple-computer-could-r...

Some startups working on this are Sunshine, PressureNet, Weathermob, WeatherSignal:

https://thesunshine.co

https://pressurenet.io

http://weathermob.me

http://weathersignal.com


Maybe this is a dumb question, but how does the weather app know if the user is outside? Many buildings have HVAC systems that are designed to have positive pressure relative to the general environment, and most phone GPS systems are not accurate enough to tell the difference between being inside a building and outside of it on the sidewalk.


That's definitely not a dumb question, it's actually quite nuanced. First I will stress that it's not necessary to know if the phone is inside or outside, because the trends in pressure will be the same. Over a period of 6 hours, the pressure will drop inside if it is dropping outside, and knowing the rate of that drop and the timing of it is key for the weather forecasts.

In the work I've done on this problem we've never been too concerned about inside vs. outside. A much bigger problem is small altitude changes that mess with the exact readings by huge margins, but again, if we're focusing on trends instead of the immediate exact value, we can smooth out the noise relatively easily.

However, to actually answer your question, an app can make a very good guess about whether it is inside or outside. Key sensors and metric for this are: ambient light sensor, humidity sensor, GPS location / barometer. During the day it is normally brighter outside than inside, and during the night is normally brighter inside than outside. So that's clue #1. Clue #2 is to check GPS to see if the user is moving - if you are moving rapidly then you are probably not inside. And if the barometer is rapidly going up and down, but GPS is not changing, then the user is probably inside (elevator, stairs, etc) and that's clue #3.

Again, none of that is necessary to solve for this particular problem, but it's interesting.


I didn't even know that modern smartphones included humidity sensors and/or barometric sensors.


That's essentially a statistics / data analytics question.

Statisticians (et. al) are well versed at receiving a whole bunch of messy data and figuring out the value that they want to observe.

From a rough perspective, I think it would look something like this:

1) There will be a large concentration of people reporting pressures that are on the low side, this is the true outside pressure.

2) There will be clusters of people reporting high pressures, and that will be inside buildings.

3) To ignore funny circumstances where large amounts of people are reporting lower pressures than the true outside pressure (I'm not sure why this might be, but this is not my domain of expertise) you can compare it to the pressure that people are seeing around that area, and to official pressure readings.


Little bit of machine learning and here you go.

The biggest deal is to collect the data. After data arrives, they can then look for any patterns they need. Filter anything that seems wrong.


When will phones begin to also incorporate thermometers? And don't most already include hygrometers?


I'm also interested in this question. Can someone with this expertise answer?

Is there a reason why we haven't seen a thermometer in a phone so far?


The Samsung Galaxy S4 has both a thermometer and a hygrometer (as well as a barometer). Samsung removed both of those sensors for the S5 in order to waterproof the phone - they were able to keep the barometer while still waterproofing, which is neat. Someone else might have to answer how that part of it works (something about watertight and airtight having different physical requirements..?)

In fact, every phone does have a thermometer, but that is not usually accessible by developers and it does not measure the actual air temperature - it's used by the phone to make sure it doesn't get too hot from the battery or CPU. We might not see many app-level thermometers get added to phones because the data is extremely difficult to use. For example, most of the time it will measure the temperature in your pocket or in the room where the phone is - and even then, the reading will be radically affected by what the phone is doing (browsing the web is going to cause a different temperature reading compared to watching a video, compared to stand-by mode).

I do expect that we'll see more hygrometers and thermometers and other sensors make their way into phones as time goes on, but I think the primary reason for the slow take-off is because the data is really really hard to use. That's the thing with the barometer - it's very resilient to noise and even when the data has problems, they are fixable.


Appreciate the detailed response.


Installed!


Uninstalled.


Why? Do you find your barometric data to be particularly sensitive?


(I'm not the person you are asking, but): People often don't really know how sensitive the barometric pressure data could be, if used for tracking purposes. The barometer in your smartphone can help provide your physical location to within 1 meter, if used for location tracking instead of weather forecasting. If used for weather forecasting, the altitude data is treated as noise and will not be used to find your location.

However, it is now mandated by the FCC that all barometer-carrying smartphones will report your accurate altitude information to 911 emergency response if you make a 911 call from your smartphone. So the location tracking element of the barometer is a very real concern for a lot of people, as it can tell a tracking company what floor of a building you are on at any given moment.

However I think it's quite clear that weather apps, and The Weather Company, are much more interested in the local atmospheric trends than they are about your current floor.

Edit: FCC citation: https://www.fcc.gov/document/fcc-adopts-new-wireless-indoor-...


That's a good response to an admittedly pithy comment. I suppose the point I was trying to make was that most people have never considered their devices barometric capabilities. And that the OC was throwing a kneejerk "data-collection is bad" response, and deciding to uninstall because data-collection of any kind is bad.


That's why I'm still inclined to believe "privacy vs. progress of mankind, pick one" soundbite. It's a perfect example of data that's really useful if collected this way at scale. Destroying large-scale data collection because advertisers/Evil Future Government is throwing the baby out with the bathwater.


> The barometer in your smartphone can help provide your physical location to within 1 meter, if used for location tracking instead of weather forecasting.

How can this possibly be true? Wouldn't the pressure around a couple of square meters simply equalize instantly? You're not saying "within a mile" you're saying "to within 1 meter", and I don't get it.


Well first, I'm in the weather industry, not the location-tracking industry, but I will share some of the insight for how this could be done. The barometer won't help with your latitude or longitude usually, because you are correct: the pressure will be approximately equal in those dimensions when you are looking at such a small area.

The barometer is sensitive to altitude changes much much smaller than a meter, probably something like 0.1 meters. So if you are a person, with a smartphone, and you are in a tall building, the barometer can assist with determining which floor you're on. Other sensors would need to be used for calibration (since the changes weather will affect the floor-level) and it doesn't help with the x,y coordinates, just the z. But other location systems are now very very good at x,y and they mostly lack the z.

So by adding the high-accuracy relative altitude capabilities of the barometer to the existing x,y systems, you can get extremely precise locations.


They should already have GPS info, but I am always weary of unintentional data leakage through presumed sanity of data.


If anything the deal with IBM should make you more worried. It's more likely that intelligence agencies have hooks in IBM than the Weather Network, and barometric data can be another heuristic to define who/where you are.


The barometric data is only useful if you know where the reading was taken, right? So you're sending (at least!) pressure + location to their servers. Probably some identifying information as well.


Maybe he just doesn't like his data being stolen regardless of its content


[flagged]


lol


Does anyone know if it asks for these permissions on install?


Specific permission to access the barometer is not required by Apple or Google. The weather app already does ask permission for location services in general, so yes it is already there. Since they have already been tracking your latitude and longitude, the barometer gives altitude as well - along with current atmospheric pressure trends.


  permission for location services in general
This is interesting primarily because for most part I switch-off my location services as I think it's a drain on my battery. The only time it's enabled is when I need the GPS which is quite seldom. So I would assume that it would affect the accuracy of the model if say I live quite a way off from where I work.

Also if I'm spending most of my time indoors in buildings, how much of it would affect the readings as compared to when I'm out in the open?


> Also if I'm spending most of my time indoors in buildings, how much of it would affect the readings as compared to when I'm out in the open?

Indoor vs. outdoor isn't really a problem most of the time - the pressure is basically the same inside and outside. A bigger problem is fast altitude changes, which make the absolute reading from the barometer very difficult to interpret safely.

One of the reasons that it took so long for TWC to finally do this is because of the concerns you just voiced: that having this data might actually make the models worse, if they don't take enough care to quality-control it first.

A lot of the research done by scientists (like Cliff Mass at the University of Washington) is how to assimilate this data in a reliable and trusted way. It is likely that The Weather Company does not know how to use the exact pressure readings from the barometer yet (nobody does). Instead, they will probably use the trends over time (say 6 hours), because those are more resilient to noise.


Wunderground and weather.com were owned by the same company? For some reason I never would have guessed that just based on my experience on those sites.

Wunderground generally feels clean and on-topic.

Weather.com is often filled with "Doomsday scenario with newest hurricane season" and "Check out this massive shark jumping" news stories.


Once I visited weather.com and every image, literately every image, was a reference to an imminent cataclysmic event. Volcanoes exploding in fire, meteorites exploding over cities, etc. It's hard to differentiate weather.com from a parody of weather.com.


That, my friend, is known as effective market segmentation.


Yup, it is all about giving consumer a "choice".


The Weather Channel acquired Wunderground.com in 2012: http://www.wunderground.com/about/pr/news.asp?date=20120702

They probably didn't want the competition anymore because I don't think the Weather Channel has learned a damn thing about how to run a clean website since acquiring Wunderground. On the plus side, it appears they have left Wunderground to do their own thing and not interfere.


Prior to being acquired they were a much lighter weight and usable site. Now they are almost as bad as weather.com with ads and unnecessary UI clutter


I agree. These days the wunderground site drags my 2009 iMac to an almost total standstill. I don't really understand how they screwed it up so bad :-(


They've been owned by the same company for less than a year, I believe.


> For less than a year now

More than that now, I ran into a guy in the Haight wearing a WUnderground fleece about 3 years ago, right after the buyout and told him it was my favorite weather website and that I hoped Weather.com wouldn't ruin it.

He then told me he was on the Weather.com team who led the buyout -- Oops. He still offered to send me a WUnderground hoodie though, so kudos to him for having a sense of humor.

First article I found re: the buyout in 2012;

http://www.nytimes.com/2012/07/04/us/as-weather-channel-buys...


Wow time flies.


I have no problems using Weather.com. I just type in the url, they've got my locations of interest up at the top, I click on one, get a quick forecast and can drill down into different segments like "The Weekend" "10-Day Forecast" and easily get the "Map-In-Motion" doppler.

Any extraneous stuff I notice but filter out automatically. I love the site and don't have any of the sort of issues you've described. I do seem to have a higher tolerance for the minor inconveniences of life when I compare my experience to those of the people who surround me, so maybe I'm just lucky. I think you can develop this attitude though, and it sounds like it would help ease your way through the world.


Unless I'm misreading your last sentence, I think you vastly overestimate how much it annoys me.

The only issue I described was extraneous noise on the website (most of which has nothing to do with weather). It's just less usable/aesthetically pleasing than the alternatives.

I'm not losing sleep over it or anything. I just use the alternative site instead.


Whatever you do DON'T buy the ad free Weather Underground app! If you forget your password they will ignore any requests to reset it or, for that matter, any support requests. Sure it's just $.99 but as far as I'm concerned they rip people off.


Inspite of being completely superfluous I kind of like the entertainment on the Weather Channel and their website. They have done a good job making an otherwise boring topic entertaining.


I swear I have no idea what IBM does. Anyone know?


Good question :)

IBM used to make pretty good laptops but sold the laptop part off (Lenovo)

IBM used to have a pretty decent server line or lines, though they sold a good part of the Intel stuff off (now Lenovo for the server line, but had sold off many other parts earlier, such as storage, and started using more vendors)

There were things Software Group did like Websphere, which I imagine still does good money, but why you'd actively want to seek it out over free alternatives I have no idea. They also have DB2 in database land. Software Group used to be cooler and have better resources than hardware groups, or so we thought. I have no idea how well they are doing.

They also got rid of microelectronics and who knows what else. In general, innovative parts of IBM get sold off, and they keep the services parts? Feels that way.

Their margins on services and people are not good, so they also have the reputation of restructuring/laying-off like crazy and seemingly at random, with lots of pressure on stock price.

I was there in 2001-ish when they scrapped pensions, though the thought of pensions actually existing is a novel concept today. It used to be a place where people thought they could work forever, and that is decidely no longer the case anywhere, it shows how the views of what IBM was changed rapidly over time.

They also increasingly pushed for a contractor-heavy work force for more ability to make cuts and reduce benefits.


They still do lots of storage products like SAN Volume Controller, Storwize, FlashSystems and XIV.

However they are currently focusing on cloud, for that they have Softlayer and BlueMix.

In terms of other products MQ is still pretty big, so its CICS and as you said Websphere (which I believe has a free offering?). They still have the Power business and the Z server offering. They have SPSS, Tivoli, Lotus and Rational. Then they also have GBS (Business services) and GTS (Technical services).

Obviously they also have Watson. Some of the Watson services are available on BlueMix and they're pretty good. Currently I think Watson is branching out into all sorts of new areas. Health care is a big area of interest.

They also have ETS (Emerging Technology) which does client stuff, research and the like.

There is also a fairly big IOT department (But I don't know what happens there).


To add: they recently announced an investment of $3bn over 4 years in the IoT division, which will probably be focused mostly on analytics & will somehow tie in with Watson & Bluemix, which are the strongest pieces in their portfolio at the moment. [1]

They also broke out an additional "Education" unit within the IoT business unit, ostensibly to compete with PTC Thingworx, and Intel+Arduino, Raspberry Pi etc. in the student/teacher/maker market. [2]

[1] http://www.businessinsider.com/how-ibms-3-billion-investment...

[2] https://www-03.ibm.com/press/us/en/pressrelease/47660.wss


To add further, IBM's biggest revenue source is not technology and software, but its IT/business services & consulting wing which is no different than most IT outsourcing companies in the world. Of course their products do influence some of the work done by these.


We tested out Storwize years ago before IBM bought them.. What a piece of garbage, in my experience. "Oh, it won't slow down access to the NetApp at all!"... riiiight.

Had them sitting in front of the filers for two days before we yanked them out. Then it took them another TWO YEARS to send us boxes to return the systems despite repeated requests.


They used to make punch card equipment and typewriters. They've been around a long time: https://en.wikipedia.org/wiki/History_of_IBM


> I was there in 2001-ish when they scrapped pensions, though the thought of pensions actually existing is a novel concept today.

Novel. Hmm, I was working in software industry for 15+ years never seen place haven pensions. Do you mean 401k plans instead?


No, IBM had defined-benefit pensions, with 401ks as supplemental. In 2000 they announced that they would do away with pensions entirely for anyone not within N years of retirement (I don't recall at this point, possibly 5 or ten years). For those "near" retirement IBM would cease paying into the pension plan but they would still received a traditional pension from the funds up until that point.

There were a number of lawsuits over the conversion: https://www.google.com/search?q=ibm+pension+lawsuit&ie=utf-8... with a mix of results for employees and the company.


Interesting. Thanks for explaining.

Do other companies in tech have pensions? I just never heard of that before. I guess maybe in the context of the US Post Office (how Republicans forced it to fund retirement for so many year ahead, in hopes to bankrupt it and show how government services are ineffective and broken)


Pensions in the U.S. have always been a bit of a balance sheet scam. With few exceptions companies were allowed to comingle pension assets with the companies' day to day funds with a chunk kind-of fenced off. But the pension fund was fair game if the company went bankrupt, and in the 80s a very popular corporate raider tactic was to buy a company, leverage it as much as possible, and if it went bankrupt divert as much of the pension fund as possible to paying the very managers who bankrupted the company.

Government pensions are worse in that few government entities ever set aside a truly funded pension fund, using the argument that as a continuing entity theyd always be able to top off the fund with additional tax revenues if the investment projections did not pan out. In practice no elected official will voluntarily vote to raise taxes to meet the obligations to top off oension funds, hence the slow unfolding train wreck with public pensions in Illinois.

A general failure of any defined benefit pension seems to be the dependency on the sponsoring organization as a continuing, growing, profitable entity. No accomodation for changing economics, diminished industries, or reduced workforce.


Very few companies have those "traditional" pensions any more. Most of the companies that did have them discontinued them long ago. Just google for "pension liabilities" to see why.

About the only organizations that still have them feed at the public trough. And they're having problems with funding. Here's a recent story about a state transit agency. Its pension plan is over $800 million in the hole. And that's just one agency in one state. https://www.bostonglobe.com/business/2015/10/27/pension-liab...


> when they scrapped pensions

IBM doesn't offer any kind of pension as a benefit? What does everyone do? Buy privately? I think that's even illegal in some countries like the UK.


That may be US specific. i.e. Retirement pensions basically dont' exist in the US anymore, at least not in the software industry.

Basically most companies in the US offer some form of 401(k) with a selected amount of supported funds to choose to invest in, often with some degree of employer matching. Sometimes there are vesting requirements on employer matching. Startups are sometimes likely to not offer any matching.


Right - I thought a 401(k) was simply a pension though? Did there used to be something else that some companies still offer?


Pensions are known as "Defined Benefit" programs and 401(k)s are "Defined Contribution" programs.

The difference is pretty much what it sounds like. If you had a pension, you knew exactly how much money per year you'd make in retirement (the benefits were agreed to ahead of time). With a 401(k), you know how much exactly you're contributing and how much is in 'your' account, but the value of your retirement accounts in the future is dependent solely on how much you contribute and your personal investing prowess.

As companies and public entities underfunded pensions over the past 40 years, pensions started going bankrupt or dramatically cutting contractually obligated benefits that were to be paid to retirees, so companies began switching to employee-owned plans.

Shifting the burden of investing in retirement to the employees rather than the companies has its benefits but I think we're going to see a lot more elderly poverty as the boomers retire since on average, people are terrible at investing.


Getting a bit off topic, but when we find that people's 401(k) plans are not enough to retire on, we're not going to see more elderly poverty. The elderly vote. We're going to see young taxpayers making up the difference. The move from mentions to self-directed retirement plans is essentially a wealth transfer from future workers to current corporations.


I think that seems optimistic. The elderly do vote, but they consistently vote for people who cut their benefits... Every single budget negotiation results in cuts to medicare and social security, which by definition are cutting benefits to the elderly.

I think you're right on a macro level, the tax burden is being shifted to the younger and less politically active, but based on actual legislative action, it seems defense and social issues are more important to the elderly than retirement benefits.


I believe there were also regulatory changes that made the "dramatically underfunding pensions thing more difficult. (There may have been tax changes as well which drive many things.) Pensions survived in older tech companies for a long time; the company I worked for starting in 1986 had one until its acquisition in the late 1990s. But the pension was fairly modest and it also had the various other 401(k) and stock option plans.


They aren't the same. "Pension" means defined-benefit pension. You work here for 25 years and you'll get x thousand a month in retirement. It's how retirement plans worked up until 1983 or so. 401(k) means market-based investing. You put y percentage of your paycheck aside, and the company may contribute some money, and it goes into Wall Street tax-deferred to hopefully gain value.


A lot of older tech companies used to, but pensions are gone now.

For IBM, any accrued pension remains in a managed fund until you quit/retire. Most employees had a 401k option at the same time as a pension - you pay 6% of your salary into it and the company matches half that, another 3%.

If you didn't make the pension cutoff, IBM now matches the 6% plus some more depending on what date you started. I think it comes out to something like 14% of base pay for most employees. In a 401k, the individual has much more control over investment direction, whereas with a pension you had none.


I cant speak for anywhere else but the UK they offer pensions.


Yeah.

They made worksplace pensions opt-out recently as well due to the massive scare we have over so many people of working age not having saved anything for retirement.

I was automatically opted in to my workplace pension, into which I just pay 5% per month which my employer matches.

I'll likely increase that amount next month though.


They are an engineering services firm providing everything from consultation to turn key solutions to PAAS and IAAS products. Most of their work is enterprise and governmental and covered by NDAs and contracts that aren't/can't be widely publicized.


IBM is a financial services company that dabbles in tech.


you're funny. Probably half of the tech you use every day has an IBM enabling technology behind it, thankfully paid for, directly or indirectly, by this "financial services" company.

let's not forget that x86 itself could have been a Motorola 68000 if this company had not decided in Intel's favour. That's the extent to which this company defined the computer era.

The notebook you're using right now will boot DOS, commissioned by IBM. And I'm not even starting on the fact that almost all programming language roots go back to languages for the IBM 360 mainframes.

I'll gloss over the extent to which the architecture of your modern computer is essentially a scaled down version of what was invented by IBM in its mainframe business. Disk? Check. Processor cache? Check. Asynchronous bus? Check. GPU (IBM 8514) check.

It's not unfair to say that this company invented modern computing's engineering, and even if it has (very sadly) lost its way, one can hope that some of the culture of well-financed innovation remains, and this Watson business might be one of those.


I'm an IBM employee, was self-deprecating humor, actually. ;)


IBM has always been driven by finances


No disagreement here - but it is all in the past now.

With the possible exception of Watson/supercomputing, IBM is a shadow of its old self.


It seems to be a pattern that even great companies don't last much longer than the average human lifespan. Even if IBM is a 130 year-old company, its glory days lasted somewhere between 40 and 80 years.


Nothing can last forever. When you are big, you are doomed for aging. Empire, dynasty, country, all bow to age.

IMO, keeping your company rich, powerful but small (in terms of the number of products and employees) might last longer. But this seems unrealistic for real business in the real world. Reason is, the undead like AT&T (my colleague always talks about his experience working there and comments the Lab in NJ is a ghost town) will continue to live because of the deep root of AT&T in our American life.

It's a paradox and sarcastic. You either prosper and everyone admires, or you remain undead and disgusted by many. And to both you have to be big...

-- EDIT --

To the commenter: Most of the empires and dynasty power degraded after 50-100 year. That's the life span of one or two emperors. Almost all of these empires continue to split until full conquer. During splitting, you may have a couple more strong emperors in place. This is like HP splitting, or IBM selling its desktop and laptop division to Lenovo. They are merging or cutting arms to survive.

Roman might be one that last longer before a big split than the rest. In the ancient Chinese history, most dynasties you have heard of would lose dominance and prosperity after about 100 years, mostly due to incompetent leadership. In the case of the Mongolians, they conquered half of the world in less than a decade, but they divided into several ruling regimes to avoid big civil war. The one that controlled ancient China (called Yuan dynasty) went defeated to the Han people in less than a hundred years. The last dynasty in China, the Ching dynasty, did enjoy its dominance for four different emperors, and then rapidly declined due to foreigner invasion, corruption and severe political conflict, and unaccountable spending from the previous administration (plus many social issues).


Interesting focus only on China. Apols I erased my comment but the original point was the political empires seem to last many centuries as oppposed to corporate empires which seem to be at their peak for mere decades.

You make a massive error in your analysis, and that is to limit your perspective to Chinese dynastic political cycles. You ignore the idea of geographic empire building, where there is ample evidence of multi-century empires, and we are not talking only Roman. Also Byzantine, then Ottoman, British, French, and Spanish, not to mention Persian or Zulu. On China, while internal politics mean dynasties come and go, Chinese territorial integrity has been unchallenged for many centuries, testament to the fact that while the individuals, families, dynasties, may come and go, the underlying political structure is very resilient.

In ever single one of my cited cases, the nation-level structure is multi-century in longevity, and beats by an order of magnitude any commerce-driven corporate leadership structure we have ever seen.

So. My point is that corporations as a structure for human interaction/cooperation are deficient in that they are not able to deploy force. Only money. The latter is not long lived without the former.


You can say that again...


> let's not forget that x86 itself could have been a Motorola 68000 if this company had not decided in Intel's favour. That's the extent to which this company defined the computer era.

Except if IBM had it's way, we'd be using PowerPCs, not x86 and ARM.

IBM existed for 50 years before any of the products you mention. It did financial services before that.

Edit: I slipped and said Intel instead of IBM!


Found the IBM employee


I have never worked for IBM.


One thing they do is not sell to consumers anymore. For better for for worse, they got rid of that. The margins were too thing and it was harder to compete with Apple, Dell and other overseas makes of consumer harder.

They sell services, main frames, cloud, software used by enterprise (storage, project management, ...) provide consultants, and even have a bank! that companies can get financing from (when they buy all that expensive other stuff from it).


They also invented many fundamental computer hardware and software technologies. (I was a bit surprised when I first read that several years ago.) From the Wikipedia article about IBM:

IBM has 12 research laboratories worldwide, bundled into IBM Research. As of 2013 the company held the record for most patents generated by a business for 22 consecutive years.[16] Its employees have garnered five Nobel Prizes, six Turing Awards, ten National Medals of Technology, and five National Medals of Science.[17] Notable company inventions include the automated teller machine (ATM), the floppy disk, the hard disk drive, the magnetic stripe card, the relational database, the Universal Product Code (UPC), the financial swap, the Fortran programming language, SABRE airline reservation system, dynamic random-access memory (DRAM), copper wiring in semiconductors, the silicon-on-insulator (SOI) semiconductor manufacturing process, and Watson artificial intelligence.

https://en.wikipedia.org/wiki/IBM


Did you know one thing IBM are doing is working on a JIT for Ruby? They're talking about it at JavaOne today.


Consulting mostly I think


IBM tries to sell things. It is not always clear what those things are, what they are for or what they will cost, but the main function of IBM is selling.

Things.


Lotus Notes! Hey what's with the downvotes :(


Shitty marketing platforms.


Buy other companies.


Win at Jeopardy, star in some commercials, contribute little revenue to IBM's business.


This sets up IBM to compete directly with Monsanto's new Climate Corp in the ag-tech space. My hopes is that this drives the entry price of this tech down so it can actually benefit small farmers (who may have higher margins due to crop choice but lower profits due to size and economies of scale) and not just larger agribusiness operating on economies of size.

A question: how has the accuracy of weather prediction improved overall?


> A question: how has the accuracy of weather prediction improved overall?

Here is a recent New York Times article on that topic, with emphasis on hurricane prediction:

http://www.nytimes.com/2015/10/03/upshot/hurricane-joaquin-f...

"Over the last few decades, faster computers, superior models and new data have allowed all weather forecasting to improve, by a lot. But the United States hasn’t quite matched that effort. It didn’t invest in computing power and models that kept up with the potential for better forecasts."


Thanks!


This is a smart move and obvious application of the Watson engine if it can be applied reliably to the unpredictable 'global weirding' weather problems.

Does anyone know if organisations like Home Depot, for example, roll their own solutions or rely on 3rd parties to help move things like generators into stores ahead of storms?


I can't speak for Depot, but in the case of some of the larger retail organizations, major weather events will get coordination from HQ, but beyond those its typically left up to regional and store manager discretion.


That sounds like a problem that needs solving.


As far as I know it's 100% up to each store to watch the weather reports and determine the required action.


Interesting, it was my understanding that HomeDepot HQ move stock around ahead of predicted near term need.


Waffle House organizes for impending disasters from HQ, and has a mobile command center they'll dispatch to the affected area to help organize logistics. They're famous for staying open and/or recovering quickly from extreme events.

http://www.wsj.com/articles/SB100014240531119047166045765424...



Having had the fun that is working for a company that gets bought by IBM I wish their workers the best of luck.

Hope they enjoy the Blue Washing(TM)


It's said that Sun turned down IBM's buyout bid and went with Oracle because they thought that there'd ultimately be fewer layoffs with Oracle. Of course, no one really knows how things would've played out if IBM had taken over Sun.


I was an analyst at the time and an IBM acquisition of Sun never made sense to me. The Oracle acquisition did--even if I didn't personally like it for a variety of reasons.


Yes, I wonder how many of those 900 will be replaced by watson?


Hell, if Watson needs some weather data -- I offer the most precise weather forecasts already: http://mostpreciseweather.com/

Just send me a check.


Just an FYI, that page is broken for me. JS error of: "Failed to load resource: Could not connect to the server."


Doh - thanks. Looks like the geoip library I was using has moved. Fixed now for all your critical precision weather needs ;)


http://j.maxmind.com/app/geoip.js no longer exists (at least the domain j.maxmind.com)


Who cares about precise? We want accurate weather forecasts, which (as far as I've heard meteorologists say) is pretty much impossible for much of the world. Weather is just not very predictable yet.


Hi, it's a joke. :)


I'm assuming The Weather Channel was left out of the deal to die a slow death on it's own. I wonder if they will launch a new website and set of apps just focused purely on the TV content. You can't have a TV channel without a web presence, right?


I had a discussion with some weather forecasters about how we could enhance their datasets with IoT sensors (80k of them over ~200,000km^2. The answer was no enhancement whatsoever. Because they can infer an infinity of points from satellite data (all day every day), calibrated with about 1000 sensors.


FYI, if you're interested in USING Weather Company data & building apps with Bluemix, Devpost just launched a new online hackathon all about that (and Spark!) with $30K in prizes.

http://sparkathon.devpost.com/


There has been significant consolidation in the weather industry over the last 10-20 years, and it's a very interesting space right now with smartphones coming online as weather sensors. IBM's acquisition of The Weather Company is the joining of two huge behemoths, I wonder how well they can work together.

> The company also has a popular mobile app, which Mr. Kenny said collected barometric pressure from its users’ phone sensors. Millions of locations are turning in information 96 times a day.

Fascinating...The new-weather startups like Sunshine and PressureNet are going to be watching this closely.

https://thesunshine.co

https://pressurenet.io


Hopefully they'll be able to harness all of this delicious weather data and squirt forecasts to our watches as depicted in BTTF II.

https://www.youtube.com/watch?v=cI7ctWyXl5s


Weather is a chaotic system, due to it being chaotic and the lorenz attractor maximum three days of weather can be forecasted.

"While we can typically predict general weather patterns up to 3 days ahead, predictability for detailed local weather such as rainfall or fog formation is much less."

see more here http://research.metoffice.gov.uk/research/nwp/ensemble/conce...


If you think IBM may be unaware of the nature and consequences of chaos theory, you might want to look into Benoit Mandelbrot's employment history.


The thing is, is IBM as a corporate entity unaware of it? I know people who work for IBM who are much smarter than me, and who will always have a high level job as innovators in the IT industry, but they have 0 say in what IBM does, says or wants.


Unless your forecast is probabilistic.


I'm confused. WATSON is a specific NLP-based question answering system. [1] How can WATSON help predict the weather?

[1] https://en.wikipedia.org/wiki/Watson_(computer)


Not sure about anyone else, but this would make me actually want to work for weather.com or wunderground.com.


Is this the default weather app that comes on ios?


Nope - I think this is the one they're referring to: https://itunes.apple.com/us/app/weather-channel-weather.com/...


That's built by Apple but it does get it's data from The Weather Channel.


Hope to see improvements in the weather industry.


Hope they finally fix weather.com.


Funny tweet on this by @Cryo (https://twitter.com/Cryo/status/659369885991088129):

IBM Board meetings notes: "We should buy the Cloud" "What?" "You know, the Cloud" "Why?" "We want to be the biggest"

buys weather channel


Except that according to the article, the bought the Weather Company and not the Weather Channel...


The Weather Company owns the Weather Channel.


They bought the Weather Company minus the TV channel side of the business. So I would assume The Weather Channel will remain an independent company just without the digital assets.


The article explicitly says the weather channel was not part of the deal.


This reminds me a "joke" told to me by a friend who works at what used to be Softlayer (and has now been integrated into IBM) to explain why IBM bought Softlayer: "IBM wanted to have a good cloud business, so they bought one".




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: