Facebook has also recently started to require a Facebook account to use Oculus VR. It appears they are starting to crack down on the services they have acquired. I assume this more hardline approach of "give us your data or get out" will continue with any other current or future services created or purchased.

Counter-point; that was a one-way street. Data was potentially fed from Oculus to Facebook. Now, what you do on Facebook can impact your Oculus use. Get banned on Facebook and lose access to your purchased software on Oculus effectively making your device useless.

So now it is worse.

You can factory reset it.

My preference would be a flag that you can set per connection.

NULL = Undefined or No Data. Whereas a blank field can, in and of itself, be data. It may indicated something is intentionally left blank.

But for those times where you want to consider them the same, it would be nice to have a setting.

(Note that I admit the possibility that this may exist already, like most my great ideas.)

In MSSQL you have SET ANSI_NULLS OFF to do this.

At larger companies they absolutely do this. And the digital video is stored with an overlay of what is ringing up with it's price as the cashier scans it. So it is easy to see if they are not scanning items for a partner or manually entering the wrong barcode.

The issue here for some, if not many, is a matter of scale. It is one thing if an end-user, whom I am trying to service, comes to my site and gets my publicly available data. Maybe I monetize with ads, maybe not. It doesn't matter, that is the audience I am trying to service, regardless of size.

But when you scrape it my load goes up dramatically. A load I have to pay for.

It is analogous to the privacy debates going on with one said saying "hey, don't track everywhere I go and tag me with facial recognition" and the other side saying "hey, you are in public and people can see you." The issue is not complete privacy, but one of scale. And of intent.

I believe society is soon going to have to come to grips with the scale of things and legislate what are acceptable scales of action as it seems to be becoming a large issue in a growing number are areas.

So you throttle your users. We have http status codes for "too many requests" and all scraper software comes with a delay setting by default. Everybody who does scraping is supposed to know that its rude to blast a thousand requests per second.

This ruling has left open a big question of how much you need to spend to support scrapers and where the line between scraping and a DoS attack lies - and that's going to be a weird line. If my site is producing a big report off of data that changes quarterly then re-downloading that report every 20 minutes is possibly excessive and might wander into the realm of an attack - while if we looked at the same frequency with twitter it seems a lot more reasonable - maybe even a bit on the slow side.

Provide an API for public data to reduce the costs associated with rendering a full blown page, and deliver just the information needed.

Entirely feasible. Also reasonable for you to pay me for the service as it is taking my development efforts to meet your business model. The advantage to you is you have a defined interface that I won't prevent.

I guess you missed the comment I was replying to: it may cost you more money, in bandwidth and per page resources, to not provide an API than it does for you to provide one.

So no, I won’t pay you for the privilege of you saving money.

No, I'll happily just scrape your site instead. But if you'd rather not have that happen, provide an API.

Who pays for that API and the bandwidth? What’s in it for the data provider? On LinkedIn, viewing the data now shows ads or at least prompts the viewer to join the network. With scrapping and free API access, how exactly does LinkedIn benefit for their work of hosting the data?

My guess is hiQ (and others) would happily pay for an API over the data they're scraping right now.

Unless the costs exceed their current operational costs. Don't forget the time spent redeveloping on the new API, which includes validating everything is there, testing and cleaning up and removing the old (working) code.

Why buy the cow when you get the milk for free?

This isn't a great analogy here - getting the data delivered via API is simply more useful than having to re-assemble that data out of fragments parsed off of different web calls.

Could I suggest:

"Why buy the cheese when you get the milk for free?"

Look, if you build a product that relies on providing free information to the public, then you don't get to select a segment of that public and charge them for it. You can't hang a billboard on a highway, but then get upset when some people look at it the wrong way.

Now, if you want to have a walled garden and charge for entry to some and let others in free then that is fine.

The contention was not about load, it was about using the data.

While stated in what I interpret is a joking way, it is a very real concern. As the average life expectancy increases we either have to plan for a longer period of non-productivity (in the private sector) or work longer. A blend of the two seems to be what is occurring, but over my life time I have seen my expected retirement age continue to push slightly further away.

I expect it to be non-existent for a huge chunk of the economy. Well paid programmer types can FIRE, but honestly they can keep working until they can't sit upright or think clearly. Folks like welders have their eyes and knees go in their 40s.

It depends upon how one defines success. If you mean Bill Gates or The Beatles level success, I will agree that luck plays an integral role.

If you just mean a successful career that takes care of your needs and much of your wants, provides for a comfortable life and a family, I would argue motivation, dedication and skill play far larger roles.

In fact I would say extreme success requires some extreme good luck, whereas a more "standard" success requires good luck less and is more about not having bad luck to counter your abilities.

Assuming that is a legitimate question: No. That data resides in many places... with the loan servicers, the NSLDS, the Guarantors... spread in locations around the country.

To summarize: write a spec for the crime, that way when you get to development it is consistent and focused.