From the theinnovationenterprise.com article you linked to:
> It is essentially an interface that sits on top of existing data sets and displays data to users for analysis, helping to identify connections otherwise impossible to find. Users do not have to use SQL queries or employ engineers to write strings in order to search petabytes of data. Instead, natural language is used to query data and results are returned in real-time. It is not designed to do any single thing, its main strength is that it is flexible and powerful enough to accommodate the requirements of any organization that needs to process large amounts of both personal and abstract data. This makes it more useful for managing HUMINT, or intelligence from human sources, than SIGINT, or intelligence from signals.
The problem with Palantir is that everything it is said to be doing is so vague that it could just be colorful dashboards based on not particularly great or new data. For example, this is how the Bloomberg story you linked to describes it:
> Founded in 2004, Palantir is used by dozens of federal, state and local law enforcement agencies to aggregate far-flung data, find patterns and present results in colorful, easy-to-interpret graphics. Its use by police in Los Angeles, Chicago, New Orleans and elsewhere has raised ethical concerns about the potential for unfairly targeting minorities.
I'm not arguing whether Palantir is living up to its contracts. Just that it doesn't seem to produce or have the unique and expansive kind of data that Facebook, Google, and Amazon have. Though I guess there's nothing that prevented them from doing a massive data pull from FB's API, in the same way CA managed to do it.
If the task is "taking datasets and making them more searchable," it seems like that company gets the heat simply for making the task easier. Excel, SQL databases (or, heck, huge CSV files) don't get as much attention even though they could be used for the same purpose, even if very awkwardly.
You are right. But sometimes the data access technology does make a qualitative difference. That’s because a lot of our protection has always been a sort of “security by obscurity”. It wasn’t just the 4th amendment protecting you from unreasonable searches. The government was also restricted by their manpower, and by the fact that searching a house is easily visible and people would notice if you’re targeting everyone.
These restrictions don’t apply to digital eavesdropping, which is why it has become a major point of debate once the technology made it possible to do on a large scale.
Another example are license plate scanners: that data has always been there, and anyone could legally write down all the license plates they saw. But add image recognition and a database, and you’ve created a monster.
The tech community usually turns to technology to fight such technology: encryption for communication, Bitcoin to undermine (pun intended) what they see as the failings of the FED. BitTorrent for their qualms with copyright enforcement.
But laws and the court of public opinion are arguably our first line of defense. Underground printers didn’t stop the nazis or the Sowjets, and it’s not clear that technology has significantly moved power to the people in China, Turkey, or North Korea.
So we need better privacy laws. We need politicians to be scared before they use the services of Cambridge Analytica. And we need to convince our peers that they will have joined the dark side if they accept a job at Palantr. These sort of actions have the added benefit of respecting the processes of a civil society ruled by law, and not a techno-jungle where might is right.
> It is essentially an interface that sits on top of existing data sets and displays data to users for analysis, helping to identify connections otherwise impossible to find. Users do not have to use SQL queries or employ engineers to write strings in order to search petabytes of data. Instead, natural language is used to query data and results are returned in real-time. It is not designed to do any single thing, its main strength is that it is flexible and powerful enough to accommodate the requirements of any organization that needs to process large amounts of both personal and abstract data. This makes it more useful for managing HUMINT, or intelligence from human sources, than SIGINT, or intelligence from signals.
The problem with Palantir is that everything it is said to be doing is so vague that it could just be colorful dashboards based on not particularly great or new data. For example, this is how the Bloomberg story you linked to describes it:
> Founded in 2004, Palantir is used by dozens of federal, state and local law enforcement agencies to aggregate far-flung data, find patterns and present results in colorful, easy-to-interpret graphics. Its use by police in Los Angeles, Chicago, New Orleans and elsewhere has raised ethical concerns about the potential for unfairly targeting minorities.
I'm not arguing whether Palantir is living up to its contracts. Just that it doesn't seem to produce or have the unique and expansive kind of data that Facebook, Google, and Amazon have. Though I guess there's nothing that prevented them from doing a massive data pull from FB's API, in the same way CA managed to do it.
FWIW, not everything about Palantir is necessarily a secret. You can see some descriptions of what they offer and at what price via various FOIA requests that have been done: https://www.muckrock.com/foi/list/?q=palantir&status=done&us...