I don’t think on site is going to be necessary. Even the US intelligence community trusts that Amazon isn’t spying on the spies.
But a model that can run on a private cluster is certainly something that there’s going to be demand for. And once that exists there’s no reason it couldn’t be run on site.
You can see why OpenAI doesn’t want to do it though. SaaS is more lucrative.
No, the grandparent poster was right. That’s other agencies, not the intelligence community. He’s right that the cloud I was thinking of is on prem but with Amazon personal (that are cleared).
So not the greatest analogy. But still I think most doctors, lawyers etc should be okay with their own cluster running in the cloud.
Not lawyers in the US at least, that would typically be a violation of confidentiality. Even with a client's permission, it would work a waiver of attorney-client privilege. (I don't use GPT but I'm assuming the ToS is clear that someone there can examine the input material? Can it even be used to build their model, i.e., submitted information could potentially work it's way back to the eyes of the public and not just OpenAI engineers?) I imagine HIPAA issues would stop doctors. Can HIPAA data be stored on the cloud? Every instance I've seen they store it locally.
I agree with you on the SaaS version but the scenario I was thinking of was where there is a licensable model that can be run on a cluster in law firm’s AWS account. I think that should be okay.
HIPAA data can definitely be stored in the cloud given the right setup. I’ve worked for companies that have done so (the audit is a bit of a pain.)
I work in legaltech, and we use cloud services like aws for lawsuit data, and lawyers trust it. Any 3rd party must of course be vetted and go through NDA, and follow regional laws and guidelines ect, but using the cloud is definitely used for legaltech documents including sensitive data.
It should be added that legaltech vendors are often employed as go-betweens for quite adversarial interactions, such as e-discovery, that require them to be trusted (to a degree) by both sides of a case, even if they are being paid by one side.
Seems like there are lots of confidentiality and reliability issues in how tech is being used in law right now, but there aren't that many attorneys who understand the issues, and those that do find it more advantageous to overlook them unless forced to do otherwise.
HIPAA regulated organizations routinely store protected health information on the cloud. This has been common practice for many years. The physical location is legally irrelevant as long as security and privacy requirements are met. AWS and other large cloud vendors specifically target this market and make it easy to achieve legal compliance.
Are they even aware of where their data is? Opening a web browser might be a big hint for them, but how about editing something in Microsoft Office? Does the data there ever touch the cloud? Do Chromebooks make it clear enough where the data is?
I imagine lawyers knowing about where document data is stored as a bit like software developers being sufficiently aware of licensing. There's plenty who are paying attention, but there's also plenty who are simply unaware.
> You can see why OpenAI doesn’t want to do it though.
Except they already do offer private cluster solutions, you just need usage in the hundreds of millions of tokens per day before they want to talk to you (as in they might before that, but that’s the bar they say on the contact us page).
VMware charges people per GB RAM attached to a VM. Selling on-prem software on consumption is very much possible. It's closed source software, so as long as they require 443 outbound to tick consumption that'd work.
But a model that can run on a private cluster is certainly something that there’s going to be demand for. And once that exists there’s no reason it couldn’t be run on site.
You can see why OpenAI doesn’t want to do it though. SaaS is more lucrative.