How would I even notice? I've given up using it months ago because they've stopped rolling out models about half a year ago in most regions.
Just about the only benefit (for us) was data location sovereignty. We have customers that had to keep their data on-shore. Deploying models in our local region ticks that enterprise compliance checkbox.
However, in some other regions the already deployed models have slowed down to a snails pace, eventually timing out more often than not. No new models have turned up in something like six months, maybe longer. It's to the point that I'm getting "model about to be deprecated" warnings for the latest model we can deploy in our region.
At the end of the day, OpenAI models are just software. Some of the models updates are not even that, they're just data. The runtime is essentially stateless. It's bad enough that the regions are not kept in sync with model data, but global outages? That's a bizarre level of failure.
We currently deploy to an entirely odd region way outside our geographical location because it had the most available models when we set up, even then it still lagged behind and only got gpt-4o recently.
I find the whole concept of a "deployment" odd, clearly it's not deploying anything other than a config enabling access for your account. Great, now I have to keep track of every model release and have a DevOps process to keep on track!
Also yes we noticed slow downs, but as we've stopped using openai directly I don't know if this just applies to azure.
Ruled out because they prohibit sharing your chat logs with other AI (means you can’t post outputs of these on the public Internet without technically violating legal terms of a megacorporation and thus exposing yourself to a legal risk):
# Closed AI (Prohibitions):
1. OpenAI claims it’s illegal, harmful, or abusive to use outputs to develop models that compete with OpenAI, which is beyond anticompetitive and goes into extremely unsafe territory given how vague “develop models” is
2. Anthropic forbids use for competing products or services which is everything since it’s AI which can do “everything”
3. Llama can only teach itself and if you have a successful llama project they can unilaterally revoke your license when you cross some arbitrary number of users
4. Microsoft AI including GitHub Copilot is prohibited to use for AI (ironic since they trained on FOSS codebases, and then any Copilot code taints your codebase)
5. AWS models are prohibited for use to develop competing things (aka, you technically can’t use it for anything)
6. NVIDIA models unless licensed under the Open Model licenses (aka, you technically can’t use it for anything)
# Open Source AI:
1. Local models from HuggingFace with Apache or MIT licenses (or roll your own) - Qwen, Mistral 7B instruct models are good, could potentially use Gemma 9B
2. Groq as long as you *change the default off of Llama every single time you reload the page* (use Gemma or Mistral, no prohibitions on learning from them). Groq has the best ui of the free options otherwise, but has no chat logs. Groq is insanely fast
3. Mistral is free and good. Wish I could pay for no limits. Not sure what they’re thinking with that, but they seem to only charge for API usage or production codestral licenses
4. Google Gemini fixed their “additional terms” with the prohibition on use of the ML tool for ML, so you can use Gemini for work, but it’s a head scratcher for them to require you to give up privacy just to keep a chat history.
Otherwise, I wouldn’t know,
pretty sure Character AI is “noncommercial” based on the BS excuse it’s for personal use only, so Character AI belongs in the closed AI list,
so does perplexity because they scrape but have prohibitions on training on their results, so you can’t share or train on perplexity results, any gpt / Claude wrappers are out, etc.
Grok from xAI wants to be open but their policy is you can’t copy the content so that’s closed AI afaik
Also I believe the Amazon stuff is closed because if you ctrl-f “compet” on their service terms they have prohibitions on using the AI to develop competing stuff.
Same deal with NVIDIA, they prohibit use to develop competing things, and everything competes with AI, so they’re out.
I am not a lawyer but suggest everyone avoid the ones in the former list, not worth your time reading their long legal terms, and to pay for output you can’t use, don’t use em, don’t support anticompetitive AI companies,
Ah, one more goofy one is you dot com, they went as far as to write they’d call authorities if you offend them. So that’s a hard no.
use the ones in the 2nd list and you don’t have to worry if you can share outputs and train on em
Do you know any model not from OpenAI or Claude Anthropic that offers JSON schema bounded responses as a feature? That feature really helps me a lot when developing LLM-powered services.
you forgot one obvious solution, but seemingly most people ignore this option: contact the companies
If you are big enough, you setup your own terms and conditions and negotiate a master agreement. I work for a rather large insurer, and we can use azure openai, nvidia workbench and gemini
We are also only using it because of GDPR requirements of not transferring data outside of the EU. The slowdown is huge, preventing any chat use-case where users will get bored waiting for a 10 second response to trickle through.
That OpenAI themselves aren't spinning up a data center in the EU is beyond strange for me. The only thing I can imagine is that if this is stopped by some legal provision as part of Microsoft investing, but it has clearly not worked well for OpenAI which now has to deal with Microsoft's shitty delivery of a subpar OpenAI-branded service for EU customers.
We're currently using the swedish region btw, where model updates have been quite fast (weeks after official launch). Still, the slow response times makes them non-viable anyways...
Just about the only benefit (for us) was data location sovereignty. We have customers that had to keep their data on-shore. Deploying models in our local region ticks that enterprise compliance checkbox.
However, in some other regions the already deployed models have slowed down to a snails pace, eventually timing out more often than not. No new models have turned up in something like six months, maybe longer. It's to the point that I'm getting "model about to be deprecated" warnings for the latest model we can deploy in our region.
At the end of the day, OpenAI models are just software. Some of the models updates are not even that, they're just data. The runtime is essentially stateless. It's bad enough that the regions are not kept in sync with model data, but global outages? That's a bizarre level of failure.