It does make it easier for the end user who doesn't want to fiddle around with python dependencies, command lines, building C++ projects, etc.
Just install it, point it to a model, and go. Now you have a local LLM.
If you want something more, click the "start server" button and you have a local OpenAI compatible API which you can point more advanced front-ends to.
Unless I just can't find it, there seems to be no setting for customizing the prompt format for local models. You can edit the prompt itself, but not the format of the prompt or the subsequent messages. This would make using many models difficult, or give poor results, since they don't all use the same format.
GPT4All makes it annoyingly difficult to run any other than their "approved" models. I'd like to kick the tires on a whole host of random GGUF quantizations on Hugging Face, please.
I've poked around the doc, not sure if Jan can do that better.
In the mean time, I use text-gen-ui (Oobabooga) as a back-end and have it run with `--api` to use the front end of my choice.
I got say I’ve been using LLM studio as it exposes the models in the ui as well as through a local open ai compatible server so I can test different models against my workflows locally.
I try some LLM on my notes and well... They was unable to give me insights that are hard to spot, like follow the flaw of notes identifying patterns, find similar notes from the past and so on. In ALL cases classic tags/riprgrep full-text search was far quicker and equally or more effective.
Long story short: LLMs might be useful on hyper big mass of information, like a new kind of search engine that try do achieve a semantic goal mimicking it. But not more than that IMVHO. Marginally LLMs might help computer-illiterate to manage their files, seen https://www.theverge.com/22684730/students-file-folder-direc... but I doubt they can go any further for the next 5+ years at least.
They've been very useful in quickly answering common questions using a too-large-to-manually-scan knowledge base in my experience at my job, and I don't consider myself or my colleagues "computer-illiterate".
That's follow the "might be useful as a new kind of search engine", though it might be a sign of an a bit messy KB. The issue of potential hallucinations however is still there so even such usage, a different search engine, demand extra attention.
It's not a free critic to those who have designed, implemented and trained LLMs, it's just the observation that practical usage is far less than the advertised one and it's still not much good. It's still an advancement, a good thing to have, the start of a revolution, but still far from being what many dreams.
This looks awesome. Trying it out. Suggestion, can we please change the "Download Jan for PC" to perhaps just "Download" or "Download for Desktop" or whichever that makes sense but not "PC". I almost move away thinking this is Windows, thus not for us.
I recently stumbled on https://mindmac.app which is a non-subscription app that uses multiple AI tools (not just OpneAI). Looks Promising.
These are some of the really good ones. I'm tending more towards trying out the likes of MindMac just for the fact that I can plug and switch between multiple tools.
What's the value proposition for TypingMind as a commercial product ($3500 to run locally for 5 seats)?
But let me contrast that last "native app" with Machato and MacGPT:
== Machato ==
Machato is feature-full for system prompts and transcripts, connecting to to OpenAI, Claude, and any "server" endpoint that's OpenAPI API compatible, and surfacing parameter and token settings per conversation right on your text entry bar. You can also point a given conversation to a local ollama endpoint such as Mixtral 8x7B and it works as well.
The best feature is the selective forking and suppression of exchanges within conversation threads.
MacGPT is highly integrated throughout MacOS, and works with either OpenAI key or a ChatGTP Pro login. It's quite similar to BoltAI mentioned elsewhere in this thread, but in addition to the OpenAI key based mode, also works with a ChatGPT Pro subscription in a ChatGPT web UI pop-up.
This looks great relative to others (very similar to MacGPT), and I particularly like how advanced settings are available but tucked away behind discoverable affordances.
It's interesting that you have team pricing.
Can the Team leverage shared system prompts and/or assistants from a OneDrive-for-Business (SharePoint) folder or GitHub repo?
If not, what makes it "Team" instead of just individual?
Hi. Actually I don’t have a pricing plan for teams yet. It’s still under (heavy) development. I changed my headline to reflect the direction I want to take this year (focus on teams)
And yes, some of my customers wanted team and collaborative features like shared prompt, internal plugins and integrations, RAG on internal documents…
But I haven’t launched these team features yet.
Are you interested in this? Would love to talk to you if it’s something you’re looking for.
From the screenshots it looks like there is an activation limit, with a maximum of four devices. Reading the license, I could not confirm this. Is there a limit, and if, what is the maximum?
Sorry for the confusion. I need to improve my pricing page.
The license is per user, and can be used on maximum 3 devices. I figured this is enough for most users. If you have more devices or need a custom license, please send me an email (my email is in bio)
Still hoping we'll eventually stop using Fibonacci to show off recursion, because that's one of those examples where the maths might be expressed as recursive relation, but the implementation should never be =)
Good AI would go "you don't want that, that's horribly inefficient. Here's an actually performant implementation based on the closed-form expression".
Is it implied anywhere? That's a feature I'd love and also why I haven't bothered delving into LLMs very much; I didn't know there were any that could locally index your library and train on that data. I'd love to ask it a question and have it reference my local ebook library.
It probably doesn't. The only one that read PDFs for me was the Nvidia ChatRTX.
It would be easy to add modules from pip that do this but you'd have to code up the input pipeline. It's not terribly difficult but it is definitely not point and click.
It seems like that until you actually try to use them. Not many are actually polished, support formatting, history, and multiple endpoints. There's lots of trivial apps abandoned after a few days, but what are the actually functional, good quality alternatives to this one? (That don't pass your query/answer through a third-party for data collection)
I use https://www.typingmind.com/. It is paid, but I've found it to be a reliable front end to OpenAI/Claude/Google, supporting everything you mention. I haven't done any hyper detailed security audit but after watching network requests I'm pretty confident it's not sending my chats anywhere except to the relevant provider endpoints.
Considering how much I use it, I've found it to be well worth the cost. The creator is pretty on top of model/API changes.
i’ll second that recommendation… i use it through the SetApp store and i’ve been very pleasantly surprised by its documentation and ability to work with most services.
For AI projects, afaict, 12k stars or forks is more akin to downloads than contributors & downstreams. GitHub is the app distribution, not just source distribution. I've been curious how to model this better..
> If I had a key for some OpenAI paid shit I can go to their website and do not need an app for that. I really do not get it.
Perhaps you don't have some OpenAI paid shit?
While you can clunk around in a sort of OpenAI playground in a web tab, it is designed for dev experimentation (a "fiddler" type of UI), and not a good experience for much beyond testing.
> excuse me you fuck did you not just tell me you are "local first" ... I try out the model, and it turns out it runs on my CPU with heavy RAM usage...
I'm not sure it makes sense to have both of these objections at once.
OK, there is a switch under "Advanced Settings" to enable GPU. Why the fuck is this no-brainer option off by default and "advanced"? Suddenly my 7B Model is shown as inactive and "start" does nothing, gives no error message either ... this is such an unusable alpha version.
Just install it, point it to a model, and go. Now you have a local LLM.
If you want something more, click the "start server" button and you have a local OpenAI compatible API which you can point more advanced front-ends to.