Seems like a reasonable and thought-out decision to me. Not sure why the comments here are so negative. Clearly LLMs don’t fit into Vivaldi’s vision and the features they’d like to provide to their users.
For me I think it's because the article appears biased. It's not just "we can't think of any problem it'd solve", it's "LLMs are bad and you should feel bad for liking them".
I don't really have that feeling. The author says that LLMs make stuff up and are often wrong about non-widely known facts. They present hallucinations in a confident way, which makes them come across as liars. This is true. That is exactly what the most popular LLMs do.
They don't say that you should feel bad. The author simply says that the characteristics and unreliability of LLMs make them unfit to be implemented in the Vivaldi browser at this moment. They even close off the article with a positive sentiment about the future of the ML field. They don't burn any bridges there.
> While they do this, they also use vast amounts of energy and are happy using all the GPUs you can throw at them which is a problem we’ve seen before in the field of cryptocurrencies.
Really hit the false equivalence nail on the head.
Cars enable long-distance travel, which in many parts of the world is now essential for survival, and often that is, ironically, a result of climate change.
LLMs do nothing to aid human survival. In a post-apocalyptic world, cars would be sought after for both transport and shelter, but nobody would give a care that we lost ChatGPT.
I'm not sure if cars use more energy than LLMs; huge "citation needed" there. But for the sake of argument, let's assume that they do.
Cars use more energy than LLMs because most of the world is built around requiring a car to get to places.
It's not that we don't mind it; quite the contrary, in fact, as we're always reminded of when the price of gas/petrol shoots upwards.
When the cost of living forces you to live 45 minutes away from your job by car because living within walking distance is astronomical, many folks are conditioned to want a house with a garage "for the kids" because apartments are "for the poor," and EVs, in their current state, are not financially feasible yet (though that is changing!), it's more that we're forced to tolerate it.
That's about the cost of energy, not about the consumption. If you don't think the cost of gas is worth it, you don't use a car. If you don't think the cost of electricity is worth it, you don't use an LLM. It's literally priced in.
I know people don't mind the cost relative to the value they're getting, because if they did, they wouldn't use LLMs.
Are you giving my post a fair reading or are you trying to state that there are no LLMs that cost nothing to run? I'm referencing the LLMs that let me query them at no cost... It'd seem as if you didn't read my post at all if you thought I was making reference to something else.
I don't know how clear you meant to be, but it's two sentences. You may want to elaborate, because all I can see in your post is you attacking a straw man of "I can use ChatGPT for free, therefore it must use no energy, which MAKES NO SENSE".
Not sure if it’s biased, it’s a serious limitation of the technology. Some countries are already struggling with their energy grid and water demands as is.
I think it’s fair to ensure that you create something that isn’t subject to possible quotas in the future
... But I mean this is the case. What do you mean by 'biased' here? Is it 'biased' to mention bad things about a thing? To be truly unbiased, must one praise everything while glossing over the problems?
Yes, I agree with you. The author made the point about LLM's facilitating plagiarism not me.
My point is, if that is his argument, then copying is the same thing (which is ridiculous of course).
The plagiarism argument is not necessarily directed at the user of the browser. Ask yourself who is doing the publishing? And how would you cite the output of an LLM if you did publish the output?
I took it as more of a recognition of the downsides to LLM tech that the hype train tends to ignore, as well as the legal / moral / ethical gray areas that exist in both training those models and determining who is liable for what they output.
> While they do this, they also use vast amounts of energy and are happy using all the GPUs you can throw at them which is a problem we’ve seen before in the field of cryptocurrencies.
How is that weird? Current energy consumption levels for LLMs are definitely problematic. That's not to say they can't improve, but at the moment it's definitely bad. The estimate of ongoing power consumption for Google's AI summary feature, if it was made available for every search result, is roughly equivalent to the consumption level of the entire country of Ireland. That's not awesome given most of the world is not powered by renewable energy sources.
Don't look at what it says, look at how it's written:
> While they do this, they also use vast amounts of energy and are happy using all the GPUs you can throw at them which is a problem we’ve seen before in the field of cryptocurrencies.
"Vast amounts of energy", "happy using all the GPUs you can throw at them", and then the random cryptocurrency reference to evoke all the bad stuff associated with those.
That's how a lot of folks write when they want to make a shorthand comparison and they trust their readers to understand what they're doing. The author is making a comparison of the current cost / benefit of LLM tech to the cost / benefit of Crypto. I don't necessarily agree with that comparison, but using an evocative writing style that can say quite a lot with very few words isn't a problem to me, especially given what this post is intended to convey.
Edit: I do think you're making a fair observation, I just feel there are a few reasons not to draw too strong of a conclusion from it
> Despite all this, we feel that the field on machine learning in general remains an exciting one and may lead to features that are actually useful. In the future, we hope that it will allow us to bring good privacy-respecting features to our users with a focus on improving discoverability and accesibility.
It's likely because people enjoy the benefits AI brings in their current form. I honestly cannot imagine my life now without gpt4/claude as it saves me hours writing shell scripts and transforming vaguely structured data.
Edit: yes it does make mistakes, but it takes vastly shorter amount of time to find them rather than writing something from scratch.
> Edit: yes it does make mistakes, but it takes vastly shorter amount of time to find them rather than writing something from scratch.
To me, it feels like there will definitely be features or even entire products out there that wouldn’t exist without something that lowers the barrier of entry far enough for someone to overcome the zone of proximal development: https://en.m.wikipedia.org/wiki/Zone_of_proximal_development
Just the other day I was able to introduce some custom ESLint rules into the project to make the import names match the filename (e.g. when someone renames a Vue component but the import names don’t change or you don’t notice those usages, so you end up with bunches of <OldComponent /> all over the place). I wouldn’t have had the time to do so without ChatGPT/GitHub Copilot nudging me in the right direction and giving me examples to work off of, because frankly my searches for about 15 minutes didn’t turn up anything either.
Clearly that counts for something because nobody in any of my projects had done that previously, now it’s obvious that basically we need a local package added to the project (in package.js) but also that we can store it in the file system and just use the file: syntax. It was much the same story with some .cjs Node prebuild scripts that remind the developers that they should make examples of the new components they make in a development showcase page and so on.
Aside from that, LLMs are useful for boilerplate stuff (hello, ASP.NET and Spring Boot), as well as simple one shots when you don’t want to rack your brain coming up with a particular RegEx, amongst other cases.
If someone wants an LLM in the browser, good for them! If someone doesn’t want to add them to their browser project, that’s their rightful choice.
LLM's are not useful enough to me to want them integrated with my daily-drivers (web browser, email client, etc.) - so this is a great take & one that I personally identify with.
This very much. I think folks should center around a couple of dedicated LLM web apps like chatgpt.com (plus maybe gemini.google.com for a second opinion) but that should be it.
If every company and their dog started churning out their own LLM as a business or revenue model, one can easily see the supply overwhelming demand here apart from many other problems.
> If every company and their dog started churning out their own LLM as a business or revenue model, one can easily see the supply overwhelming demand here apart from many other problems.
This certainly seems the case on the iOS/Play stores. Primary AI apps from the big hitters (OpenAI/Gemini/Copilot etc.) but then loads of knock-off ones from those riding the AI hype train. All the knockoffs being basically wrappers around APIs from the big hitters with ads/in-app purchases.
I don't want them in my browser either, but if implemented correctly, it would just be another feature you wouldn't use. So Vivaldi deliberately not having this feature only hurts people who _would_ find it useful, and thus the adoption of their browser. This is not a good strategic decision.
LLMs are difficult to run and very few people have the hardware to run a big enough model to be useful. This means you have to integrate LLMs over an API, which costs you money. LLMs are to expensive for a free browser to just give away free usage. Implementing APIs that you can login with an account and use a specific LLM is a lot of work and difficult to communicate the privacy nightmare that comes with sending that data to some LLM.
You don't have to cater to every user. Focusing on things you do better for a huge portion of potential users is a much better decision. If features turn out to be so useful that everyone needs to integrate them to not be left behind you have to do it. LLMs are not there yet. They are to unreliable, expensive and difficult to run locally. In a few years when most users might be able to run decent models locally you can reevaluate that decision
The opportunity cost of it though, like every major feature decision in a browser, is presumably very significant. What could they be doing that their users actually want?
As a dyed in the wool Firefox user, I am of course very used to disappointment about how they direct their development effort. There are a million things I'd rather see them work on than strapping in an LLM. I don't know much about the Vivaldi community but I assume they took this into account when making this decision.
I dislike the trend of baking LLMs into browsers as well, but this is not a good strategic decision. LLMs still provide features that users find useful, even if they're not perfect. So if Vivaldi doesn't provide these features based on some philosophical stance, users will just choose another browser. Not that Vivaldi had a significant user base to begin with, but this decision won't help them.
Their marketed stance might be philosophical, but it also costs time, money and energy to add LLM-backed features (someone's got to do the inference, for instance). So it could definitely make sense to them for other reasons too, right?
With so many LLMs I thought the least they would do is let the search bar redirect to an LLM of your choosing (preferably with search keyword shortcut). Instead I read this:
> So, as we have seen, LLMs are essentially confident-sounding lying machines with a penchant to occasionally disclose private data or plagiarise existing work. While they do this, they also use vast amounts of energy and are happy using all the GPUs you can throw at them which is a problem we’ve seen before in the field of cryptocurrencies.
Odd to compare to crypto which only has "store of value" as a use. It would be interesting to compare the energy of an LLM doing research tasks vs other means (e.g. a human working 8 hours a day).
It is in my screen reader, but it is used mainly in the browser - accessible description of an image without alt text. Translation of languages which I don't know is also nice.
Generally, I thing that the llm should be its own service and everything else should have an easy way to connect to it, but I'm a lowly user, not a product manager.
For accessibility it is truly awesome. Yes it can be wrong and isn't perfect, but the alternative is having no knowledge of the image at all. Alt descriptions are often missing or not detailed enough to be useful for vision impaired.
Having much better text to voice could also be nice for the blind. While screen readers are fine I don't know how bothersome that robot voice is for longer texts.
I'm okay with robot voices, tbh. The neural voices need modern hardware, and there is latency even then, together with some artifacts. Especially when speeding through a known screens I prefer responsiveness over fidelity or other nicities.
one:
in Arc browser, if your Ctrl-F find window doesn't find and exact match, it turns into a LLM question box that will try to answer based on the page content
Personally, I’m not interested in LLM in a browser as a chat bot. I do like Arc with their summary, but I’m not interested in asking questions etc - I’ll use ChatGPT/Perplexity for that.
In fact, I’d prefer it not to be included as it’s bloatware.
Was using Vivaldi on Android, quite a solid browser.
Unfortunately, the lack of AI functionality has made Edge a more attractive browser. Very useful to let AI summarise a large wall of text when browsing Hacker News for example. If only they'd add a 'close all tabs but this one' feature.
> Seems strange that people were damning algorithmic social media walls but are now welcoming preprocessed reality.
Are they the same people? The vast majority of users are consumers, they don't even think about these stuff, they only complain when the product degrade.
I don't see the privacy crowd who were conscious about their data and algorithmic feeds, are treating commercial LLMs products differently.
It's different though, isn't it? Social media walls are the social media sites giving you what they want to show you. AI tools are (mostly) you asking for what you want - albeit you're at the mercy of whatever the AI gives you back, but usually it's what you ask for.
> AI tools are (mostly) you asking for what you want - albeit you're at the mercy of whatever the AI gives you back, but usually it's what you ask for.
Reality is "modified reality". There's no reliable way for you to get an unfiltered view of anything. This has been one of the large issues of philosophy since pretty much forever.
> Very useful to let AI summarise a large wall of text when browsing Hacker News for example
This right here is why I'm against LLMs big time.
Walls of text are long for a reason. Yes, many of them might have filler. But sometimes there is nuance in the filler that a summarizer will happily throw away.
I don't want long form to be Instagrammified --- condensed into a ten-second paragraphs with 40% of the details that perfectly fits an endless consumption model that's perfect for advertising.
I think the user has to mentally tune out the bad websites that significantly use fillers -- most "news" sites. AI summary is definitely time saved for such websites.
Well written articles deserve to be long form and not summarized as you note. But the large majority of web articles belong to the former category.
> Unfortunately, the lack of AI functionality has made Edge a more attractive browser.
To be perfectly honest, it’s a good browser. Being able to explain console errors with a click and them adding some resource limit functionality (like Opera GX) are all welcome. I even tried it on my MacBook and it works without issues.
I did go back to Firefox later though since I prefer those DevTools otherwise and because a recent update no longer makes Firefox fail to do DNS through my VPN.
I understand this point of view; it's a reasonable action to stop going in this direction if some valid concerns are still unanswered by the industry. At the same time, I suspect that a good part of the people who complain about LLM hallucinating underestimate the human rate of hallucinations/errors.
Of course, there are a lot of other considerations linked to topics like accountability, etc.. but take autonomous driving as an example: we don't need self-driving cars to be perfect and do exactly 0 incidents to be a better solution than human driving.
They just need to be significantly better than us to be a very useful technology for mankind and let's hope this happens soon.
Honestly, I see no reason why AI should be part of browser itself. If user wants those capabilities they should be free to choose extensions that add them. So they get to choose which AI or LLM model they use.
It's pretty nice that a web developer can leverage an offline local LLM, without leaking data, instead of piping stuff through their servers and then OpenAI servers.
My browser should be my user agent, not my user nanny. I don't want it depriving me of features or working against my interest, ever, on the grounds that the browser author's personal opinions on the social impact of various technologies.
The AI hype relies on most common people misunderstanding what's in front of them.
I've seen business people write prompts. They don't understand the limitations or strengths. They tell the model it has a degree in something, they tell it not to hallucinate, they anthropomorphize it so far that they greet it in the morning "to get into the mindset that you have a virtual colleague".
All the people who fearmonger about AGI really don't help, because it implies our LLMs are anywhere close to sentience. They're not.
For people in agreement with Vivaldi, how are you summarizing articles today? I often find content that's not important but also too verbose, and having an LLM summarize it for me is perfect.
Right now, I'm doing this very manually, but am actually thinking of switching my browser just to have this capability baked in.
To be fair most people don't read it, they read the headline and move on, the oft heavily editorialized clickbait headline. At least there's some semblance of neutrality, on the assumption that the article is correct and the LLM correctly summarizes it.
For fictional/narrative content, sure; reading the Cliff notes version is missing out on the full experience.
But for a lot of non-fiction content, I think it's perfectly legitimate and common to have some specific piece of information you're looking to extract, or want a rundown of key highlights.
For both, I'd say it could also help in choosing which content to read. Books already typically have a form of summary on the back. If you're researching something, you're going to have to determine which articles are relevant before reading every article top to bottom - and a summary seems helpful for that.
With summarization, you still read, but have to read less. These days a lot of articles have a lot of verbose content, just to keep a person on the page for a long time. Reminds me of students writing essays just to get to a word or page count.
The way you ask that question implies that an article summary is some kind of necessity. What happened to just... reading the article?
Most of the time, I don't want a summary of an article. If the article is really good and I want to know the most important details, then I should read the article. Maybe I can write a summary myself, so that I can use it for future reference.
If an article is bad, then why would I care about a summary? I wouldn't read it either way.
There is a a possibility that an article is actually really good, but it's badly written, or way too verbose, or I don't understand the language/terminology/topic. In these cases, a simpler summary written by someone else (or an LLM, I guess...) would be useful. Domain-specific articles or scientific papers would fall into this category for me. Even for this subset of articles, if I don't really understand the topic being discussed, would a summary really give me any benefits? I still don't understand the topic well enough for it to be useful to me. I would get more out of actually trying to read the article and studying the material being discussed.
How do you know if an article is good or not? You have to waste your time reading at least some of it. I've found it useful to have ChatGPT generate a summary, then if the summary sounds interesting, then I read it.
There are heuristics developed over time for determining if an article is worth it. Far harder to tell whether ChatGPT is just making things up (it does all the time).
Pretty easy. Is the headline/topic interesting? Read the introduction. That's interesting? Read the next paragraph. Losing interest? Skim a few para's, pick out interesting headings, illustrations or keywords or skip to the end and see what the conclusion is.
This is active reading, not some robo-moron digesting an article, shitting it out and then spoon-feeding the contents of the toilet-bowl back to you.
That's what most people do I imagine, me included, but we might have to adapt. If more and content out there is just a bit of information padded with unholy amounts of LLM lorem lipsum, there won't be a choice.
What kind of articles are you reading where this is useful?
I either read articles that are 1) fun to read and silly, or 2) fun to read and informative. If they're not fun to read, it's unlikely that the content is any good. Good educators often know how to write in an engaging way. If the content is difficult to read, or super lengthy, it's probably shit content.
How do you judge quality of the information you're given when you have it summarized? How do you make sure the LLM didn't misunderstand something, like sarcasm, or a quote, etc.?
Gruber has always been loquacious, but this one seemed excessive to me. Still, I was curious for his overall impressions on Apple Intelligence.
The summary sounded interesting to me, so I then dedicated time to read the source. To your credit, ChatGPT's summary was much more critical than the actual article.
By choosing data sources that respect its readers, the same way I pick lean code architectures rather than pile on more and more layers that require additional tools to deal with the bloat.
> how are you summarizing articles today? I often find content that's not important but also too verbose
Then why are you reading it? You’re conditioning yourself to have a mental corpus of uninteresting, unimportant drivel. At best, that’s a waste of time.
Have no problem with them not following the trend, maybe even a good thing.
Their reasoning is shoddy. Misunderstands LLM's and plagiarism is of no practical concern to a browser, may as well remove the copy function from the browser if that's the view, theres also use cases where hallucinations don't matter such as the AI tab sorting and semantic search. He should've just said they don't see the value in it and left it at that.
Vivaldi user here. I also use LLMs - I subscribed for ChatGPT for a while, now I mostly play around with ollama. I also played around with Stable Diffusion extensively.
I think the article is excellent in its analysis and conclusion.
As much as I like the recent advancements in Generative AI, I don't think it should be hamfisted in every software, and I also don't think they come nearly close to all the hype surrounding it. It is a useful tool when you understand its shortcomings, and there is a long way to go until they are ready for more widespread use. Good on Vivaldi for taking this stance.
As a long time Firefox user, I replaced FF with Vivaldi on Android and haven't looked back, since FF has failed to address the battery drain issues that so many people are experiencing:
I use Vivaldi because I used to be an Opera user a long time ago. Absolutely loved it, and Vivaldi leans a bit on the same interface.
Firefox is my backup browser. I really like it on desktop, but I agree that on mobile it feels clunky.
I see no reason to hamfist LLM into the browser itself yet. A lot of the ideas surrounding LLM seem to be more on the style of devs trying to latch on the latest trend without any proper thought of how useful/desirable LLMs can be on their product.
Also, while I don't share the ethical concerns surrounding how LLMs highjacked intellectual property, I think those are legitimate concerns. I have no issues with a dev taking a stance based on it, especially on a product I use for free.
I don't know what this is meant to prove or disprove, I can only assume the constant urge to compare human beings with LLMs has simply become a pathological drive at this point. Yes, humans are capable of reproducing copyrighted text verbatim, as are LLMs, and when they do, as with LLMs, it is copyright infringement and plagiarism.
An irreducible and unavoidable tendency towards committing copyright infringement and plagiarism are not features one generally wants in either a human author or a piece of software.
It means that if LLMs mimic humans in learning and creating, then we should probably apply similar standards to LLMs. It is impossible to create an LLM without training it on copyrighted text. Just like in humans.
Should the world refuse to use LLMs because of that?
Spoilers: They couldnt even if they wanted, they dont have $budget for it. They even struggle paying normal developers, moving most of the stack to cheaper javascript.
I don't use Vivaldi, but I think it's fine on their side to avoid falling into this AI trap. You have to consider that there were 2 possibilities, they used their API key to send requests to openai/claude or they did what Mozilla did and integrated gemini/chatgpt at the expense of the user data, both of which are pretty scummy and barely help the end user. The only way to integrate AI fine into browser is to ship a model to run local, but that would require the user to have the computing power to run it. Google is trying this with chrome by integrating gemini mini in the browser as a 100mb blob.
I think this is the correct move. The correct one (and I'm not sure if this is feasible) would be integrating the OS level LLM facilities. AS an independent browser vendor, trying to chase the AI trend mirage is like trying to empty the ocean with a bucket.
Still, there is some marketing damage control dismissal of LLMs in the note that could have been avoided.
Author is francophone (Belgian). This explains the order of the words. The french order would be the same "Pourquoi Vivaldi ne va pas suivre l'actuelle tendance IA"
Statistically speaking, maybe you are not in the minority. Given that the vast amount of people get their information from social media and take it verbatim.
> Doing your own research has lead to an incredible advance in public intelligence
The situation is analogous to war reporting. War hasn’t become more brutal (holding scale constant). We just see it more clearly. Similarly, I don’t see evidence we have more morons. They’re just given more attention.
Yet there are too many non-experts thinking they "did research" and then go on to ignore medical or political facts. Those people are easily influenced by adversarial misinformation to incite hate and anger for political warfare.
that just screams "I don't know what this technology is and I am too afraid to use it" to me personally. Don't use it as a source of information if you don't want it to spew hallucinations to the user? There's about a million other uses for LLMs.
Hahaha... search is essentially a solved problem if you are literate and work with well-structured data sets. The market for a tab sorting solution is zero.
Search datasets still are not really semantic search, and I'd wager many require online search if they required a well structured dataset, or a dumbed down query.
You only have to look to the relative popularity of something like Arc browser to determine there is a market for such things.
We are discussing their decision not to put AI into their browser. It's a good decision, because AI is the Star Wars-like arms race (Star Wars from the Reagan era, not the movies) designed to bleed the competition dry of their resources. Microsoft is putting it into everything and bankrolling OpenAI, because they want to hurt Google not because AI is good for anything. Vivaldi made the right choice to stay away from this distraction.
I have no issue with the decision not to use AI, and agree perhaps there's little merit.
I'm merely listed potential uses for it in the browser, in response to a question why it would be in a browser.
Sure, I mean i'm not their product team but surely LLM's could easily complement things like parental controls, autocomplete mail / language assistance (there is mail in the browser), in-browser language translation etc
> was made by having many (underpaid) people to look
"underpaid".
I wonder what this has to do with the discussion on how LLMs work.
I'd like to know more why does Vivaldi team thinks these people were underpaid, how do they know how much per hour they were paid and what would be a fair price for the job according to Vivaldi's evaluation of the scope.
I'm really interested to know how did the Vivaldi team conclude that OpenAI forced unwilling people to do the work while paying less than what these people expected to receive for their time.
And most importantly. What does this have to do with LLM integration in browser?
At a glance, it feels more likely to me that they're criticizing training LLMs on content without compensating the original creators, not the salaries of OpenAI engineers. It's a pretty common moral / ethical stance to take now.
That seems more likely than my initial interpretation, in which case the moral and ethical implications just so you can have a "Summarize with AI" button or other such features in your web browser are obviously much worse.
> It feels more likely to me that they're criticizing training LLMs on content without compensating the original creators,
No, the phrase they have written specifically talks about the RLHF workers and they do evaluate the pay of these people and somehow bring this issue into the discussion of how LLMs are useful to browsers.
And because I'm being downvoted for asking a simple logical question whether this has anything to do with LLM features in browser, I logically conclude that there are some things in current American culture that I literally (literally!) don't understand.
It was killed by fear-mongering, a common cause of death for new technologies. (Remember the press fomenting a moral panic over Google Glass?) The idea is sound and will be part of our normal day-to-day computing experience in the future.
Right. Having a plain SQLite database with everything you do on your computer accessible to any program you or Microsoft run on your machine isn’t a problem at all? That open source utility you installed and forgot about that can now gleefully compile statistics on your porn usage and threaten you to send them to your spouse, whose email address and phone number it was able to pull from that same database, unless you pay a few bitcoins-that isn’t an issue at all? The fact that this was enabled for professional users handling your patient records, insurance details, personal finance, or your stock trades and make all that available at a whim, that is nitpicking? The TLS-encrypted API calls Windows computers make to Microsoft servers multiple times per day, any of which could include data from that database as anyone at Microsoft sees fit, they are paranoia?
> Right. Having a plain SQLite database with everything you do on your computer accessible to any program you or Microsoft run on your machine isn’t a problem at all?
We're way beyond the days when every program you run has equal access to all your data. These days, different programs have different levels of data access.
Are we, though? If I run an application as a normal user on Windows, it will run with my user account's privileges. And even if Microsoft put special care into the permissions on the database file—it's not like your typical Windows user wouldn't carelessly accept any UAC dialog they are presented with…
There was no fear mongering involved. Google Glass was an objective privacy nightmare just like Recall and in fact not just to the user, which could theoretically be defended as choice, but importantly to others.
You're basically exporting surveillance as an externality with recording technologies that by design collect everything around them. These technologies were basically multi-modal keyloggers.