This highlights a big problem in the world of music and audio software: everyone likes the idea of standards, just so long as they control the standard. And while this problem no doubt exists in other domains, it's painfully apparent in music and audio applications because the market is too small.
Clap won't succeed longterm because of many of the factors outlined in so many of these posts. Notably, Apple aren't going to support it in Logic because Audio Units work just fine, and AUv3 spans both iDevice and Mac markets. Steinberg almost certainly won't support it; they aren't even allowing VST 2.4 plug-ins to run in the Apple Silicon version of Cubase -- and this isn't for technical reasons. Ironically, Steinberg have been trying to push VST 3 since the release of Cubase v4.1 back in 2006, and it's still a giant pain 16 years later. If you look back at VST 1/2, part of the allure was the simplicity of the API; VST 3 tried to change the game too much towards MVC given that it looked at the time you might want to run the processing part of the plug-in on a different system than the user interface part.
Pro Tools doesn't get a mention in this thread, for the most part, but for better or worse it remains something of standard in audio post-production in the US. AAX isn't a bad re-imagining of what what RTAS/TDM-based plug-ins pre v10, and Avid aren't going to relinquish any control to third parties. Not least because the need for AAX (such as it is) resolves around support for the company's hardware control surfaces. EuCon was originally envisaged as an agnostic hardware controller protocol by Euphonix, but Avid purchased Euphonix, released the S6 and associated controllers and made EuControl into something of a tolerated illegitimate child.
Plug-in developers will have to follow the money, and I would to see the results of a survey correlating money spent on plug-ins based on the main host software used. My guess is that Pro Tools AAX on the Mac probably is the most lucrative, followed by Cubase and Logic. Depressing, but for all the moaning musicians tend to indulge in regarding fair compensation against any company they feel exploited by, there's a reason why music and audio applications and plug-ins remain the most heavily copy-protected software.
I don't know of anyone who's ever ditched their host application because of a plug-in format, and I highly doubt the CLAP is going to harm Apple, Steinberg, and Avid in the longterm.
> Pro Tools doesn't get a mention in this thread, for the most part, but for better or worse it remains something of standard in audio post-production in the US. AAX isn't a bad re-imagining of what what RTAS/TDM-based plug-ins pre v10, and Avid aren't going to relinquish any control to third parties.
To everyone’s surprise (yours included, I’m sure) Avid is mentioned in the announcement as a company already involved with the project and working on using it for something. There’s a nonzero chance they could adopt it in Pro Tools and drop AAX.
True. But Avid are largely terrible at implementing new technologies in a timely manner, especially since they started off-shoring development as more and more of the old timers went to recreate Pro Tools as Luna for Universal Audio! I mean, although a Windows version of Pro Tools is available, the core of their professional user base is on the Mac, and they haven't even managed to get an Apple Silicon-native version out the door yet -- over two years since Apple provided DTKs for that purpose. (Meanwhile, Logic Pro, Cubase, Studio One, Live, Bitwig, and others have all managed it.)
Avid is not exactly transparent and it's extremely difficult to see what their long-term plans are. We can only speculate. The most convincing theory I've heard regarding Apple Silicon is that they may need a new hardware interface to work with Macs that lack expansion slots. Developing hardware is of course a much slower process than developing software, and it may take them several more years to get such a product out the door. In the meantime, Pro Tools has limited usefulness on machines that can’t talk to the hardware it’s designed to drive, so why bother releasing a port?
That's not quite the case... HDX cards can be used on Apple Silicon with Rosetta 2 via the Avid-branded chassis from Sonnet, and I've personally not had a problem with that. Of course, some driver-level work would be required to fully support HDX natively in Apple Silicon. Also, the Carbon interface works fine on Apple Silicon under Rosetta 2 as it uses AVB. More and more, HDX is less important for DSP, although remains vital for large I/O configurations. On the other hand Pro Tools 2022.x supports Core Audio (or ASIO) hardware with more I/O. And if you're running the Dolby Renderer on the same machine as Pro Tools, for example, you have to run native audio hardware in any case, using the Pro Tools Aggregate I/O option (or any other Core Audio device). So I wouldn't say these are cases of limited usefulness; during Covid I had to do a whole movie soundtrack album that way at home, running Pro Tools on a 27-inch iMac with the Dolby Renderer. But when that 27-inch iMac (the last Intel 2020 model) can easily outperform a Mac Studio when it comes to running a full Atmos Netflix dub, you kind of feel it wouldn't hurt for Avid to kind of hurry up!
It is not intended to harm Apple or Steinberg, the aim is to be less dependent of them by having a free and non-bloated common plugin api that has basically all the features of VST/AU/AAX/LV2 etc. All these proprietary apis can then be addressed with a bit of wrapping code when needed.
Right now, the de-facto most important standard is VST3, which is owned by Steinberg/Yamaha who can decide to revoke the license of any developer at their will. They have shown these last years that they can be really aggressive with their license. If clap takes over, then they won't have the same position of power on audio developers. This is good for the industry.
Even if there wasn't this licensing issue with VST3, none of the current plugin formats deserve to be the "default plugin format" , they are all terrible: very large codebases, bloated architectures, c++ api, unclear threading specifications... (exception: VST2 which is simple, with a c api, but Steinberg is not allowing it anymore for new developers).
I'm not sure how you become less dependent on Steinberg or Apple by promoting another standard that really needs them to support it to fulfill its logical destiny. If Steinberg can revoke a developer's license at will (I've never heard of them actually doing this, though, although I'd love to know examples of such behaviour) they could therefore do just that if they smell any wrapping code needed to bridge into the VST domain. Avid used to, for example, although I don't know if it's still the case, have a clause in the developer's agreement that prevented RTAS/AAX host plug-ins... I guess I'm old enough to remember when FXpansion! had such a plug-in to host other formats within another.
I do agree with you on the state of the current plug-in APIs. But unless CLAP plug-ins are hosted natively, they will still have to piggy back on top of all the crud you describe. So unless a product comes out that truly takes the lead away from Pro Tools, Logic, Cubase, et al... I feel like this is going to be a massive uphill struggle. Plug-in developers will gravitate towards where the money is; and unless users place serious pressure on Apple (that thought is amusing in and of itself!), Steinberg, and so on... It's hard to see CLAP making a dent. On the other hand, if it aims to replace JUCE as the intermediary development platform to make life easier for developers, that could be good. Although, that would seem to diminish the long term goals...
So if you need users to apply pressure on large companies, you're doomed. Even famous Cubase users like Hans Zimmer aren't going to get onboard this fight, despite his close relationship to Urs.
CLAP does not need to supplant all other plugin formats to succeed. Its real goal (arguably more ambitious) is to expand the set of features in the "least common denominator" of plugin formats. To do that, it only needs to win a race with one other format, which, as it happens, is officially already dead.
Plugin developers don't just pursue the largest revenue streams. They also try to take the lowest-effort paths to those markets. CLAP is carefully designed to be a low-effort path for porting old plugins to an SDK compatible with modern feature sets, which can then be automatically wrapped in comparable formats such as VST3. There are other such SDKs, such as JUCE, but they almost all require a much larger investment of effort to work with old codebases. The fact that CLAP also specifies a well-defined plugin format further enables developers to write automatic test suites, or adapt such test suites that were originally designed for other formats. In a certain subset of the market, CLAP has already been adopted for these reasons, and as far as its creators are concerned, it has therefore already accomplished its goals.
As you suggest, there is a certain possibility that Apple or Steinberg would somehow prohibit the use of third-party SDKs in plugin development, but in reality this would be an absurd thing to try. It would accomplish nothing, alienate the entire market, and promptly result in a flurry of lawsuits and perhaps even an antitrust investigation.
I think this is a very negative and old-fashioned opinion, especially within HN that is usually focused on 'disrupting the industry' and 'innovation'.
Every day we see a new project written in the latest 'cool' language, and it receives praise and encouragement, yet try to suggest that a new audio plugin format has value and it's knocked back because 'industry standards'.
Just to give one simple example, if Native Instruments were to add CLAP to Kontakt player, and suddenly orchestral samples could have more variance per note, making the realism better - there would be huge demand from composers.
Would they switch DAW ? Maybe not for their current work, but you can guarantee it would have a lot of interest and they would be pressuring companies to add it.
For too long, audio software has been stuck in the monopoly of a few big companies. It's absolutely time to disrupt that even more and open it up to more independent developers, in the same way open source has helped in other industries.
Rather than doom CLAP to not being adopted, perhaps look to the benefits and help encourage diversity and change.
It is negative, but you have understand how niche the music and audio market really is to put some of my comments in context. I'm all for new and cool and all the rest of it, but the end users here--composers, musicians, producers, engineers--tend to not care so much about pioneering in what you might call the HN way.
To speak to your simple example, Native Instruments could do what you suggest, but they could have done what you suggest by adopting the cool and new VST3 standard back in 2006. This would have set them up for Cubase 5, which introduced VST Expression (3.5) to provide exactly that variance within orchestral samples on a per-note basis. So this has been possible for nearly a decade and a half, and, so far as I can tell, while most composers agree it's a neat idea, by Native Instruments controlling the sampler market (and Vienna to a degree) and not supporting VST3.5, the adoption never materialized.
Also, how many keyboards support polyphonic aftertouch? This would be the MIDI message that would need to be generated in order to perform polyphonic note-based expression. But almost no keyboards support it, and ROLI created the MPE standard to provide a workaround by splitting messages across multiple MIDI channels. Interestingly, MPE has received quite a bit of attention, but this doesn't require a new plug-in format to support it. And ROLI, the company, has been disastrous in the marketplace by adopting Silicon-Valley-esque approaches, raising capital, growing too fast with needless acquisitions, filing for bankruptcy, rising more money, and it even has a leader who thinks he can walk on water!
So if I sound old-fashioned and against new ideas, nothing could be further from the truth. But to pretend musicians would jump on disruptive innovation just isn't what has happened in the past. Maybe this time it'll work, but I don't see that industry has changed all that much. In fact, in many ways it's worse. More and more sample library creators have created their own sampler for playback, so whereas Kontakt used to power most libraries, now Spitfire, Cinesamples, 8dio, Orchestral Tools, and others, have their own engine. The phrase herding cats comes to mind.
Diversity and change is the one thing that just hasn't happened in the music and audio world. For example, Steinberg introduced Nuendo 22 years ago with the intention of it disrupting Pro Tools... However, go to Skywalker Sound, for example, or any other high-end audio facility, and they won't be running Nuendo -- at least in the US. And as for open source, listen to Paul Davis talk about Ardour... For some reason, open source music and audio software has just never become a thing. And sure, it would be cool for there to be a kind of Red Hat model adopted in the audio industry, where money could be made from supporting an open-source-based ecosystem, but I think CLAP has more chance of succeeding than this reality.
I see the proliferation of bespoke sample players as both a good thing - it drives innovation, as can be seen in some aspects of Orchestral Tools SINE player - and a reflection that Kontakt just hasn't kept up (perhaps as NI has spread itself too thin across many interests).
I also think that the industry could change - After all, Hans Zimmer has been instrumental (!) in bringing more plugin instruments into the mainstream scores, and may well be one who'd appreciate the potential (especially being fond of u-he instruments).
I'm sure that everything won't suddenly change, and indeed I doubt the landscape will change much over the next 12 months. However, everything has to start somewhere and CLAP is well designed and connected to be able to make a difference.
I also think that a realistic approach needs to be taken with the open source aspect. It opens more doors, rather than closing them - that's what's important.
The business side will need more work, but just bringing the ecosystem to more people with less barriers to entry, as well as looking to progress the status quo and allow for more innovation without being dependent on a single vendor, is a fantastic start.
I don't disagree with your current view of the world. I just believe that it's just as ripe for change as other tech industries - perhaps more. Blender is another example of how sentiment can change - it may not be an Industry Standard yet, but it's gaining a lot more use and respect, and even the industry at large is now funding research and development into it.
There is change happening. It just needs encouragement.
The proliferation of bespoke sample players has solely been for commercial reasons, though. For example, a company receives outside money, say Spitfire; the investors ask if the company owns all their own IP; a discussion ensues that the company's products require a per-license fee to be protected with the Kontakt Player realm; the investor raises their eyebrows. What follows is pain for the user as developers who've never built a sampler player download JUCE and see what they can do, all making the same mistakes, and taking versions to iron them out much to the annoyance of end users. Plus, there's really not that much innovation that comes out of this... And some of the seemingly innovative aspects of SINE were simply stolen at the behest of one of the celebrity names they have behind some of their products.
I would love to see more change in music and audio, but when the main applications in use are all over 30 years old, you have to wonder. Cubase started in 1989 on the Atari and became reborn as Cubase SX 1.0 in 2002, Logic was released by Emagic (after the separation with C-Lab) in 1990/91 and purchased by Apple in 2002, Pro Tools also came in 1991, based on the earlier Sound Tools products, and so on...
Also, it's depressing to think how much exciting research existed in this field in the 80s that still hasn't come to fruition. Look back at Stanford and CCRMA, the close ties to Lusasfilm Computer Division, NeXT, etc... Michael Hawley who is sadly no longer with us, even talked about creating a MusicDroid (like SoundDroid and EditDroid) for John Williams to be able to prototype orchestral scores.
Surely for something truly innovative to happen, we need bigger truly revolutionary thoughts along these lines, not just another plug-in format. That's kind of boring, to be honest. Surely with developers like Urs and others can think of something bigger; CLAP seems motivated by developers wanting to unshackle themselves from the hosts they ultimately need to support them, either via wrapper code or native support. It's hard to articulate why this changes the world for the better for an end user who just wants a cool new synth.
I totally agree with most of what you have said, and it would be crazy to expect the world to just change overnight because of a new plugin format.
However, this is another opportunity to allow change - to break the "chicken and egg" situation and unlock the potential for others to come along and ride on that change.
As with any change, it needs support and nurturing to grow and become something that will have a greater impact. It's a foundation to allow more innovation to sit upon without being constrained by licensing and legacy.
This discussion is an excellent point as to why things need to change. The situation as you have so eloquently summarised, is locked-in and controlled by a few big companies, with little space for innovation in the first place.
It may not be a big or rapid change, but it has the potential to make a difference. In the spirit of your sentence:
> I would love to see more change in music and audio
Encourage the change, and don't decide there's no point because of how it's previously been.
> And as for open source, listen to Paul Davis talk about Ardour... For some reason, open source music and audio software has just never become a thing.
Uh, what? When did I say that? That just isn't true.
Apologies, Paul. When I saw your 2017 keynote (https://www.youtube.com/watch?v=dk2AMwc4e2k) from the Linux Audio Conference, you seemed less enthusiastic than in other talks you've given. Maybe I was reading too much into the message you were trying to convey; as I say, apologies if that was the case, I have nothing but respect for your work.
Your very first sentence is why CLAP is going to succeed in the long term. It is why CLAP was developed. It it an open standard that gives developers what they need in a way that LV2 never will. For many devs, it will be their primary CI/CD target of choice. It is not unlikely that the plugin devs with huge legacy code bases, such as NI, will begin to realize that CLAP provides a path forward to VST3/4/? support that will insulate them from Steinberg and Apple's decisions in the long term and save the a lot of money, and keep their legacy code bases profitable.
Avid is already evaluating CLAP support. So is Presonus. Reaper is nearly there. So in my opinion it's only a matter of time before Pro Tools, the industry mastering standard, supports CLAP. Other DAWs will follow.
I think you are underestimating just how much of a problem that Steinberg has created for the industry as a whole with their draconian limitations on the VST development. Pulling VST2 out from under everyone has created huge issues for plugin developers (like u-he) who rely on VST2 as their primary development target. Developers hate VST3, and nobody trusts Steinberg. Nor should they. VST3 is a dumpster fire that people target only because they have no choice.
CLAP may or may not succeed as a plugin format that users will use. But its greatest chance of success will be as the primary development target for developers, with all other formats wrapped around CLAP. It's a dream to develop on with its simple ABI, and a dream to wrap to other formats. This means that, over time, a large number of developers will produce CLAP plugins just to end the nightmare of maintaining VST2/3/AU/AAX versions that are feature complete.
With Avid, Bitwig, and Presonus already on board, and Reaper soon to follow, and with developers like Fabfilter and Xfer announcing interest, I think you vastly underestimate its chance of long term success.
This is even before considering what CLAP gives you. MIDI 2.0 is coming. Rich per-note expression is presently poorly supported in all DAWs except for Bitwig, (but until now, only on its internal instruments). CLAP has it, out the door, and Bitwig now supports it with its per-note modulation system. Other DAWs want this feature, but don't want to roll their own version of it. But the entire equation changes when you have major plugin developers already supporting it via CLAP.
My gentleman's bet is that CLAP will gain momentum, and the rest of the DAW world, save for Apple and Steinberg, will support it.
> It it an open standard that gives developers what they need in a way that LV2 never will.
This is false. The biggest issue with LV2 from the perspective of the people most responsible for CLAP was the governance model and/or what is required to shape an already decade-old open source API/library project. LV2 is 100% capable of doing anything CLAP can do, but the most significant CLAP players didn't want to go through the process that would have been required to make that happen.
Also: per-note expression is not a problem caused by plugin APIs. It requires a completely different data model for musical performance (whether MIDI is involved or not) than has traditionally been the case. Adding it to a plugin API possibly makes it just a tiny bit easier, but compared to the challenge of providing the user with a way to edit this, that's a nothing burger. If you want a more technical analogy: any DAW can record and playback MPE already, because it's nothing more than MIDI data. But allowing the user to control the evolution of CC43 and CC57 for the 85th note ... that's a totally different ball of wax.
LV2 has multiple issues, and its technical capabilities don't even enter the picture. First, it has a very complex API, and many developers who attempted it found that it was not worth the effort. Too little gain for little to no return. Second, the maintainers do not listen to the needs of the community. This is not an issue with the CLAP developers. This is an issue with the entire community. CLAP exists specifically because of the problems in the LV2 community, and with the LV2 architecture. This is why LV2 has not gone in any significant way beyond open source projects.
You need to ask yourself why it is that CLAP immediately has industry support upon teaching ABI stability, while LV2, so many years later, still has almost none. It's not random chance. It is because it meets a need that LV2 does not for both technical and governmental reasons.
As for per-note expression, you again made my point for me. It is indeed a completely different ball of wax that no current plugin standard can address in a way that is consistently addressable by a DAW. Sure, any plugin can implement MPE, but the implementation is plugin-specific, with no practical way to have a consistent DAW interface. There needs to be an API for it, and no API for it exists in the world of VST/AU. But it does exist with CLAP. This is yet another areas where Steinberg shot themselves in the foot. By killing MIDI in VST3, they also killed MIDI 2.0 per-note expression.
I know many LV2 developers, and I have never seen a coherent argument that it is "very complex". I do know that people without CS backgrounds find it intimidating because it tends to use more CS-y concepts than APIs like VST (and now CLAP). In addition, to be honest, 75% of all the criticisms I've seen of LV2 come from people who seem unable to understand some of these basic concepts.
> CLAP exists specifically because of the problems in the LV2 community, and with the LV2 architecture.
This is false, or at best misleading. CLAP exists because when abique and Urs made a minimal level to engage with the people involved with LV2, they didn't like what they saw, and quickly disengaged with it. Neither of them were or are willing to deal with the relatively normal stuff that goes on in open source API/library development, and have explicitly said that they would rather create their own API than deal with what they perceived would be involved.
The supposed "problems" with the LV2 architecture are all resolvable, or would be by people who (a) had a stake in them (b) were willing to participate in the process that an open source API/library project tends to involve in 2022. The people behind CLAP match (a) but not (b).
> Sure, any plugin can implement MPE, but the implementation is plugin-specific,
This is false, or meaningless. It's like MIDI itself. What does CC33 do with a particular synth? Could be this. Could be that. MPE is just the same, it differs in no way from vanilla MIDI 1.0. The same issue is true for CLAP, though CLAP does at least acknowledge that it's an issue. Any given plugin that implements "polyphonic expression" may have wildly different parameters and responses to controls than any other. CLAP does not address this (and cannot).
You also miss my point entirely. For hosts, the hard part of MPE/PE/MIDI 2.0 is providing a way for the user to view and edit a time-series of events connected to a particular note. This is so totally different from what happens with regular MIDI 1.0, where other than polyphonic aftertouch, all controls are per-channel. There's absolutely nothing a plugin API (CLAP or anything else) can do to assist with this.
> By killing MIDI in VST3
There are lot of audio software developers who are painfully aware of the horrible limitations of MIDI it and would prefer to avoid it. This generally does not include the hundreds of plugin developers who have churned out VST2 after VST2, often doing cool things but generally unaware of the problems that MIDI creates as a representation of musical performance. Did Steinberg jump the gun on believing they could create a new standard representation/protocol for this? Why yes, yes I think they did (after all, academia has been trying to do this for decades, without much success). But there's nothing really about MIDI 1.0 to be salvaged, except that it was simple and a lot of people think they already know it. In theory, what Steinberg did was to create a much more flexible system (than even MIDI 2.0). In practice, a bunch of developers didn't want that because they have existing code that just works, and another bunch didn't want that because they think Steinberg didn't get it quite (or even close to) right.
Most are quite similar, which is why the vast majority of commercial plug-in developers gravitate to using something like JUCE (or, to a lesser extent, iPlug). Such a framework makes it easy to target a single API and output cross-platform versions for AAX/AUv2/AUv3/VST2/VST3/etc... I have no doubt that JUCE will support CLAP, and therefore make it easier for developers to spit out yet another format.
Alternatively, CLAP could end up as the intermediate format, with bindings supplied for presentation within other native plug-in formats.
VST3 is the one that sticks out like a sore thumb, because it represented a radical change in how the user interface classes interact with the audio processor class. This created a massive problem because you had to assume that the UI and the processor were essentially running in separate processes and communicated via message passing through a specific interface or just plain text. It's the reason why VST3 didn't ever gain the traction and support of VST2, since Cubase was for many years the only VST3 host, and since more complex plug-ins like, say, Kontakt, would require a huge amount of developer time to upgrade, a stalemate existed for too long. So it's hard to see everyone getting behind yet another format to support and test, especially if the core code has to conform to the lowest common denominator, as it were.
On closed vs. open, although AU is controlled by Apple, and VST by Steinberg, they are both open to the point that anyone can download required SDKs and develop for them. Avid traditionally kept their SDKs through application forms and NDAs, so not everyone had access. Even now, IIRC, JUCE doesn't come with the AAX bindings necessary to build such a plug-in, they have to be copied in from the official distribution.
> VST3 is the one that sticks out like a sore thumb, because it represented a radical change in how the user interface classes interact with the audio processor class. This created a massive problem because you had to assume that the UI and the processor were essentially running in separate processes and communicated via message passing through a specific interface or just plain text.
Adding VST3 support to Ardour/Mixbus, we did not experience this as a notable issue at all.
Far more problematic from a hosting perspective in VST3, though it has also been present in AU since the start, is multi-bus output. At the "processing audio" level, it's still pretty simple, but "allow the user to connect stuff" level, its far from simple, unless you just do what reaper has done and flatten all the busses.
VST2 was not "open" in a way that made it usable by libre software developers. AU doesn't really matter much since it's single platform.
I think it is also important to stress again that most plugin developers that do this stuff for anything close to a living have not targetted specific plugin APIs for years. As you noted, many will use a framework like JUCE, others have their own in-house equivalent, which lets them create the DSP and GUIs in a plugin-API agnostic way.
> Adding VST3 support to Ardour/Mixbus, we did not experience this as a notable issue at all.
Not as a host; but this was a major stumbling block for more advanced plug-ins like, say, Kontakt. Given that VST3 was based on VST-MA (largely the work of one of the original Studio One authors), which is derived from Steinberg's COM-like interpretation, there were some genuine pain points for plug-in developers.
But I think your point about the framework approach is a reason against having yet another API to spit out. As you say, developers who do make a living out of this, aren't going to be able to dig their heels in and say "it's CLAP or nothing if you want to run my wares." I imagine it would be suicide for smaller developers like Fabfilter or Valhalla. And if CLAP is just another API supported in branch of a framework, where is the "killer application/plug-in" going to come from?
I think VST3 shows how well this is going to be received. Nobody was keen on VST3 from the start, and AFAIK this wasn't to do with philosophical objections to commercial licensing. It was largely due to the fact VST2 worked well enough, was understood, supported, and nobody saw the point in adopting it. The fact Steinberg had to deprecate 2.4 and then announce they were going to end-of-life support for it in their own applications to convince market forces says it all. Even when Doric came out a few years ago, in v1.0 only VST3 was supported. But they quickly backtracked and added whitelist support for VST2 plug-ins.
> there's a reason why music and audio applications and plug-ins remain the most heavily copy-protected software.
This is sadly true. I used to make a bit of music and I knew several people who would simply hoard pirated plugins and sample libraries, even if they'd never use them. I think it's an extension of "gear acquisition syndrome".
When even Kanye has been caught pirating plugins, what hope do vendors have.
Are you the same kibibu who made a bunch of Buzz plugins? Because I used them. Somewhat on topic, the Buzz audio plugin format is some single-header sweetness.
Disagree. Musicians like their platforms, but they also go where the good sounds are. There will be third party wrappers for CLAP plugins that will allow them to work in Logic or Cubase, and if high quality CLAP plugins are available (highly likely given U-he's stellar DSP record) then people are going to want to use them.
> ...if high quality CLAP plugins are available (highly likely given U-he's stellar DSP record) then people are going to want to use them.
And with Arturia onboard as well, who besides their V Collection are now making officially licensed Korg virtual instruments. (This week's Arturia user survey questions suggest that more official Arturia Korg collaborations are on the way.)
And of course there's TAL & Valhalla as well, who both make fantastic stuff, but Arturia is a much larger company.
If they could convince iZotope, Native Instruments, Waves & IK Multimedia to migrate to CLAP, I think that would cover everything else I need.
Yes for some reasons the audio world doesn't speak much about figures: cost of development, revenue etc.
There is a huge market of let's say $25-$50 plugins that serve the amateurs just as well the pros and the former have been pushed out of ProTools for a while now, so I'm not so sure AAXs beat VSTs nowadays, but hard to argue volume or revenue without the data.
I and some other folks in the Rust audio community have put together some low-level bindings for the CLAP API: https://github.com/glowcoil/clap-sys
They're relatively straightforward due to the fact that CLAP is a simple, pure-C ABI, and there are already some fully functional plugins making use of them (e.g. https://github.com/robbert-vdh/nih-plug).
Reaper and FabFilter getting on board would definitely help make this standard gain some momentum.
The list of “Following companies and projects are already evaluating CLAP for their host and plug-in software” is pretty impressive and personally accounts for a large amount of my workflow. Hopefully it gets big enough for Native Instruments to implement, Kontakt could benefit significantly from some of the new features.
I made that 2nd comment. I don’t use Bitwig. But I’ve been an Ableton user for 15 years and have been intrigued by Bitwig since they started. I’m really drawn to their design philosophy and the programmable interfaces.
Ultimately, the format will live or die by the ease of porting, which probably hinges on JUCE’s decision to adopt it or not. Having followed the rise and fall of a proprietary plugin format (Reason Rack Extension), both Reason’s storied place in the industry and the format’s technical merits failed to convince a critical mass of developers simply because the many SDK restrictions made porting too costly a barrier to entry for what was already a much smaller market.
If CLAP can make porting straightforward while gaining sufficient traction among DAWs, I could see it succeeding.
The article states that there's contributions from other vendors as well, I suspect that this may actually catch.
I've not really followed the industry, but my understanding is the controlling interests behind VST are unpleasant to work with for everyone involved - so this is a chance to build something better and in the open.
I'm interested to hear your take on this, as I suspect you're one of the few of us on HN actually qualified to have one.
I'm also curious (if you're comfortable answering), were you involved with "design and implementation contributions by a group of commercial and open source audio developers from across our industry."?
I don't think Ableton is that petty. They'll back this if they think they stand to gain something from it. And given the mess that the other plugin standards are I think the value proposition is clear.
It's not just a new standard. CLAP plugins work better than their VST counterpart. They do more.
Polyphonic CC modulation is a very fundamental upgrade Live could really have that would still fit its modular strip UI paradigm and justify even more its Push hardware. It would even reduce the gap with Bitwig.
Project portability is also extremely desirable for Ableton, who is looking at mobile as a new platform.
All the actors are Berlin-based companies located about 30min from each others, knowing each other very well. There is hope.
Also developed by u-he thats one big player in plugin synth world already. If CLAP solves issues plugin devs have and maybe makes plugins better than customers will demand support.
I'm not big in the audio community but I thought LV2 was the open standard for plugins that people are pushing for? That being said if Cockos adopts it into Reaper I'm absolutely willing to try it.
It also never got enough support from big guys. Clap looks like it fixes a lot od LV2 stuff as well, and it can replace VST, which is good because VST is a crap technology.
First is the licensing. VST2 is hard deprecated and not possible for new businesses or developers to target it (* it is possible via some reverse engineered header files but they are GPL licensed, which turns people away). VST3 has an awful license.
Without getting into the weeds too far, VST2 is pretty easy to target without a bunch of work. VST3 is pretty much impossible without some kind of heavy wrapping technology (even the bespoke distribution doesn't use the API directly, it wraps it in a heavy and confusing SDK - most people use JUCE, IPlug, DPlug, or the other wrappers anyway).
Also VST3 doesn't really support MIDI (1 or 2). They pretend to, by making the host do the work to translate MIDI into VST3-specific API calls (which are tricky to keep sound, makes it a pain in the ass for host and plugin authors, and is generally a terrible design) - search for JUCE authors trying to understand why their MIDI plugin isn't working in VST3 (it's because VST3 can't do that, and Steinberg won't accept that plugin authors and users want to be able to do it).
Lastly, VST3's i/o configuration for multichannel audio is a nightmare to try to understand and arbitrarily limited in a way that can't be expanded easily. It added a ton of complexity for no benefit over VST2.
The fact that its taken nearly 15 years and legal action to get anyone to support VST3 for their plugins is ultimately the biggest indictment against it.
It is licensed under two licenses. One of the is the GPL v3 or later. Evidently you don't develop libre software, and additionally dislike the other Steinberg license.
VST3 for all its issues was the first version of VST usable by the libre software community - VST2 was not legally re-distributable even though it was available gratis. This prevented legal development of any libre VST2 plugins, which was not a good situation for anyone.
I have multiple projects licensed under GPLv3. The biggest request I have gotten has been to relicense them so closed source projects could use them - which I would be fine with, but I'm not actually free to do that because of upstream GPL dependencies.
There are other legal issues with Steinberg and the licenses, particularly the recent changes (some of which are contradictory and the language needs adjustment). But if you're not a developer of a well known project, Steinberg will ghost you. The devs are kind, but unhelpful, and legal is unreachable.
And ultimately, users don't care about libre software. They care about software that works. That's why VST2 was so popular despite the licensing situation, and why DAWs that supported it are DAWs that people wanted to use.
correction, SOME users don't care about libre software. I know I didn't for a long time. As my concern grows about the unsustainable consumption we take for granted, and its deep environmental and social impact, libre software has been getting more and more important in my life. A ten year old desktop PC can meet all my computing needs - but it seems that's only true when it's running software that was developed at least partially separated from profit motives.
I would go further than this, I'm interested in audio programming and decided to give it a shot using the Steinberg's SDK after reading the VST3 docs. The way the SDK is organized put me off VST3 entirely.
Sort of. The breaking of MIDI is not, by any means, a good API. In a few other aspects it seems needlessly complex - feels like it was designed to serve one company's strategies and interests, not that of an industry.
Note that unlike pretty much every other c/c++ plugin API, the plugin code does not need to include any header, everything is done through reflection of struct members at compile-time. This means that it's dead easy to port plugins to e.g. microcontrollers.
since it binds directly to audio APIs at compile time, it has pretty much zero code size in itself, the smallest plugin it generates for VST2 is around 7kb IIRC
As a professional freelance mixer I must say if the programming behind CLAP is anywhere close to the quality they achieved in Bitwig, I am sold on the idea.
Bitwig does up to audio rate modulations of nearly every parameter, the stock plugins are all really top notch, the way they implemented voice stacking or polyphonic pitchbends for MPE synths etc. are all done in the way I dreamt it should have been done.
It still lacks a bit on the hardcore audio editing and mixing front (e.g. in ProTools you can edit extremely fast, just using the keyboard).
Their pricing and licensing model is fair. The price is actually quite okay given what you get in addition to just the DAW (a really good sample library with all the famous drum machines, a reaktor-like modular stock-plugin for synth, effects and midi manipulation, a powerful live-performance tool, etc.)
Is a MIT license a good fit for an open standard? What prevents any big nasty player adopting and sneakily subverting it with additions/changes in a truly EEE way?
> As we proceed beyond the initial set of extensions, we are committed to establishing a transparent process to govern the standard which allows participation from the entire audio software community. We welcome participation from the development community today, and we will share details of these processes and governing models over the second half of 2022.
Very vague and doesn't really instil confidence: "Join us now, and months later we'll share how we're going to protect your work."
Pro audio plugins have lower latency and harder constraints than graphics processing. You cannot drop a frame, and roundtrip latency is ideally below 5ms. GPU Audio (the company) has a lot of hype but it's not really clear if they gain a ton. The existing analogs to GPUs are expensive pieces of special purpose DSPs like Waves Grid, Universal Audio's UAD, and the biggest which are Avid PT HDX accelerators.
It turns out modern CPUs are plenty fast for heavy audio DSP. The point of friction is synchronization and the memory wall, most of the time.
It turns out that audio DSP algorithms don't generally benefit from the same kind of parallelism you get on a GPU, since even the basic stuff you'd like to do (filtering, algorithmic reverb, etc) is a bunch of recursive/sequential algorithms that have a non-trivial cost to convert to a parallel form for SIMD evaluation, and the gains are not linear due to repeated computation (classic example, converting an exponential moving average which is used everywhere to N lanes of SIMD is a lot less than an N times speed up).
That's not to say there isn't exciting work being done (GPU audio, for example). It's just really hard and not a magic bullet. It's not an Audio Processing Unit, after all. It also has video output, but not high fidelity audio i/o.
GPUs have some use for physical modeling (e.g. convolutional reverb — which at least 5 years ago was not practical to do in realtime on a CPU). i toyed around with such a plugin a while back but i think it takes more expertise to program any reverb to sound obviously superior to the CPU-optimized algorithms we’re more practically familiar with.
also additive synthesis greatly benefited from GPU acceleration at one point (particularly if you’re making full use of panning or detuning individual harmonics), but with SIMD/CPU improvements it’s much less relevant now.
latency as a barrier to GPU use is sometimes overstated. you can reach sub-ms latency overhead on a GPU roundtrip. maybe relevant if you’re really targeting 5 ms through the whole pipeline, but not for amateur audio. of course, one issue is that these latencies stack if your host architecture uses something like VST/LV2 which only knows how to pass frames via the CPU. it’s one reason why i’d love for some declarative approach to plugins instead of shipping binaries which assume so much about their environment (even just shipping LLVM IR could let you batch GPU calls). but it seems that approach may forever be niche.
Convolution Reverb was definitely practical on CPUs for much longer than 10 years, and a lot of people were using it, even on underpowered computers (I was using a base model Macbook Air myself since 2011, and yes, LOTS of long reverbs). Physical modelled reverb has been possible on consumer CPUs since the 90s of course (convolution reverb isn't in the category of "physical modelling" btw). If you're talking about physical modelling synthesis, it uses lots of recursion, so it's less interesting for GPUs.
I'll also dispute the claim that low latency isn't necessary for amateur audio. Recording with low latency is necessary for everyone performing on an instrument. One can achieve low latency with external DSP or external sound processors, but most amateurs are only relying on plugins instead of more expensive hardware, and that's where their monitoring comes from. People programming instruments rather than playing (and not exclusively using samplers, which are I/O bound rather than CPU bound) still need real time processing for other tasks. Try mixing with Bluetooth: it's hilarious how much latency can slow you down even when you're not performing.
Some things are embarrassingly parallel, like additive synthesis. Some things can be made quite parallel like convolution. But a lot of stuff is not.
The distinction I think that needs to be made between GPUs for graphics vs audio is that all GPU algorithms are built on the assumption that there exists a reasonable compute pipeline that every kind of processing can use or exploit. That is not true for audio processing (your compute kernels for convolution and IIR filtering are completely different, for example, even static IIRs and IIRs that undergo real time modulation have very different topologies).
The thing I don't yet know about GPU Audio (the company) is how they deal with routing and process chains. Sub millisecond latency is fine, but if you have two dozen plugins in a chain (or god help you, a real DAG) then how do they handle the delay compensated I/O without adding extra latency for each sequential edge in the graph without the host being aware?
It's not big things that hurt latency, it's more a death by a thousand cuts in practice.
How practical is a "split reverb" with the low-latency parts computed on CPU and the high-latency parts computed on GPU (like how partitioned convolution is already performed)? Or is reverb/convolution not processor-intensive enough to reduce CPU workloads significantly?
Well, in principle, you can operate with single-digit-microseconds latency via CUDA at least, it just involves breaking free of the normal kernel grid approach and using dedicated pre-spawned threads that run as persistent hardware threads on the GPU cores.
It's just that Windows resets the GPU driver if anything "hangs" for more than 5-10 seconds (like a persistent audio processing thread), and GPUs aren't always good at running multiple applications at the same time concurrently, so it may prevent graphics from happening while that audio thread is running (easily solved by not using that GPU to run any displays).
Edit: if anything, you can probably run harder real time code on a normal Nvidia GPU than on an Intel/AMD CPU, all of recent generation(s). Reason being that GPUs don't run management work on the application cores and have manually managed local scratchpad memory with bounded latency, but x86 CPUs have a system management mode for firmware/"BIOS" hooks that can interrupt the kernel at any time.
I don't think there's a meaningful difference between high and low latency segments of reverb algorithms except for some exotic hybrid models that use a mix of convolution for early reflections and FDNs for the longer tail, but the synchronization would absolutely suck to write and test
Naive convolution takes time proportional to the number of input samples multiplied by the length of the reverb impulse response (kernel), which becomes rather impractical for seconds-long reverb tails with >100k samples, convolved by ~50k samples per second. The usual solution is overlap-{add/discard} block convolution, where computational power decreases as the amount of input data processed at once increases (to a degree). But if you want short latency, you must process input data in small chunks, which increases CPU load. I think the trick of block convolution is to convolve the beginning of the reverb kernel in small chunks, and the later portions in large chunks (buffering small chunks of input into large chunks, convolving large chunks once available, and playing the result back in small chunks), and adding the two reverb tails (though I haven't fully studied or tried implementing it myself, and IIRC it's patented anyway).
Wow. A low level, technical, detailed post about audio processing on HN that leaves me without feeling the need to correct any of it. Congratulations, and thanks!
GPU Audio is doing it with VST 3. And it's probably possible with CLAP too?
Processing audio on a GPU is a lower level issue than a plugin standard which describes the interface between the DAW and the plugin (how audio, parameters, notes, modulation data, etc. are exchanged).
Honestly, it is an unfortunate naming choice ... no doubt it's going to lead to all kinds of double-take usages in surrounding lit. I certainly read the title to the post twice.
While not perfect, I really liked Will Pirkle’s book[1]. It describes the popular ABIs and has great intro to DSP. I would say the downsides are that the description of various interfaces takes up too many pages and that the book uses his own framework ASPiK for cross-compatability, but it’s easy enough to convert to a more popular framework like JUCE if needed.
Not the original person that asked, but my questions are more elemental, google-level maybe (I'm googling it now): What even are audio plugins? How people use it? What are some examples?
Caution: This explanation might not use 100% accurate terminology, but should get the point across.
Most modern music is made via software named DAW (Digital Audio Workstation), which is kind of like an IDE but where music is made. These allow you to play with audio (ex: import audio files, arrange them across a timeline, record sound from external sources like a microphone). This is great if you already have all the audio files you need (eg: you're a live band and recorded a song in a studio), but sometimes you want to create new audio without having access to a recording studio.
Audio plugins allow you to create new sounds. You've got two main types of plugins: ones that generate sound (eg: instruments, like a synth) and ones that modify sounds (takes an input sound, modifies it and then outputs it again, like an equalizer/EQ).
Audio plugins are external (or bundled) software that you can use inside a DAW, you can download a lot of them for free but they can also get quite expensive (eg: thousands of dollars). The most popular format for these plugins for a long time (and still today) was VST, but Clap aims to be a new standard to compete with VST.
From a user point of view, kind of like effects pedals[0], but inside your computer, and not necessarily guitar-oriented.
From the computer's point of view, a host program (probably a DAW) feeds an audio stream to a plugin, eg. 512 samples at a time, the plugin does whatever and sends back 512 samples, which the DAW may optionally route to another plugin and so on. Plugins are routed by software, instead of the 1/4" jack cables used by most hardware effects pedals.
Modern audio production often uses a lot of these effects / plugins, like dozens or hundreds across a multi-track project, then a bunch more once the tracks are mixed together. Each plugin can either affect the sound very subtly, or make it completely unrecognizable, or something in the middle.
Contains some omissions and half-truths, but that's kind of the gist of it.
Hopefully some people create a new "Atmos" style audio standard (eg multiple height surround sound), but OSS.
It's a pita with the current situation, where everything past ~7.1 audio channels is proprietary and needs extremely expensive hardware (eg decoders) just to direct the audio to the correct speakers. :(
Ambisonics predates Atmos by many decades, is license free and has both gratis and libre toolchains available. It can handle any number of speakers, in any configuration.
Ambisonics will never be widely adopted. Wide adoption occurs when a significant corporation in consumer technology decides to push something, and in general that occurs when they are in a position to license it.
Ambisonics has always been freely available, there is no possible license structure for it, ergo no consumer tech company will ever push it.
There are also some important technological differences between Atmos (and similar schemes) and Ambisonics - the former does better with specific speaker configurations, the latter does better with (most) arbitrary ones.
A "pure C ABI" doesn't really mean much, last I checked, seeing as how just about every combination of OS and CPU has its own quirks re: calling conventions and struct packing and such. Ain't sure what's meant here.
Old standards suck. VST is owned by Steinberg, and has caused all sorts of trouble in the quest for the "perfect" plugin format. Apple had a couple different solutions to this, but all of them were obviously proprietary to their ecosystem (and only a sidegrade to being shacked to Steinberg). It would be nice to see "one standard to rule them all" for Windows, MacOS and Linux, and u-he has enough weight behind their name to influence a change.
There's a lot of hope behind this, and unless Apple decides to open-source their plugin format, there's nobody to compete. VSTs are so bad that most DAWs would adopt an alternative with zero additional features, simply because Steinberg's positioning is a pain for the community. I reckon this little format has a chance, but it remains to be seen whether it's successful or not.
Since CLAP was announced last year, I've seen lot of online interest in CLAP from electronic musicians, plus independent audio developers who are sick of dealing with bureaucratic BS around Apple's AU Avid's AAX, and Steinberg's VST plugin standards. I would put Urs Heckman from u-he at the tip-top of not just audio DSP developers, but small business owners who truly care about advancing the state of the art. If u-he and Bitwig can get other Berlin-based audio shops on board this has a good chance of succeeding.
Looking at Ableton 11's feature set it seems they're running out of ideas; CLAP support would definitely be the kinda thing that could compel people to upgrade to a future version.
It being MIT licensed also gives it a much better chance of getting integrated into all sorts of smaller DAWs and cheaper ones too that might not have wanted to deal with all the nonsense that comes with VST.
>Looking at Ableton 11's feature set it seems they're running out of ideas
Ableton adds pragmatic stuff. They have tons they could add without running out of ideas for those for 50 years. A new format is not one of those (and will have small workflow impact on users anyway).
Examples: ARA support (somebody already mentioned), melodyne/flex/style vocal correction, soft track re-arragement (versions), and 50 other things besides...
Ableton is just slow at developing Live. I wonder what are the reasons behind this, perhaps they want to make sure they don't stray away from the core vision (which is what made it popular) or perhaps it's stability. Or perhaps everyone is just too busy working on their own music.
They can't even add ARA2 when every other major DAW has had it for years. Something is deeply wrong with Ableton, either they have trash product managers, SWE, or the 20+ year old codebase is too fragile. For example, why can't I have more than 7 collection folders?
Live 11 is becoming more of a cpu hog to my iMac Pro with each M1 update. It crashes the most often of all my DAW software. That said while I could go back to Logic or Cubase I still get 10x more music done with live vs the others. I remember about 10-12 years ago live version 7 or 8 was way behind schedule so much so that the CEO/Owner made a statement about it. This is all from memory though so some details may be wrong. I say that to say that is suspect that the code base is probably more fragile than the others.
I've been using Live for 15 years. The workflow is better in some ways—or at least it's extremely familiar—but I recently forced myself to learn Reaper. My mind is painstakingly slow to adjust, but Reaper's features make Ableton feel outrageously archaic. If I'm being honest with myself, I'll need another year or two to match my productivity with Live, but the bandaid had to be ripped off at some point, and I'm glad I did.
Note that Bitwig does not have ARA2 support either. Bitwig was developed by some folks who left Ableton to push in directions that Ableton did not want to go.
So ... consider the possibility that the fundamental data models in both Live and Bitwig make it hard to support ARA2, in an inverse of the way that Live (and later Bitwig) had clip launching for 16 years before any other DAW.
I've invested a lot of time in VST3 lately and honestly, it's got problems.
The actual API isn't terrible, if you can find it under the huge pile of abstractions and classes and toolkits built overtop of it and shipped in the SDK. Steinberg's SDK has too many normative lifestyle assumptions. From VSTGUI to their own COM-like component model to the huge mile high pile of macros they've thrown onto CMake, it's just ... pushy, its own universe. Doesn't play well with others.
I built for it because I wanted what I was working on to be workable in the majority of DAWs. But as the thing I was working on went from "maybe a $$ project" to "I'll open source this", I'm not sure I want to invest anymore time in VST3.
> CLAP 1.0 is the result of a multi-year project initiated by u-he and Bitwig, with design and implementation contributions by a group of commercial and open source audio developers from across our industry.
Looks like it's only a matter of time until others pick it up.
I'll give them credit right off the bat for spelling "plug-in" right. It's not a "plugin."
Doh, but then there's this on the page: "the transparent client-server model..." Nope: It's "client/server."
But in non-nitpicking news: I hope this succeeds. OpenFX seems to be sticking around in the video realm (being supported in Resolve and Nuke and a couple of other applications), and I hope CLAP enjoys greater adoption.
From this project's existence I presume that VST has some burdensome encumbrances.
Clap won't succeed longterm because of many of the factors outlined in so many of these posts. Notably, Apple aren't going to support it in Logic because Audio Units work just fine, and AUv3 spans both iDevice and Mac markets. Steinberg almost certainly won't support it; they aren't even allowing VST 2.4 plug-ins to run in the Apple Silicon version of Cubase -- and this isn't for technical reasons. Ironically, Steinberg have been trying to push VST 3 since the release of Cubase v4.1 back in 2006, and it's still a giant pain 16 years later. If you look back at VST 1/2, part of the allure was the simplicity of the API; VST 3 tried to change the game too much towards MVC given that it looked at the time you might want to run the processing part of the plug-in on a different system than the user interface part.
Pro Tools doesn't get a mention in this thread, for the most part, but for better or worse it remains something of standard in audio post-production in the US. AAX isn't a bad re-imagining of what what RTAS/TDM-based plug-ins pre v10, and Avid aren't going to relinquish any control to third parties. Not least because the need for AAX (such as it is) resolves around support for the company's hardware control surfaces. EuCon was originally envisaged as an agnostic hardware controller protocol by Euphonix, but Avid purchased Euphonix, released the S6 and associated controllers and made EuControl into something of a tolerated illegitimate child.
Plug-in developers will have to follow the money, and I would to see the results of a survey correlating money spent on plug-ins based on the main host software used. My guess is that Pro Tools AAX on the Mac probably is the most lucrative, followed by Cubase and Logic. Depressing, but for all the moaning musicians tend to indulge in regarding fair compensation against any company they feel exploited by, there's a reason why music and audio applications and plug-ins remain the most heavily copy-protected software.
I don't know of anyone who's ever ditched their host application because of a plug-in format, and I highly doubt the CLAP is going to harm Apple, Steinberg, and Avid in the longterm.