Microsoft made one called Garnet, I wouldn't say its a fork though, its basically compatible with Redis and implemented mostly in C#. It supports the RESP wire protocol from Redis for ease of compatibility.
Garnet fascinates me. Their benchmarks even claim that it is better than Redis and also Dragonfly. Are there any papers or write ups explaining what makes Garnet fast? (I do know its based on FASTER)
The tl;dr is it's just a lockless hashmap attached to a TCP server with a log. Simple Get/Set operations are highly optimized, so with high batching they are able to efficiently fetch a lot of data efficiently. The architectures scales very well when you add threads and data access that is uniform.
It struggles a bit on certain types of workloads like hot keys, think heavy hitting a single sorted set. It's a cool architecture.
The difference is that Redis is single-threaded, the value proposition of Redis was/is a fast (in-memory) simple (single-threaded) server to serve things so much faster than a traditional DB. It was a perfect fit and became popular with the Ruby/Rails community,etc since those environments combined with traditional SQL servers are slogs compared to what a fast server like Redis could do.
As good as Redis is, modern computers with multiple cores potentially leaves a lot of performance on the table. Garnet seems to be a well designed multithread KV-store (though sadly the benchmark page doesn't list benchmark for more complex objects even if the simple cases looks good).
I feel like most of the obsession over Redis came from old school cgi-esque server side apps that couldn't easily maintain state themselves...
Nowadays your webapp is often a persistent, multi-threaded web server where you can cache whatever temporary state you want far more efficiently than reaching out to a KV store.
Using postgres and caching common results becomes a non-issue even at scale. The main area where a super fast KV store shines is when you need to share a very large number of keys, where many servers only need to access a random subset and isn't interested in maintaining the full set and getting updates on a bus, where the values often change and needs to be revalidated so caching is ineffective, and where persistence is not needed so a database doesn't matter and a lighter KV store is acceptable.
Nice. I've been using Memurai (https://www.memurai.com/) for development on Windows (native, no WSL or Docker - for reasons), but this looks much better.
EDIT: Weird that being a program from Microsoft (well, it's Microsoft Research, so that probably explains it?) it has no installer and doesn't run as a service on its own.
Conventional wisdom from 10 years ago on HN is that Microsoft Research just pays some top researchers (with commercially interesting, err, interests) to keep doing their thing. I wouldn't distrust anyone from there based on their employer. That is from someone who doesn't trust MSFT very far.
Microsoft has a habit of "fake" open source. Particularly on Github.
By "fake", I mean Microsoft largely treats their Open Source codebases like they are closed-source, in-house proprietary codebases that they happen to let members of the public look at.
They'll accept a PR once in a while, but mostly it appears Issues and PR's are used as a free alternative to UserVoice.
Every decision is made behind closed doors (probably a MS Teams call actually), and you'll notice how top-heavy their staff is on these projects (half a dozen employees with "Manager" in their title on any given repo).
Beggars can't be choosers, and I'm glad they are dipping their toes into the FOSS water... but I don't really get FOSS vibes from their projects on Github.
Open source does not mean you accept contributions from others. It means that the source code is available, and that people are free to take it and modify it themselves. You are using a definition which is... not exactly incorrect, because definitions are what they are, but certainly not what everyone else uses.
I'm fairly impressed Microsoft managed not to name their Redis competitor "Cache", just to pollute the keyspace like they do with so many of their other products.
I guess it will depend on your definition of "extinguish" and a few other things but:
gestures wildly at the MIT-licenced VSCode codebase
Yes, they are not rug pulling the VSCode source but by locking down the marketplace (and never giving a truly open source VSCode, what developers think of as "VSCode") they are in the processes of locking out forks.
Ah so this is like how Android being open source was (almost) always bullshit. You had to play all kinds of google dances to get the google apps on them.
Fair point! good example of an attempt to extinguish.
Like seriously - I haven't heard of them doing it in quite a long time. I know they were atrocious in the 90s and 00s, I'm not disputing that at all. Are there any examples of them continuing this behavior in the last decade though?
Further, a huge percentage of the people making those decisions back then are no longer with the company, different people are in charge, etc. The industry has changed it's relationship with open source - company board rooms aren't scared of the consequences of loading open source onto a server, the legality and liabilities have been hashed out, and MS isn't really even capable of pulling those fear levers anymore. MS itself has repositioned in the industry - their dreams of total computing dominance have been shattered: there's no chance of a windows derivative owning the server market any more, there's no money in browsers or consumer OSes (heck even MS's domination in gaming is showing cracks due the the efforts of valve). Point being - would it even make strategic sense for them to try to EEE anything anymore?
Note: I have almost no ties to MS. I haven't used an MS os or desktop software since before covid (in any capacity, even moving the mouse on a computer running windows). I don't use any of their SaaS products personally or professionally. There are integrations between the products I help build and azure, however those are not a major source of revenue for my employer and I do very little work that even touches that stuff. Point being - I'm pretty non-MS in my life and don't have any sort of loyalty or incentive to defend them. I do abhor their EEE actions back in the 90s and 00s when they were doing them, and those still make me angry... but that's not a reason to assume that different people at a company are going to act the same as the old-school ones.
Perhaps... when I look at an Open Source codebase, I expect there to not just be source you can read, but also a way to contribute and engage with the codebase creators, beyond just bug reporting (aka. Github Issues).
While it may technically be open source due to it's license and you can literally go look at the source - Microsoft is operating these codebases as-if they are proprietary. Every decision is made out of public - and I would not be surprised to learn they have their own internal tracker for the "real" backlog items. You frequently see command/comments by Microsoft employees which clearly are triggering workflows in their real backlog tracker, and near-zero discussion happens in Github by Microsoft employees, and when it does - it's clearly through a PR filter.
Like I said, beggars can't be choosers and it's better to have this than nothing - but I don't really think Microsoft has grasped the true concept of FOSS as-of yet.
> Is SQLite not open-source?
Not really in my opinion either. There's no way to contribute at all... best you can do it raise hell on the forums about a particular issue you want to see fixed. So while it's Open Source in the strict sense, it's not Open Source in the general sense.
Whether a project is developed by a community or by a single entity holding all keys is completely orthogonal to being Open Source/Free Software. There's nothing wrong in putting one kind of projects above the other, but you may want to revise "your opinion" if you want to stay communicative, because terms and definitions are only useful when people agree on what they mean.
The terminology for this is the cathedral model and the bazaar model. Under the former model, code is released from 'on high'. Under the latter, it's developed in the open and with cooperation with the community. Both count as Free and Open Source software though, provided of course that a FOSS licence is used.
I think most projects are genuinely like this though, even ones that accept outside contributions more earnestly. I understand why you'd associate open to contributions with open source but I think its a mistake to treat the relationship as required rather than common.
The sibling comments contain some sharper critique of Microsoft if you haven't read them yet.
I make an open source, MIT licensed piece of software. I don't accept unsolicited contributions, but I document that people are free to fork the code and provide instructions on how to develop, test and build on your machine.
In my opinion, the spirit of open source goes beyond just tossing code over a wall for people to look at. In my opinion, it means accepting engagement from your users, their inputs and their contributions when/where warranted.
In my opinion, for something to be truly open source, I should be able to fix a bug I ran into, or implement a feature from the backlog and contribute it back upstream. If upstream is just going to ignore my contribution, pretend it doesn't exist, or reject it just because - then that codebase is just pretending to be open source.
That's not to say you are required to accept all contributions - I'm saying you should be open to contributions that A) save you time B) enhance the codebase or C) fix confirmed bugs.
In Microsoft's case - I don't see a lot of that going on. I see lots of Issues (bug reports), some PR's, but mostly opaque decision making, and complete silence on things the Corporate side of Microsoft doesn't want to comment on yet. Which is the beef - it's a corporate project run like it's proprietary but you can go look at the code. Again, better than nothing, but it's not really what I consider true open source.
> In my opinion, the spirit of open source goes beyond just tossing code over a wall for people to look at. In my opinion, it means accepting engagement from your users, their inputs and their contributions when/where warranted.
I disagree. There's nothing about open source or the various open source licenses that require accepting engagement from the community and/or contributions.
Open source means allowing modifications, and sharing those modifications. It's in most licenses that the software is provided as is and without warranty.
> In my opinion, for something to be truly open source, I should be able to fix a bug I ran into, or implement a feature from the backlog and contribute it back upstream. If upstream is just going to ignore my contribution, pretend it doesn't exist, or reject it just because - then that codebase is just pretending to be open source.
A project not accepting outside contributions is still open source, not pretending to be, and the beauty of it is - if you want it to accept outside contributions, you are able to fork it and accept contributions on your own fork, or otherwise share your modifications. But there's absolutely no obligation of the original dev/owner to accept or engage with anything from the community. It's in the license, the software is provided as is and without warranty of any kind.
Maybe it's worthwhile to coin a new term, community software, to specifically make a distinction between projects that are community developed (accept contributions) vs those that don't.
I'd say the spirit of open source is that others are free to modify the code and that's it. This requires a good license, the possibility to fork, some documentation and a way to build the project yourself.
But why would accepting contributions be required?
A lot of people are very entitled. They think that an open-source project gives the right to make request/demand of the project. Even if they are willing to write the contribution themselves, they still think they have the right to have their pull request accepted. They forget that 1) it may be outside the scope of the project, and 2) the project owners are going to be the ones that have to actually maintain the code they commit. Crazy stuff.
What if you want implement a feature, but they don't have time to look at it and make sure it's secure, or support future bugs? Look at the xz (IIRC) hack - not everyone has tons of free time.
How long after they release their code are the required to keep this up? Do they need to respond to your requests within 5 business days?
If they retire / move on to another project, does the source code stop being open source?
I have the same feeling. I got used to infrastructure being run as a democracy, not merely “source available under GPL/BSD/MIT”. (It’s a big thing to want, sure, but I don’t mind wanting big things.)
IMO, no. The open source definition says nothing about requiring outside contributions, and IMO the spirit of it in the beginning was never about that.
It was about being able to have access to, fork/modify, and redistribute the software. That's the important thing - that the software I use can be modified by me, and if I wish to do so, the license allows me to then redistribute those changes vs. proprietary software, which cannot be modified nor redistributed.
I'm not sure why the zeitgeist went from the above and turned into "all open source must be community software and engage with users and accept contributions" because that was never what it was about. It was a benefit of some projects, sure, but at the end of the day the important thing isn't whether or not the project accepts contributions but whether I have the power and license to modify something to my liking, fork it if I wash, and/or redistribute my changes.
Having worked a few places that have major open source projects, it was eye opening to see how pervasive this seems to have become. It feels like over my lifetime I've watched the value/expectations/demands/definition of OSS shift from "source is available, and I can make changes to my own copy/fork of of it if required" to "source is available, and I will demand you make changes to it for me"
> source is available, and I will demand you make changes to it for me
There seems to be a disconnect here. I agree, demanding someone else make changes for you is in poor taste. The issue is, some of these projects advertise themselves as being open source while having no meaningful way to contribute changes back upstream - ie. the spirit of open source.
If I use your program for free, and encounter a bug, I should be able to fix it and contribute that back upstream so everyone benefits. That's my way of "paying back" and helping the community. Projects that reject contributions, or make contributing difficult are not in the spirit of open source, even if they are technically open source via their license.
That's what I call "fake" open source - you want the positive image of being Open Source only.
> There seems to be a disconnect here. I agree, demanding someone else make changes for you is in poor taste. The issue is, some of these projects advertise themselves as being open source while having no meaningful way to contribute changes back upstream - ie. the spirit of open source.
I mean, you're not demanding someone else make the changes. You're only demanding that they engage and build a community around their project and all that entails.
As long as the license is reasonable, anybody else could pick up the project and do the community building around it. Apache HTTP Server developed around community patches to NCSA httpd. I don't think hitch (from the people who brought you Varnish) is well know, but it started as stud with community patches. If something like this takes off, the originator will either start taking the community patches too, or not; it's their choice.
In the meantime, as a user, I get the benefit of whatever changes the origin provides and I can patch it as I want, and fix up problems that I see. That works for code dump open source and community open source,.
That's too vague. Free software is software you can't take and then modify and keep your modifications hidden or proprietary, thus depriving your users and the community of access to these changes.
MIT license allows this, hence it's not a free software license. Because very large corporations don't like freedom they've poisoned the discourse around software freedom and launched PR campaigns trying to substitute talk about freedom with talk about openness or open source. To some extent they've succeeded, as evidenced by the chatter about it in this thread.
As a end user, I find it quite restrictive. There might be some software I won't be able to fork/modify because they were MIT. I'm not the owner of the software anymore.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons..."
The MIT license always allows you to fork/modify the project. You wouldn't be the copyright owner of the existing code in your fork, but you could be the owner of the project, and the copyright owner of any new code you add.
https://github.com/microsoft/garnet