I think all you need to know about this CEO/CTO, Dgraph as a product, and the underlying key-value database Badger, is that when people report data loss bugs where the database fails to return data written to it, they'll claim it's not data loss if the data still happens to reside on the disk somewhere, and *then delete the bug report*.
> However, if the bug only causes Badger to not return data, and the data is fully recoverable by an easy fix to the code -- that's technically not a data loss (See Badger unable to return data after value log GC #578).
Do rockets sometimes blow up on the launch pad? Yes. Does that mean they shouldn't have tried to build a rocket ship?
Have you tried to build a data layer engine?
This kind of attitude is so one dimensional from my perspective... they are trying and it's hard to always be perfect when it comes to data. Inevitably if you're in the data game sometimes data loss happens, that's just the nature of the game. It's not black-and-white like you portray it.
The point is, when the rocket explodes on the launch pad you don't have the head of NASA saying "Well technically all the pieces of the rocket are still there, so technically we didn't lose it."
This is in the context of essentially a bug bounty for data loss issues that they were running… sounds like a NASA administrator saying something like "show us how our rocket will fail"
On the contrary, it sounds like a cover-up. Looking into the Github Issues, every-time someone does demonstrate data loss, the admins ignore it, argue it isn't data loss, or in one case, claim that the person who showed data loss is a disgruntled ex-employee (???)
Given that the former CEO is here right now accusing people of shilling while he scrubs unpleasant details from his past posts, I'd say this just appears to be a dead, toxic startup.
I feel for this person, but there's a line in here I found revealing:
> Unfortunately, Dgraph was always undervalued by investors.
This feels a bit like blaming the investors for why the company failed. To put it bluntly, there's a point at which you raise capital and build on vision. Your early investors and users are taking a giant leap that a. you can execute on the vision and b. customers will one day want to pay you for that vision.
Then there's point at which the rubber hits the road, and either customers pay you money for the thing you built or they don't. If they don't, you need a really convincing story to convince your investors they shouldn't cut their losses. It's a shame the company failed, it seems like they were working on some cool technology. But based on this post it sounds like no one wanted to pay for whatever it was they built.
Their last Series A [1] was led by Redpoint Ventures which means you would have Tomasz Tungus involved in this decision. Of all of the VCs I've ever seen, heard or read about he is the most metrics driven. He regularly benchmarks companies against each other and has a deep understanding of the health of each aspect of a business. I would highly doubt this decision was personal or politics driven in any way.
At a guess I would say that they did an analysis of the company and recommended it be shut down. And the other VCs i.e. the Australian trio of Airtree, Blackbird, Grok who are far less experienced in the VC game went along with it.
Since the CEO has quit, there’s a high chance someone with authority might delete the ex-CEO’s public posts because it paints the company in a very bad light.
As you may have read, the person who originally coded Dgraph is no longer with the company. Over the last few months it had become clear that this was inevitable. Only the final form it would take was unknown.
After resigning, he improperly used company confidential credentials to make an illegitimate post in the Announcements section of this site in which he made many statements, some accurate, some not. We are considering deleting that post to avoid misleading customers, but for transparency we will not do so for at least 7 more days to give you a chance to read or save it.
Terrible new name though. Outcaste manages to both combine the negative associations of being a reject and the Indian caste system, and permanently remind users of the project's troubled history.
I guess, both. Cockroaches are not so smart in the end, in fact, they often become victims of their powerful drives, which can be exploited for their own demise. I'm sure knowing the architect and behavior of the DB can be exploited in a similar way! I'm joking, of course.
We invested a fair amount of time into a Dgraph-based solution but had so many issues with it that we had to migrate away. We couldn't get basic things (like database dumping and/or reimport, or some large queries) to work at all reliably which scared us away from going with their enterprise product.
It seemed neat from the outset but I can definitely see why they might have problems finding paying customers after getting hands-on with it.
I worked for a company last year that chose Dgraph as its primary data store.
Thanks to Dgraph's query language, we were able to implement a GraphQL API server that had no resolvers and no need for dataloader—the n+1 problem simply couldn't occur! It was like magic.
Interestingly enough, we had been working with a salesperson from Dgraph Labs who was affected by the June layoff—along with the rest of the sales staff and CEO—right as we were about to pay for Dgraph Cloud! For a solid week we seriously debated switching to another database, but eventually decided to go with Dgraph Cloud anyway—telling ourselves we would self-host if Dgraph Labs folded.
I ended up leaving in November, but I think I know what my former team will be doing over the next few sprints!
I wish Manish and crew all the best, and hope they succeed in their new endeavor. Dgraph is pretty dang cool.
"They are now contemplating a few purchase offers. I have no control or say over this process. There’s a lot of politics. I’m a builder, not a politician and I see no point in staying on."
The investors poured ~$15m into this company, and the board is now trying to fulfill its fiduciary duty by trying to recoup some of the invested capital. I understand that open-sourcing the code base would be a better outcome for its customers. However, I am having a tough time seeing if the investors were truly political or antagonistic in any way.
A bit of a tangent, but board members (and company officers) do not in general have a specific duty to maximize shareholder profit. They have a duty of care (think carefully about decisions) and a duty of loyalty (act in the best interests of the company as a whole). Those could lead reasonably to a profit-maximizing decision but it isn't a straight line.
Author goes on to talk about how difficult it was to raise capital, and the company being constantly undervalued. Doubtful they're only referring to this singular event.
Open sourcing their code would be a terrible idea for the business unless they opted for a Elastic/MongoDB style license where you are prohibited from offering a cloud version of the code.
Because you know AWS would be circling this news very closely for another product to add to their lineup.
Building a database is such a hard undertaking. I have to give a shoutout to RethinkDB, which also had cool tech and also failed commercially. And then if you have success as open core, AWS will just fork and offer a hosted version of your software. Good luck continuing Dgraph as open source.
> They are now contemplating a few purchase offers. I have no control or say over this process. There’s a lot of politics. I’m a builder, not a politician and I see no point in staying on.
This statement seems a little naive. If you’ve raised money from investors, started a company, been it’s CEO… did you not realize there would be a lot of politics involved in this role? And more importantly, why would anyone want you to be CEO again?
At the end of the post they mention they’re going on to start a new company building on the same product. What would be different this time?
We recently migrated our db to DGraph and are blown away by how good it is. This is really a bummer. I feel for Manish, hopefully the next steps can work out.
It's easy to roast a CEO from the sidelines when things go south, but it's really hard to build such good tech.
Manish and his team have built wonderful pieces of software and have been nothing but amazingly helpful to me when I dabbled with Dgraph three years ago or so (early versions).
Feeling sad for them but I'm sure they'll come out stronger and current users will follow them.
No clue about the politics and $$$, but the product was already promising and different from the competitors (ArangoDB and Neo4j).
yea, that's my biggest fear right there (in terms of raising money). I doubt he walks away with anything after pouring his life into the company for years. I get that they raised $15 million, but to lose your code, company and time is horrible.
He ended up being powerless at his own company
Agreed. We all celebrate the big raises and never talk about the very significant potential downsides of those raises for both founders and employees when things don't go well.
Yeah, that's a bad look for both dgraph and the new company imo. What guarantees the same thing ran by the same people won't have the same issues again?
Not only that, if the core team moves to the open source product to "rebuild" the same features, isn't that way too dangerous IP-wise?
This is also one of the reasons I didn't use dgraph: Open source core doesn't work. All the companies that were sold to now have nowhere to go for support.
I would respectfully disagree that open core doesn’t work - companies like Confluent, Databricks, MongoDB have shown that you can succeed with open core. DGraph I think tried to do too much rather than focusing on graph databases, which caused the commercial product to fail.
I think an issue with dgraph is their "open core" didn't include a lot of things you'd expect in a "core" DB (iirc backups, multi tenancy & acl). I'm not familiar with confluent & co. but imo for "open core" to succeed you need a product attractive enough to gain market share, and then you can have extra features for commercial use. But then why not just make it completely open source to gain even more market share and offer support contracts (eg redhat & co)?
$employer had a need for some of the bits of Dgraph that are unfortunately hidden behind a paywall (bits that IMHO should have been in the community version as they were not particularly enterprisey, but hey such is life).
A frustrating conversation with Dgraph sales-droid ensued:
Us: We need access to X/Y/Z features, but on-prem
Droid: But Cloud
Us: Cloud = No-go because X/Y/Z
Droid: But Cloud
Us: Come on mate, I already explained why we cannot do cloud, plus you say on your website on-prem is an option
Droid: One meeelion dollars
Us: Goodbye
I’m not sure why you think this is their fault, or why you think the enterprisey-ness of a feature determines whether it should be behind a paywall. Clearly it was valuable enough for you to want it. People should pay for value. Especially if they want companies to stick around for the long haul to support the technology.
I think you missed the point. What I was making a point about was first the tin-eared nature of the salesdroid (i.e. "cloud cloud cloud cloud" despite me explaining why not). Second about the price point of on-prem. Of course I will "pay for value" but there is a difference between paying and paying through the nose.
Basically it gave the impression that Dgraph was so up itself that it thought it could dictate how their customers should run their IT and thought it was special enough to be able to ask silly money prices.
This topic came up in a group of early stage investors I'm associated with and I know I'm not alone when I think: they couldn't raise more than 15M? And lost control of the company??
Sure, I get that it's a database, and databases are harder sells than a lot of other things, but christ on a bike they must've fucked something up if they couldn't draw the allure of any of the big funds currently throwing gobs of money around.
I know a few founders at smaller companies than dgraph, where similarly engineers are their target audience, who have complete control of their company despite raising in the ball park of 100M over two or three rounds in the last couple of years.
There's just no excuse. Either their fundamentals were so fucked, or they were too arrogant even for VCs which is almost impossible.
When you raise money, you become responsible for the execution/growth to people who gave you money. If you don't like that, you should not raise the money.
VCs are holding onto companies much longer as, (a) the average time to reach IPO stage has gone from 4 to 11 years over the last two decades and (b) the amount of unspent money floating around is so high. There is definitely more to this story than just the VCs trying to exit.
I just refactored my whole app from MongoDB to DGraph... Guys I really like this idea I don't want to abandon it. What should I do: eat section 13 of Mongo's license, or try to maintain a database server myself?
I'm not sure what "eat section 13 of Mongo's license" means, but if you're asking whether you should self-host Mongo or pay Mongo Corp to host it, I would say the latter. It's not that expensive, they have a nice management UI, failover, performance hints, backups etc..
And I say this as someone who doesn't like Mongo DB and believes it's almost always the wrong technology for business to use. But if you're using it, might as well get the experts to host it for you.
> I'm not sure what "eat section 13 of Mongo's license" means
Buncha weeks ago I had a paranoid fit about the legalities of this app I wanna try to launch. I read some conserning things about the MongoDB license, mainly that the open source initiative didn't approve it bc Mongo pulled out. I read the controvertial paragraph and now I am 80% convinced that I'm not allowed to self-host a MongoDB server for my own profit.. I'd have to pay.. which why should I when SQL is free. BUT I DON'T WANNA USE SQL. And frankly MongoDB wasn't living up to the stresses I wanted from it but Dgraph does.
I was planning on self-hosting my Dgraph database anyways, and if this app actually ends up being something, I would beable to calculate milestones that I'd need to reach before considering upgrading to Dgraph Enterprise. Now I'm more worried about future support. Is Dgraph going to be sunsetted, and will I have the features I might need in the future. IDK I think I just freaked out about something I didn't understand, sry guys.
IANAL but from what I know about Mongos current license (Server Side Public License) the restrictions of section 13 only apply if you are making a service with the express interest in providing DB functionality directly to your customers/users. IE: If you are making a Database as a Service platform, similar to Mongo Atlas or AWS RDS.
If you are just using mongo as a backend DB to fulfill some other internal requirement in your application, like general CRUD operations, then you are fine.
Arite, now the tally is even. I've heard the same number of people say to me that it's safe to use Mongo to power an app so long as I don't expose it, as have told me that I can't because I have to opensource all of my code that interacts with Mongo.
Euxodus is right. As long as you’re not building a product where the primary value of it is just delivering MongoDB as a service, you’re fine. There’s an FAQ that addresses that and more here https://www.mongodb.com/licensing/server-side-public-license...
I suggest to differentiate between selling MongoDB as a service (which would break the SSPL) using MongoDB as a database in your solution where you bind your application to the MongoDB driver that are licensed under Apache 2 (and which is the reason the knock offs can use MongoDB drivers and state they are MongoDB whatever compliant and be not because their stuff has nothing to do with MongoDB server but I digress). Nothing prevents you from utilising MongoDB Community in your own project/solution as DB backend.
What does “eat section 13” mean? The SSPL part? Are you looking to offer a cloud-based managed MongoDB database service to the public? If not, you can do pretty much anything you want with the code. There has been a lot of FUD posted but really, taking the community code and offering a parasite service like Amazon did with other databases is the only thing prohibited by section 13. And unless you have a lot of energy and time you probably can’t be competitive with Atlas, their managed service, as it has a lot of stuff built around the database. Just my opinion, but I am curious about the “eat” part. Of course if you are just allergic to anything different than pure Apache 2.0 regardless of the fact that SSPL has no business impact on you, that’s completely ok
Or understand that MongoDB's license is kind of a necessary evil that I guarantee DGraph will likely implement at some point. Amazon's embrace and extinguish approach is too successful to ignore.
I've followed Dgraph off and on. I enjoy looking through their open sourced work. A lot of it is interesting.
Manish seems to be THAT person. The technically competent, unwavering to the technical vision, doesn't work well with others person at your company. He went on to found a startup so he could do what he wants, and unfortunately has now realized board members are very similar to managers at Google.
> I’m very confident in my abilities to bring in a solid VP team and lead the GTM motion. But, I’ve also realized that Dgraph is an intrinsically technical product. While I can find other S&M folks, finding a true engineering replacement for me has been, and will continue to be a real challenge.
> Massive layoffs with no warning and almost no severance pay. This is a sinking ship, not a rocket ship - avoid it!
Inept founder (was CEO and CTO while I was there) who only believes in his ability and doesn't trust employee expertise. He would rather be right than make money or see his company succeed. Manish doesn't listen to employees or managers and even disputes data showing false assumptions or failed business approaches. Micromanages in departments where he has minimal knowledge or understanding, leading to low morale and poor results.
Manish here. I don't know who you are or what your motivations are.
Glassdoor Reviews are anonymous (read trolls) and a lot of them were created post the layoffs, to create a story that would take the blame away from the hired CEO. Read between the lines about why the layoffs happened in the first place. If not, read this:
> I don't know who you are or what your motivations are.
I'm a rubbernecker passing by a car crash. I have no other motivations. I don't know you personally or anyone who works at Dgraph. I'm piecing together a guess of the culture from what's available externally.
I would love to read the details when you're ready to share.
Editing to say that the only reason I remembered the blog was because there were multiple parts of it that made me cringe when I initially read it:
> I can be a good CEO. I just can not be a good CEO, a good CTO, and the top engineer on the team for a highly technical product, all at the same time. If I could have found a true engineering replacement for me, I’d have stayed on as the CEO.
> And two, we lost a couple of deals last quarter due to engineering challenges I can help solve in weeks but would take my team months on their own. Some of these problems wouldn’t get solved without my help.
Publicly throwing your engineering team under the bus repeatedly isn't something you commonly see.
> And two, we lost a couple of deals last quarter due to engineering challenges I can help solve in weeks but would take my team months on their own. Some of these problems wouldn’t get solved without my help.
Those were not my words. Those were the literal words used by my Engineering manager out of desperation to convey to me that the team needs my help in solving these problems.
I'm not sure why you would take these as "throwing the team under the bus." This is actually expected -- given I wrote a lot of that code, and was the most senior engineer in the whole company.
And we had the best team we could build with the resources we had access to. 4 of my team members have created 3 companies -- one is YC, one has a wild acquisition offer, and one is Sequoia-backed. And they all seem to like me a lot.
As a lurker, I'd suggest you stop talking because you're not helping your case.
As a leader/CEO/CTO, people judge you based on your ability to lead the team, not your personal technical skills. Your posts are throwing away the former in an attempt to demonstrate the latter.
That sentence is being taken out of context. So, I clarified the context.
Moreover, the team in question -- I have spent countless hours working with them -- and they know exactly what I meant. No one ever complained about this blog post or this particular sentence. They knew exactly where I was coming from.
If Glassdoor is a benchmark for the effectiveness of leaders, then might as well vote for whatever Russian trolls tell you to vote for.
It sure seems like you used to work there since you're making specific critiques about company culture and decisions that only someone internal would know.
And you're taking statements like "lost a couple of deals last quarter due to engineering challenges" as being a personal slight against the engineering team when an impassioned observer would not really see it that way.
I don't know the specifics of Dgraph's failings but reading the Glassdoor reviews it sure looks like the main issue is that the CEO failed to organise a bridge round and that the disputes between the CEO and CTO and poor metrics meant the VCs didn't want to provide one.
You can fix any problem at a company except running out of money.
> And you're taking statements like "lost a couple of deals last quarter due to engineering challenges" as being a personal slight against the engineering team when an impassioned observer would not really see it that way.
I have no opinion on this broader company, but you focused on the wrong part of the sentence.
OP was focused on "I can help solve in weeks but would take my team months on their own" -- something that indeed is throwing your engineering team under the bus to me, an impassioned observer.
The CEO's statements come across as insulting to anyone who isn't blinded by founder worship. I'm saying this as someone who did not even know about dgraph before today.
Presuming the GP must be a disgruntled employee because he can read a Glassdoor review is just naive.
Glad you quoted statements from @mrjn / Manish's original post because it looks like he edited a bunch of those out of his post.
And yes it sounds not only like he's throwing his engineering team under the bus but also thinks they were incompetent which is beyond something any C* level person would be silly enough to write.
Deep familiarity with a complex code base is a thing and is not easily replicated. I have seen this first hand. In my day job - I work on a very popular but complex open source project and in my team there is a engineer who has been there since beginning and I am like second person after him.
I have noticed that when we get certain class of bugs from customers(and customers want fix NOW) that spans 2-3 layers of the project, only me and him can pick those bugs. Some decisions are there in the code that is not easy to figure out(like why it was done this way) and I have also seen engineers who are founders of this project - point design issues that I could not have thought (and I am doing this for 5 years!).
Can we hire someone new to provide design reviews that spans network, storage, kernel? May be - but engineers who created this thing and have been doing this for long time, have got really good at it. To me - some of them seem god like. I know they will probably suck same as me, if they had to start developing a new iOS app or write new game for Nintendo Switch.
I have to agree. I worked on a pretty large Java project, we hired 2-3 capable engineers, but they couldn't grasp the complexity of the problem, not necessarily the code - the code was pretty readable if you know what it does and how.
Brittle or hard-to-grok code that other engineers avoid is a code smell. Eng leadership/senior engineers ought to prioritize refactoring that code ASAP
There wasn't any implication that the code is "brittle" or hard to grok due to a defect in the software architecture. It may be approximately optimal from the perspective of software maintainability relative to functionality and performance. All code has non-local side effects, but in most software that is buried below the noise floor of inefficiency.
Some types of software are intrinsically difficult to understand e.g. due to highly visible behavioral coupling across multiple unrelated parts of otherwise tidy, modular code. This is a strictly correct and intentional design in many cases but it means even a seemingly small code change can only be evaluated for correctness relative to the detailed internals across diverse code components in various runtime contexts. Reasoning about these systems has a very high cognitive load.
This is one of the reasons writing competent database kernels is so difficult -- the best designs asymptotically converge on being exquisitely complex and subtle monoliths, even when the code appears clean and modular. In my experience, there are always parts of these code bases that only one or two engineers dare modify even though the code is straightforward, because any local change can't be evaluated for correctness without deep expertise in half the system internals.
> There wasn't any implication that the code is "brittle" or hard to grok due to a defect in the software architecture
Re-reading my comment, it comes off dismissive - I apologies for that. I agree with most of your points
> Some types of software are intrinsically difficult to understand e.g. due to highly visible behavioral coupling across multiple unrelated parts of otherwise tidy, modular code. This is a strictly correct and intentional design in many cases but it means even a seemingly small code change can only be evaluated for correctness relative to the detailed internals across diverse code components in various runtime contexts
Sometimes - the above is the case, but other engineers are unable to maintain those hard parts because the information required to do so is in the heads of the experts (the original author, or the 2nd engineer). In those cases, docs may help, but sometimes the code exhibiting unexpected coupling and/or side-effects may need to be further modularized, OR paradoxically, consolidated and abstracted by an easier-to-keep-in-mind abstraction as the "modularization" is illusive since the seemingly independent modules covertly belong together.
I may be projecting because of my past experiences: at different points I was the only one who could make changes to specific modules, one because it was brittle and had low test coverage (and prone to race conditions), and the other time, it was because there was non-obvious coupling which resulted in high cognitive load as one had to keep several layers of state in mind. On both occasions, refactoring and improving the abstractions would have made it easier for other engineers to safely work on the code.
YMMV if the high-cognitive-load architecture is deliberately chosen - in my case, it was an accident of (commit) history.
Yep this so much. It is not that code is brittle but problem space is hard. What seems like an obvious fix may break other vendors that depend on same behavior etc.
Having said all of that. Modesty is a thing and many engineers wouldn't word things the way Manish worded in aforementioned blog.
A lot of shit will be flung but the fact that the core team is on board with you to create Outcaste, a Dgraph fork, is a strong enough signal for many. Hopefully, you folks make it work!
Don't take it to heart that people will happily create a narrative just to get karma points.
Could you have better optimized your personal brand? Maybe. Could you have also done the engineering you've done if you had? Who knows. Don't over think it. State is hard
https://github.com/dgraph-io/badger/issues/578 titled "Badger unable to return data after value log GC" vs https://github.com/dgraph-io/badger/issues/570#issue-3621168...
> However, if the bug only causes Badger to not return data, and the data is fully recoverable by an easy fix to the code -- that's technically not a data loss (See Badger unable to return data after value log GC #578).
Never let this person store your data.