I lost a Talos II board due to what we are fairly sure was a shitty PSU on the first few powerups. I replaced the PSU with a store bought Corsair one and the new unit and replaced (under warranty) board has been very reliable.
Raptor is a small company and one guy is fronting most of the support load so keep trying, he's not out to screw any customers or provide an inferior product.
> he's not out to screw any customers or provide an inferior product.
Can he actually manage returns? His suppliers should be eating the cost of faulty boards, and the article makes me think they aren't. Continued problems customer side should result in a replacement.
In the US you have full rights to a return to the person who took your money for 30-90 days (depends on state). Requests that you use the warranty or contact the manufacturer are unenforceable. Considering this, it is unwise to wait for longer than that period before requesting your money back. The federal statute of limitations for being sold a defective product is 4 years, but that is a lot harder to enforce.
Remember the AMD Ryzen returns? Why wait 2-3 months for a replacement? Demand your money back from Newegg and buy another, they have to process the refund. Also -- most FFLs either state gun returns are outright illegal (a lie) or only accepted for 3 days (against state law) with a steep restocking fee (also against state law). This is the worst case I know of.
>Raptor is a small company and one guy is fronting most of the support load so keep trying, he's not out to screw any customers or provide an inferior product.
But Radio Silence for a few months... surely that is a bit too much?
On the other hand I didn't know Raptor was this small.
It's not that small, the information above is not accurate.
However, I think people that do not see what we see in the support system are greatly underestimating the volume of Email an OEM/ODM receives on a daily basis, which then must be filtered and sorted through to determine the valid issues from the flood of spam and general inquiries. It's not something we have a solution for yet.
This is not accurate, one person is not fronting the support load. However, you are correct in that Raptor is not intending to provide inferior product -- in fact we take any reports of issues very seriously, preferring to determine a root cause and fix it instead of just swapping parts to make it go away.
> ...preferring to determine a root cause and fix it instead of just swapping parts to make it go away.
May I recommend a more customer-friendly way of doing this? Allow the customer to exchange their iffy hardware first, then do the investigations on your time, with the hardware in your labs.
We did try that, only to find that in the majority of the cases the customer was doing something quite unexpected that allowed the fault to manifest. When that results in multiple shipments back and forth, no one is happy in the end -- the techs get frustrated with yet another seemingly functional board returned under RMA, and the end user wastes time with their product in shipping.
I have to agree with sangnoir. I don't want to provide free engineering services. I want a working product. If you are explicit about component compatibility and provide a troubleshooting checklist (all ways it must be plugged in and turned on) then you should assume there is an issue. You should also treat the boards as interchangeable; if a board comes back, you test it and it's good, ship it out again.
A family friend will help me troubleshoot my electronics. So, I have seen the other side of this: I get something which is completely nonfunctional and send it back to the manufacturer who claims it is functional, and tries to bill me for the shipping both ways. That's why if I can not get a working replacement in 2wk or so I just send it back for refund.
For a real example: Are you using very well made power supplies? Maybe your board's power filtering could be better, and customers notice because they buy the cheapest supply that meets their power requirements.
All of the Raptor products use very high quality regulators and power filtering. The only cases of PSU-related damage or malfunction that I am aware of were as a result of a PSU operating way outside of specification (think 12V+ on a 5V rail) or a poorly made PSU exploding and catching on fire, with predictable consequences for the attached devices. Thankfully such incidents are rare, and have never happened with our prebuilt systems that use high quality PSUs.
In general we can diagnose and repair or replace the systems rapidly. This one case was an extreme outlier, and in the future we will just be authorizing RMA and replacement / repair.
Ok, that sounds good. The example was more just to be illustrative. But there is the (slight) possibility of harmonics on the power lines, etc. There are EDA suites that can do such signal analysis on your board but they are quite expensive. If you picked just the wrong values there could be a filter system which amplifies noise from a power supply.
The trick is to get your board manufacturer to eat the return cost. If you can buy testing, do so; usually it comes with some assurance that the boards work, and they either make new stock or remanufacture what you send back. If you can point to a comprehensive testing instructions that the customer would have performed it should not matter if you receive the board and it works for you, for some reason it did not work for the customer.
That's a shame, but I hope this doesn't deter others from trying and popularizing non-Intel architectures. We need to break their grip on the desktop/server/laptop market.
>We need to break their grip on the desktop/server/laptop market.
I'm curious as to why you think this "needs" to happen?
I have my own reasons, for example, openness of ISA and IP from up top at the browser all the way down to the firmware on the chip; however, I'm curious to hear yours.
- A monoculture of HW leaves us more susceptible to security issues. See Spectre and Meltdown. While not eradicated by other hardware designs, the problem is generally mitigates. See relative difference in what Spectre and Meltdown affected as evidence.
- A monoculture of HW may lead us to local maxima of capability. As we approach theoretical maximums of the hardware in specific materials areas, we are presented with more and more complex architectures to eek out smaller and smaller gains. This makes the pursuit of alternative architectures more and more costly to invest in from a business standpoint, knowing that it may be years or decades before it starts to compete favorably with existing products. The more concurrent alternate architectures that can be pursued at the same time will allow for a much shorter time to market for alternatives, should a specific niche fit their capabilities well. (e.g. it's taken a decade, but Apple is making ARM competitive to the point it may be a viable laptop processor competitor, possibly even the better option at some point).
- Multiple avenues of research often yield benefits of cross-pollination. CISC/RISC in the past ended up converging somewhat in the middle as they adopted the best features of each other. The most lines of research we have (and used in the market, so they get sufficient funding and attention) the more likely we are to see benefits for all related hardware.
A slightly different framing than openness that is used from time to time is owner controllable. Older x86 systems were underdocumented, but at least there was the hope of reverse engineering to replace boot firmware, limit device firmware and develop libre device drivers. Not so on today's OEM systems, the CPUs are coming cryptographically fused to only accept the OEMs boot firmware and with locked down, privileged, security co-processors.
I don't want to live in a world where everything new is an iThing that says I can't help you Dave. That could turn retro-computing from a hobby to an essential life skill to preserve freedom!
> I'm curious as to why you think this "needs" to happen?
I think Intel has been too dominant for too long. They've gotten complacent and sloppy, as shown by microcode-level vulnerabilities like Spectre and Meltdown.
Also, I figure breaking Intel's dominance will further marginalize Microsoft. I still hold a grudge over how horrible an experience Windows 98 was.
> breaking Intel's dominance will further marginalize Microsoft
Microsoft is not stupid, Microsoft has ported everything to ARMv8 already, both for the Snapdragon laptops and the servers (which they only use internally for now, but I hope Amazon's Graviton is pushing them in the ARM-public-cloud direction)
Fwiw, the NT kernel was designed for portability from the ground up. During initial development they targeted x86 and the i680 and it has been ported to pretty much any CPU that had relevance over time. Itanium, Alpha, ARM and.... PowerPC. While that branch is probably not maintained currently it would be pretty easy for Microsoft to get that going again.
If you want to run Linux in the cloud, you've got a bunch of providers (Google, Amazon, Microsoft, etc)
Microsoft simply wants to be in that market, so they'll offer the same products their competition does. Anything they can do to make Linux better on Azure is considered a net gain for them
I'm sorry to hear about the problems with your Blackbird. I was an early purchaser as well but my unit arrived rather earlier than yours and didn't have any hardware issues (and I had experience with the platform anyway from the Talos). Raptor support was very good in the early days but I haven't had cause to need it lately and I suspect the load is exceeding their capacity. That's a good problem to have in some respects but if your story is any indication, it could be a sign of looming trouble.
Kudos for that post! Exactly my way of thinking and working. My laptop isn't 11 years old since my last one died in 2014 but I specifically bought a slow Pentium as a replacement. On the up side I still get 6 hours of battery life today.
My former boss always said that all devs worldwide should get the fastest machine possible for development but should be forced to use their own software on a 386. :-)
> My former boss always said that all devs worldwide should get the fastest machine possible for development but should be forced to use their own software on a 386. :-)
I appreciate this thinking, and have always thought that web developers should be forced to test with lynx on 9600baud dialup. But I'm just a wee bit sadistic when it comes to web developers.
"No tears, please. It's a waste of good suffering."
Hm, I have wanted to get RISC-V support on builds.sr.ht, but there are a couple of problems. The main blocker is that I still haven't got a good Alpine port finished for it, the main blocker for that being the lack of a good bootloader available that lets me load a kernel from the filesystem. uBoot is almost there but they need to add an mmc driver before it's ready. Once that's available (or after I have time to investigate it myself), I'll be rebuilding the Alpine port on top of a stable release, then I can start thinking about putting together support for builds.sr.ht.
Do you mind mentioning which RAM module you purchased? On the wiki it looks like some are officially supported and others have been added by users. Maybe if the issue is related to the RAM, others have experienced similar problems. Also, I am looking at the bundle, but don't see any RAM included with it. Was this something that had been offering in the past but since took down?
> Also, I am looking at the bundle, but don't see any RAM included with it. Was this something that had been offering in the past but since took down?
It's an add-on. Click "Add to cart" and then you'll have the option to add it on the next page.
> Do you mind mentioning which RAM module you purchased? On the wiki it looks like some are officially supported and others have been added by users. Maybe if the issue is related to the RAM, others have experienced similar problems.
Not interested in troubleshooting in HN comments, but thank you. There are some leads in the IRC channel that I'll follow up on when I have the chance.
I'm "going" to write a post about my experience with the SolidRun MACCHIATObin as a desktop… (spoiler: I wish it was faster but it's pretty dang awesome in general)
Sounds like a faulty PSU. $70 USD is quite on the low end of the spectrum, especially for a chip with a TDP of 130W. Is it a working pull unit? Do you have the ability/option to try another unit?
The chip he appears to have bought is a 90W TDP part. I have a pretty beefy 650W power supply in my Blackbird, but I think that was overkill, and it was $94. I would certainly try another PSU but I wouldn't immediately think that would be the problem.
You're pretty close, though in this case it's apparently the RAM controller inside the CPU itself. This failure mode is incredibly rare but it just happened to hit this one system, unfortunately.
This information is not public, however I do want to note that the products are not exactly niche. We'll have some announcements shortly that will help clarify where we intend to go with the newly created POWER desktop market segment.
As much as I like the whole "oh we're a boutique business!" model this kinda stuff has been my experience routinely with anything computer related.
Don't be the small business/entrepreneur outlier unless you're a multi-million dollar business, they can't afford to care about your 2k dollars too much
Please don't dismiss Raptor over this. I've had nothing but a great experience ordering from them. I do not have a Blackbird but my Talos II Entry system (1-cpu slot vs. 2) works flawlessly.
So you were lucky to receive a system that just works without issues so you never had to contact their support in the first place. That way of reasoning would work for even the shittiest company on this planet.
I'm really glad I read this piece. The geek in me was playing with the idea of getting a Talos 2, but the pricing makes me really hesitate. Knowing that if I'd end up with a problematic unit their support will be absolutely worthless is important information.
We've shifted our policy to allow rapid RMA of problematic units, which is more in line with other vendors. Please be assured you won't be stuck a problematic unit, especially with our well-established Talos II line. For Talos II, which has been on the market for two years, we have a well established support system in place, which asks for some basic diagnostics to verify a faulty board (this is rare) before authorizing RMA for repair or replacement.
With Blackbird there was more of a focus on trying to understand any new faults found in the field, given their tendency to disappear when the units were brought back for analysis. Over the months after Blackbird shipments started, we learned that the majority of those were related to improper customer component installation (CPU, RAM, PSU, chassis), and are in the process of expanding the existing RMA policy to cover the Blackbird line.
Actually I did have to contact support since the shipping took a few weeks. Understandable given the exotic hardware and, well, they're small and they support freedom in computing, so I have no qualms waiting.
I got a reply relatively quickly, but this was before they started shipping the Blackbirds so it's possible they are just swamped with orders.
Timothy Pearson from Raptor here. I'd like to first extend apologies from the apparently poor support experience from Raptor Computing Systems -- that being said, there are a few exceptional circumstances in play with this system:
1.) This particular case is one that Raptor needed to consult with IBM engineering to determine the correct course of action for. There is a very rare CPU failure mode (we have only seen it happen twice) that manifests with similar symptoms and a ZCAL failure. The root cause here is an apparently defective CPU that slipped past IBM final test, and work is underway to make sure that these marginal, but defective, CPUs are caught instead of being shipped out.
2.) There is definitely more than one person in support, however Raptor has been overloaded with both spam and off topic inquiries. Raptor has attempted to manage this load without much success, despite deploying new features such as the troubleshooting guide at https://wiki.raptorcs.com/wiki/Troubleshooting/Support_Reque... we continue to receive support requests that are missing vital information, and with descriptions that are so generic that the problems could range from "PC not plugged in" to "used non-ECC unbuffered RAM" to "tried to install Windows" (yes, all of those have happened, repeatedly).
3.) We are currently gearing up for a major push on owner-controlled computing. Higher level technical resources are assigned to those projects temporarily, which means unusual issues like the ZCAL problem are being resolved at a slower rate than normal.
I'd ask that the general community please bear with Raptor as we work through these problems. What we're doing is hard, it's not as simple as shipping a RISC-V SBC in a known configuration, we have the entire complexity of the traditional PC ecosystem in play. AMD's release of the uber-fast (but ultra-locked-down and owner-disrespecting) Rome CPUs has not helped the owner controlled computing world, and I am still of the opinion that POWER is a better long-term investment than RISC-V in the desktop and server space. FWIW I use a Blackbird desktop as my daily driver, and have not had any complaints with it (in fact it's given me less problems over time than my previous daily driver -- a corebooted ASUS KCMA-D8).
Raptor has a Wiki with a ton of information available at wiki.raptorcs.com and I am definitely open to further suggestions on how to make the nascent POWER desktop community a bit more self sufficient than it is now. Remember that x86 vendors don't need to offer significant post-sale support, partly because the Linux community has already filled that role, and as a result they can continue to offer low prices, strengthening the lock-in effect. If owner-controlled computing is to succeed in the marketplace, we need to figure out the best way to ensure that vendors aren't bearing the brunt of those costs and being forced pass them on in higher product pricing.
I believe support has reached out to you now that we understand the CPU will need to be replaced. Can you confirm this?
>I believe support has reached out to you now that we understand the CPU will need to be replaced. Can you confirm this?
Yes, support reached out. I said as much in the email, but to address some of your comments publically:
(1) Totally understand this and I posess a tremendous deal of tolerance for weird problems with unusual hardware from small companies. But the support experience here was abysmal and no amount of weirdness in a problem excuses the lack of communication. I would be with you every step of the way if Raptor had communicated the progress - the first I heard that IBM was even being consulted about the issue was today - or even sent me communications at the times they promised to in earlier replies.
(3) I think you had really better fix (2) before you increase your volume. The reply I received from support today insinuates that my support experience is rare, but I can't throw a stone in the IRC channel without hitting 3-4 people who have had similarly bad support experiences. Whatever investment you're putting into your upcoming "major push", you should probably siphon off some of it for investing in better support.
It also bears mentioning that 50+ days of silence, not including an additional 30 days where communications were devoid of information outside of unfulfiled promises about future communications, followed by a detailed response on the day I publish a scathing review online, is worth note. I should hope that other customers who don't have a soapbox to complain on are also having their concerns addressed.
This was originally going to be published on the internal ticket, but since the issues were raised here I'm going to post here instead.
First, I'd like to point out that Raptor already identified one of the major problems here -- that front line support failed to keep you notified of back end progress in diagnosing the problem. Raptor Computing Systems is an OEM and ODM first and foremost, not a services and support firm -- their capacity to handle requests is limited by the amount their customers are willing to pay at time of product purchase, and internal management of RCS is trying to balance support costs with all other costs and overall product pricing. They may not always get it right, but they are trying nonetheless.
Yes, this was brought to my attention as a result of the review. That review has apparently caused sufficient damage that Raptor will now be reevaluating whether they can afford to service the individual market at this time, or whether they need to focus on business users only.
My personal take:
I am saddened to see x86 and locked computing profiting from this, but at the end of the day individuals have always been a difficult segment to target. They tend to want extremely low (below cost) pricing, high levels of service, and generally want to run various proprietary games on Windows while simultaneously settling for a false sense of security through open source firmware wrapping large hidden binary components. Out of sight, out of mind, purchase price above all other concerns seems to be the general takeaway for that market.
As a result of this mentality, individuals are not currently Raptor's core market, and at the end of the day the individual computing market will get what it is willing to pay for. If that's locked (but cheap) x86 computing, open source itself may be in serious trouble due to the impending final lockdown of those platforms. If it's instead cheap low power computing (RISC-V, low end ARM) there's an awful lot of fat that will need to be removed from near-essential projects like Qt, KDE, Gnome, Firefox, Chromium, GCC, Clang, etc. to better fit them on the lower end platforms. I have not seen any developers yet willing to put in the time and effort to re-architect those systems for low end computing resources; in many cases the fundamental design would need to be changed (i.e. remove interpreted scripts as much as possible, go back to C/C++ or a similar low level language, etc.). In the case of projects like Chromium, it may simply not happen as the intent of that particular project is partly to serve ads and DRM protected content through a portal system, which aligns nicely with the idea of locked end user computing on x86.
Just to close the loop here, we've pushed up the timetable for implementation of some of the support-related improvements (several are live now) and are giving the individual market another try. Our status dashboard is showing no unanswered tickets at this time, so we should be starting from a clean slate of sorts at this point.
To anyone having support issues, please feel free to re-raise any issues that were not addressed to your satisfaction earlier this year (including ones that may have been auto-closed)! Management is watching closely at this time to observe the process and make sure things work well for Raptor's end users.
This reads like blame shifting, not accountability.
> Yes, this was brought to my attention as a result of the review. That review has apparently caused sufficient damage that Raptor will now be reevaluating whether we can afford to service the individual market at this time, or whether they need to focus on business users only.
See: the review caused the damage, not Raptor's failure to provide a good support experience. In any case, if you can't handle the support load, then you're probably right in reconsidering.
I am really rooting for Raptor here, I'm not your enemy. I agree that x86 monoculture is awful for the ecosystem and ARM hardly presents anything better. This is also why I got heavily involved in RISC-V, and I wanted to find something similar in POWER9. I haven't laid hands on a working Blackbird yet but if it is as good as I hope it is, then I think you have a good shot if you just focus on scaling and fixing your support problems. If you can solve the support and get the system working, I would be thrilled to amend my review and publish a new one with all of the things I like about my shiny new Blackbird system.
> I am saddened to see x86 and locked computing profiting from this, but at the end of the day individuals have always been a difficult segment to target. They tend to want extremely low (below cost) pricing, high levels of service, and generally want to run various proprietary games on Windows while simultaneously settling for a false sense of security through open source firmware wrapping large hidden binary components. Out of sight, out of mind, purchase price above all other concerns seems to be the general takeaway for that market.
I can't claim to know what the firehose of support tickets is like, but know that there are users who don't fit this strawman. I'm one of them. I'm not here to run proprietary games on Windows. My stated goal is to promote the development of portable software and the adoption of freer systems by using my board to introduce POWER9 support to builds.sr.ht, giving access to POWER9 cycles to tens of thousands of developers. It's causes like this which help address the problems you mention of fat software needing to be improved to make alternative architectures viable. No small part of why Chromium is slow on ppc and fast on x86 is because there's a JIT for x86 and not for ppc, not necessarily because of fundamental problems in Chromium's design.
But yeah, there are fundamental problems in our entire ecosystem with respect to disreagarding performance thanks to the luxuries afforded to us by fast x86 systems. Is the answer to this that we should be writing off alternative hardware models? No, the hardware should strive to be fast and the software should strive to meet it. Both parties have a responsibility to performance and we can't lay the blame at each other's feet and ignore our role in it. It's a slow process to change the industry, we must exercise patience.
EDIT: it's been pointed out to me that Chromium has a JIT on POWER9. This doesn't substantially change my response.
> This reads like blame shifting, not accountability.
I don't think I was clear enough here, sorry. Yes, Raptor Computing Systems is accountable for its poor support experience. Their management is trying, unsuccessfully, to manage the support load. I am actually on the engineering and technology side moreso than the support and business side, and I have no direct ability to change how support is handled beyond making (strong) recommendations.
Fundamentally, however, the blame does lie primarily with consumers that are unwilling to pay for owner controlled computing. Raptor took a risk with the Blackbird, attempting to broaden the reach of owner controlled compute beyond business and high end workstation users, and fundamentally underestimated > 20x additional support load that new market would end up causing. Raptor does not have the resources to handle that kind of support load, and the increases in product pricing required to do so would put Blackbird back out of reach of everyone except business users.
Raptor was working on fixing these issues regardless, but AMD Rome has made that nearly impossible due to the (predictable, but unfortunate) customer reaction that I detailed earlier. We have had to pivot to a different mechanism to try to resolve the problems, and I am not allowed to say more publicly as to what that will be at this time, other than it is coming for October.
I am an enthusiastic customer, and in case you missed my comments to you elsewhere, if you do shift sales in another direction, please consider continuing to sell to the hardcore old customers willing to do their own work on this hardware whatever it takes. This project is too crucial to let randos with just a light PR attack choke off the supply of this kind of hardware to people to whom it is so utterly critical.
Not sure if you're following IRC, but just in case I'll mention it here as well: Raptor products are already outpricing most consumers. Even in casual conversations with hacker friends, most of them balk at the price. Point being: if you increased your prices most of your buyers wouldn't blink. At present your buyers are blinking at the poor support experience. I would have paid a 50% premium for a product with actual support.
A compromise would be to offer ala-carte support packages, perhaps billed yearly. I would also have paid for that.
I know these decisions aren't in your court, but I don't have anyone else to share my opinion with, so there you have it. I hope that I will publish a more positive review of Raptor with these problems addressed soon. Thanks for coming by the thread to share your thoughts.
(1) I would have paid 50% more but I also imagine that 50% is hardly necessary to fund a proper support system. That represents an increase of nearly a thousand dollars per unit.
(2) Having known the person (singular)* who said they'd be priced out at +50% from other interactions, I wouldn't take their comments at face value.
What you're saying is disingenuous at best and gaslighting at worst. Just because you apparently have a very generous budget to throw at a problem doesn't mean everyone else in the same space does. I really struggle to see what the point is in making such a comment about other people's financial situations, and I would hope you can either justify making such an inflammatory remark or walk it back.
The specific person I'm referring to has lost any goodwill from other interactions I've had with them. I don't trust their statements.
And like I said, I would have personally paid a 50% increase, or for an ala-carte support package, but I doubt a 50% bump is actually necessary. One which doesn't push the board out of people's price ranges is probably feasible. Remember that we're talking about $2-3K, for the cheapest option with the simplest loadout. Customers already have disposable income. These are not priced like consumer hardware.
I beg you to reconsider the repercussions of holding grudges, especially in the FOSS community. All too often I see this pattern where people take a few disagreements too personally and hold them against each other for eternity. I know who you're talking about specifically (I was on the IRC channel at the time) and I know that they are genuinely trying to make it up to you (full retractions, refraining from speaking ill of the sort in the future, attempting to adjust their behavior so that they are not so quick to jump to so-called "hot takes" or other aggressive behavior). I hope that you, in turn, could also, in your own time, learn to forgive and let go as well. Please, for the sake of good FOSS everywhere - I know the two of you could accomplish so much!
Since we seem to have reached HN's reply nesting limit, posting here...
Good to know on the paid support side. This is something we've been considering for a while, and is probably coming as part of the mitigation strategy next month.
Tip: click the timestamp to reply to any comment without regard to the nesting limit or flamewar protection. Don't tell dang about this, it's probably a bug.
I'm glad that things are moving foward in a productive way. I hope to be an early customer of paid support.
I would just add that the #talos-workstation channel on freenode is a very good support resource. While we may not have been able to FIX the issue mentioned in this thread a lot of issues do get discussed and fixed daily. As a new and very happy Blackbird user I got a lot of great support there.
Raptor is a small company and one guy is fronting most of the support load so keep trying, he's not out to screw any customers or provide an inferior product.