My YOShInOn RSS reader uses BERT-family embeddings to capture the "gist" of text the way the neural network for that search engine did, then I treat content-based recommendation as a classification problem. (e.g. "Will the user like this?")
My system is quite a bit different from others because, like TikTok but even more so, it demands a thumbs up/thumbs down judgement for every article so I get a set of negative samples that are really reliable.
There are numerous frontiers of improving this system. One of them is that there are certain things, like "roundup" articles that cover a wide range of topics (say https://www.theguardian.com/world/live/2024/jun/03/russia-uk...) that the embedding doesn't capture well, adding some new features could clear out maybe 10% of articles I'd rather not see but I am not in such a rush because overall the system is very satisfying to me and I am already blending in more random articles than that to get samples to keep it calibrated and also sometimes discover new topic areas I find interesting.
Another interesting frontier is sequential recommendation
but I'm not sure if I really want to take an ML approach to this because I'm not sure there is enough training data for one person's content-based recommendation. I'm not sure exactly how I want to do it, but when I post to a place like HN I do not want to post a stick of five articles from phys.org, rather I want there to be some diversity in my feed not just over the course of a 300 article batch but on the scale of individual posts. Items can be hung up in queues for several days in this process; most "news" on HN is fairly evergreen and it doesn't matter if it is delayed a week but articles about sports have a short shelf life as you look like a fool if you post an article about what happened in week 3 of the NFL after the games of week 4 have played. So I need some way for sports articles to "jump the line" ahead of other articles but I don't want that to privilege sports over everything else.
Similarly there is "the probability that article A is relevant" but there is also "Is A or B a higher quality article?" One Google innovation was using a document quality score (PageRank) asides a normal document-query ranking which is tricky because now you're not optimizing for one thing but trying to optimize for two things that could compete with each other. I am thinking about switching that system from a batch to a streaming mode and need some answers for that.
Take a look at HL15 from 45Drives. Many will rush to tell you it’s expensive for homelabers. Sure, but not that much. The retail price of the motherboard is 800 USD, CPU, PSU, NVME RAM around 400-500. That leaves you with the case which is 800 USD. This is a premium case, but probably 200 USD more than I would like it to be.
You will get a motherboard up to 1TB of ECC RAM, 10G networking, space for 15 HDDs, IPMI, a 4U case that will last forever, space for GPU, and a lot of expansion and upgradability possibilities via PCIe.
Dell, HP, supermicro, … all sell entry level servers.
> Why would you use home-grade "servers" when a startup has access to a variety of cheap cloud solutions? What business market segment in a server space would these mini PCs occupy?
This is more of a hypothetical question. From simple use, the Beelink SER5 I have seems to have better performance than my 13-year-old Macbook Pro, so I was curious about the upper bound / limits of little machines like it.
May I ask what you mean by "a variety of cheap cloud solutions", do you refer to services such as DigitalOcean's Droplet and/or similar products?
> From simple use, the Beelink SER5 I have seems to have better performance than my 13-year-old Macbook Pro, so I was curious about the upper bound / limits of little machines like it.
Beware of Beelink. They are known to suffer from weird hardware design issues (i.e., mouse/keyboard activity required to wake up from sleep mode but their USB stack is disabled when entering sleep mode) and are riddled with other hardware issues. I bought one and for example video randomly shuts down a few seconds after booting in without a way to recover.
Their hardware issues are surprisingly not the problem. The problem is that their support is limited to their message board, and it's heavily censored to hide all issues. I bothered to go through their china social credit score process to register a user account to report the issues I experienced and all posts were heavily censored. In fact, the first post I noticed being censored out was a reply to a boilerplate post with troubleshooting tips where I simply said that I tried out the suggestions but they weren't effective at fixing or mitigating the problem. This left their board with a thread where a problem is reported, they chime in with a list of fixes that they automatically post to all problems, and by magic no one bothered to reply.
Azure, AWS, GCP, DO as you noted all have viable alternatives, from IaaS to PaaS to SaaS.
Why run a web server when you can run an Azure Web Site? Why distribute servers around the world to serve as CDNs at POPs when you can put Cloudflare, Azure Front door, etc. etc. in front of it on the cheap?
Anyway, these small machines don't have a place in the modern server room. They don't fit in racks, they lack management features, expandability, replacement parts, warranty, and so on and so forth.
There is more to a server than "CPU goes faster". If a hypothetical server had the fastest CPU available but only provided a single NVMe interface (like many of these small PCs do), it'd be a poor candidate for a critical RDMBS, for example.
> I am using a ZimaBlade (2 cores, 16gb DDR3L RAM) as my cloud server, and an old Nvidia Jetson NX (very outdated ARM CPU with negligible GPU power) as my email server. I also run a bunch of frontends and apps on both. They work flawlessly.
How do you like the ZimaBlade and NVDA Jetson? I actually want to get these two, and have been consider going with ZimaBlade or ZimaBoard...
I want to have a Jetson for LLM purpose, but after checking the specs, I don't think it would be powerful enough though...
> ...... we were in the process of moving a bunch of hosting services on a mid-tower server which actually was less powerful than nowadays NUCs
Do you mean you are moving these hosting services to a mid-tower server (English is not my first language, so to is easier for me to understand than on). Would you say that even if they are not more powerful from performance perspective, but they offer some server-specific functionalities that NUCs don't, such as better thermals?
> I want to have a Jetson for LLM purpose, but after checking the specs, I don't think it would be powerful enough though...
Don't waste your money, you can make a way more useful and powerful build for slightly more money. Unfortunately the Jetson Xavier (at least the Xavier, no idea about the Orin but seen the price I'd say you can build a mid tower for the same price) is unable to run any LLM in a decent way ( i got like 0.1 token / s on a 7b model lol)
> How do you like the ZimaBlade and NVDA Jetson? I actually want to get these two, and have been consider going with ZimaBlade or ZimaBoard...
I use the Jetson only because else i would have to throw it away, to be honest I prefer the ZimaBlade hands down even if it is the dual core version.
About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID.
I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.
> Do you mean you are moving these hosting services to a mid-tower server (English is not my first language, so to is easier for me to understand than on). Would you say that even if they are not more powerful from performance perspective, but they offer some server-specific functionalities that NUCs don't, such as better thermals?
I am not native too, sorry if something was unclear. Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).
In general, I'd suggest trying the ZimaBlade 2 cores which is pretty cheap and can give you an insight of SBCs used as servers.
In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)
> About that, I am waiting for the quad core version to arrive. I liked it so much I bought another unit that I will use just for Nextcloud in RAID.
>
> I am amazed by the zimablade performance especially as it is a very low specs machine theoretically. Never tried the zimaboard but I have no reasons to do it.
Thanks! Then I will consider buying one for myself!
> Yes, we were moving these hosting services to a mid-tower "server" (actually a normal PC with linux). Thermals surely were better handled due to the size of the build but from my personal experience you can add some thermal solutions to NUCs and get the same / better results. Note that the mid-tower build I am talking about would be considered low spec nowadays (I don't remember exactly the specs but it was surely pre-2019).
No problem, thank you for the clarifications!
> To give you a practical example, you can try
>
> https://view.tcsenpai.com/feed/popular (invidious instance hosted on the Jetson Xavier)
>
> https://swing.tunnelsenpai.win/#/ (hosted on the ZimaBlade)
>
> In my opinion, while being way cheaper, the ZimaBlade obliterates the Jetson Xavier (which btw is forced to run at 20w vs 6w and must use Ubuntu 18.04 with a 5. something kernel)
May I ask which countries in east Europe would you consider to be good and convenient, esp for foreigners (non-EU)? When I visited Budapest I like it there, but don’t know much about the tech scene.
Estonia and Bulgaria both have good English speaking levels, decent visa schemes, low and flat taxes, very good in the case of Estonia and maybe getting there in the case of Bulgaria government digital services, and lots of people in tech (so hiring shouldn't be a problem if you can provide good salaries).
Also, Romania, Poland have good reputation in this space but I don't have any direct experience with them.
Thank you for the reply! Do you mean the Estonia e-residency? If you got one, may I ask what your experience with it has been? I have always been thinking about it.
Also, what would you say about the advantages of hosting yourself, compared to say fly.io or similar services? For instance Docker or container seems to “the way” right now? I’m relatively new to web dev…
I'm old school; i don't do Rails or Django; I've got a mod_python on my apache but I haven't used "in anger" yet; all im serving now is static HTML.
However, previously I served several thousand users daily over a span of some years using a dual Pentium with 128MB of RAM and 6GB of spinning rust hard disk. Using earlier versions of the same software stack. Most of that traffic could be characterized as "static" as well.
The prime advantage of self hosting for me is editorial control. Hosted elsewhere, who knows when an opinion previously expressed becomes "problematic." If I manage to really piss someone off; they'll have a slightly larger amount of trouble having my site taken down than the usual single complaint to a service provider.
Thank you for your input! I always considered a Raspberry Pi, though I bought an Arduino first last night……
With sufficient and appropriate configs, how many users do you think a Pi can serve at the same time, say, with Rails or Django?
I also agree with you on that many things were not invented or appeared during those early time, and C (or Common Lisp) was common choices than say Ruby or Python.
