This repository contains dumps of the monthly Chrome UX Report (CrUX) datasets.
Dumps are automatically generated by exporting URL and rank columns from the CrUX dataset via BigQuery, grouping URLs by rank, and storing the grouped URLs as XZ compressed archives. This repository is intended as a convenient alternative access, since exporting the data from Google BigQuery is both cumbersome and expensive.
The latest dataset (from March 2023) contains more than 18 million websites.
Note: There is no unique rank assigned to websites in CrUX datasets. Instead, they are grouped into buckets of similar ranks.
This repository contains dumps of the monthly Chrome UX Report (CrUX) datasets.
Dumps are automatically generated by exporting URL and rank columns from the CrUX dataset via BigQuery, grouping URLs by rank, and storing the grouped URLs as XZ compressed archives. This repository is intended as a convenient alternative access, since exporting the data from Google BigQuery is both cumbersome and expensive.
The latest dataset contains more than 18 million websites.
Let's assume just for a second that copyright violation isn't a problem, that 1B is enough to motivate enough people, that it's not overrun by bots, etc..
What kind of blockchain would actually be able to deal with the expected transaction count? According to [1] Ethereum peaks out at 15 transactions per second, but [2] claims around 6.000 tweets per second on average.
A difference of 2–3 orders of magnitude is not something a team of 5 engineers can simply solve in 6 months.
Don't forget that tweets are not just tweets. In 2016 Twitter handled 3000 images per second (around 20GB per second) [1]. The number undoubtedly increased since then. There's also video.
> Don't forget that tweets are not just tweets. In 2016 Twitter handled 3000 images per second (around 20GB per second). The number undoubtedly increased since then. There's also video.
> Good luck storing those on blockchain :D
I'll admit that a normal blockchain would be impossible to store that level of transactional data. Of course that doesn't mean that it isn't possible to store it in a decentralized manner: It does, however, require different methodologies to make it work.
Currently, the best attempt at this comes from both Filecoin & Arweave, but they're both coming at the problem at different angles [1]: Filecoin tries to solve the storage problem as a "Pay for X storage for Y time" problem, whereas Arweave is trying to solve it as a "Pay to store X storage forever" problem.
Going by Arweave's current stats [2][3], it is possible that the network could be able to handle that level of content generation with a high enough node count. In fact, a stress test of sorts is currently underway, as there's an attempt to try and store the Russia-Ukraine conflict on the network [4].
> I'll admit that a normal blockchain would be impossible to store that level of transactional data. Of course that doesn't mean that it isn't possible to store it in a decentralized manner
Translation: blockchain a) doesn't solve this problem, and b) isn't required for this
> > I'll admit that a normal blockchain would be impossible to store that level of transactional data. Of course that doesn't mean that it isn't possible to store it in a decentralized manner
> Translation: blockchain a) doesn't solve this problem, and b) isn't required for this
Clarification: The data structure of a typical blockchain that is regularly talked about (single-chain, no shard/web) definitely cannot store that level of data. In order to reach that level of data throughput, better data structures are required to do so.
Single-chain blockchains are great in terms of determining transaction finality, but are poor in terms of data storage. This doesn't mean that research into decentralized & computationally-assured data storage techniques shouldn't be pursued.
> This doesn't mean that research into decentralized & computationally-assured data storage techniques shouldn't be pursued.
No, it doesn't, for one simple reason: blockchains is not the only such tech, and "computational assuredness" probably isn't really a requirement for this.
> > This doesn't mean that research into decentralized & computationally-assured data storage techniques shouldn't be pursued.
> No, it doesn't, for one simple reason: blockchains is not the only such tech
Again, reiteration must be applied here: The standard single-shard reference-to-previous-block data structure that is the initial blockchain structure is non-conducive towards decentralized data storage. As stated beforehand:
> > The data structure of a typical blockchain that is regularly talked about (single-chain, no shard/web) definitely cannot store that level of data. In order to reach that level of data throughput, better data structures are required to do so.
> , and "computational assuredness" probably isn't really a requirement for this.
Towards the latter half of your statement, if computational assurance is not required, then standard trust-based storage solutions can be implemented instead.
HOWEVER (and it should be stressed with extreme emphasis on the word), in that scenario, concerns about the centralized nature of such a storage solution CANNOT be launched by critics: It was their criticism of decentralized storage solutions that caused the shift towards standard trust-based storage solutions, and thus they cannot criticize the move towards the latter. Otherwise, their criticism is not out of technical concern, but out of personal opinion.
> concerns about the centralized nature of such a storage solution CANNOT be launched by critics: It was their criticism of decentralized storage solutions that caused the shift towards standard trust-based storage solutions, and thus they cannot criticize the move towards the latter. Otherwise, their criticism is not out of technical concern, but out of personal opinion.
This sounds like a rant devoid of meaning.
Yes, there are centralized solutions. Yes, there are decentralized solutions. Yes, critics have full right to criticize both, because both have their failings.
This has literally nothing to do with whatever ideological angle you're trying to force.
> > concerns about the centralized nature of such a storage solution CANNOT be launched by critics: It was their criticism of decentralized storage solutions that caused the shift towards standard trust-based storage solutions, and thus they cannot criticize the move towards the latter. Otherwise, their criticism is not out of technical concern, but out of personal opinion.
> This sounds like a rant devoid of meaning.
> Yes, there are centralized solutions. Yes, there are decentralized solutions. Yes, critics have full right to criticize both, because both have their failings.
> This has literally nothing to do with whatever ideological angle you're trying to force.
The ideological angle in this case is: "N solutions have a mixture of good & bad, but at least one of them is required for system S to function. Critics that criticize N-1 of N solutions cannot criticize the usage of the Nth solution when that is the only other solution left, after catering to their desires for (1...N-1) of N solutions to not be used."
> What kind of blockchain would actually be able to deal with the expected transaction count? According to [1] Ethereum peaks out at 15 transactions per second, but [2] claims around 6.000 tweets per second on average.
Admittedly, the only way to be able to achieve that level of throughput would be to require zk proofs for assured transaction finality, on top of being a Layer 2 scaling solution.
Currently, that level of throughput can be obtained by using a specialized ZK-rollup network like
StarkNet [3] if it needs to be deployed right now. Otherwise, zkSync 2.0 [4], an EVM-compatible general ZK-rollup, is currently in its testnet stages & is currently the desired goal for scaling Ethereum's TPS up to the desired amount.
The technology is currently still in it early stages, as demonstrated by zkSync & StarkNet, the capability to reach that level of throughput is possible.
Radio.net is a leading audio platform that allows you to listen to 40.000 internet radio stations and over a million podcasts in your browser or through our iOS and Android apps — but we're also integrated with car multimedia systems and third-party devices like Sonos.
The current backend team of 3 is looking for an experienced engineer who's excited to help build the next generation of our APIs and services. Our tech stack is built on Golang, Postgres, Redis, DynamoDB, and Elasticsearch running on AWS' Fargate and Lambda via Terraform. We embrace the "you build it, you run it" principle by owning all operational responsibilities.
Radio.net is a leading audio platform that allows you to listen to 40.000 internet radio stations and over a million podcasts in your browser or through our iOS and Android apps — but we're also integrated with car multimedia systems and third-party devices like Sonos.
The current backend team of 3 is looking for an experienced engineer who's excited to help build the next generation of our APIs and services. Our tech stack is built on Golang, Postgres, Redis, DynamoDB, and Elasticsearch running on AWS' Fargate and Lambda via Terraform. We embrace the "you build it, you run it" principle by owning all operational responsibilities.
I think this is too harsh. A company can establish an enjoyable and empowering culture, that attracts and supports talented developers — and can still try to avoid hiring obstacles at the same time. It might not have been what therealmarv meant in his post, but your comment makes it sound mutually exclusive.
As a counter-argument: I've experienced several times that "niche tech" companies offered non-competitive salary packages and perks, because they offered the cool tech instead; "sure, we can't match that other offer, but we built our stack on that language/tech that is so hot right now".
Not arguing about the quality of Elixir, just about the gatekeeping that happens in this thread.
I have never used Elixir, so maybe it's a great language, maybe not. But I have to question the reasoning of the post simply going by the comments about other languages in the "Conclusions" sections — some of which I did use extensively.
Really, Go "is the choice if you need to 'sell it' to a 'Boss'" and the imperative programming style leads to more complexity? And Python/Django can only be used if you "don't need anything 'real time' and just want RESTful 'CRUD'".
I get it, you guys like Elixir, but painting the world using such broad strokes doesn't really sound like "kaizen learning culture" to me, but more like "Negative Nancy".
I really like Elixir. There are a lot of practical realities that can make Elixir not the best language to use in many situations, and the same is true for any language. Just ignore the hype train, because you'll find one for every language.
I'd say Elixir's killer feature in today's day & age is concurrency. I'd argue that using concurrency is appropriate in most programming situations IF your language's concurrency model isn't a pain in the ass to use. You can write completely non-blocking, async code in Elixir (and Erlang) without losing your mind. The preemptive scheduling is nice, too.
I love a lot of other stuff about Elixir, too. Pattern matching, process supervision, tooling, documentation, etc.
Phoenix speed/scalabity is quite on par with Go frameworks like Gin. I wouldn't use Django in 2019 for a new project. It's not even async/non blocking.
Dumps are automatically generated by exporting URL and rank columns from the CrUX dataset via BigQuery, grouping URLs by rank, and storing the grouped URLs as XZ compressed archives. This repository is intended as a convenient alternative access, since exporting the data from Google BigQuery is both cumbersome and expensive.
The latest dataset (from March 2023) contains more than 18 million websites.
Note: There is no unique rank assigned to websites in CrUX datasets. Instead, they are grouped into buckets of similar ranks.