Near-zero cost transactions are not sustainable. They're basically burning through their VC money to pay a small number of validator nodes to host their blockchain, which is not that different from using a centralized database.
I was starting to look at the same thing. I used to do a bit of Django development (5ish years ago) and things seemed to have changed a good amount with the tooling / frontend.
Does anyone have an opinionated stack they like with Django to get things up and running quickly that could also set me up well for the future for a website and an app(if I want to transition the website to a stable business, etc)?
The things I'd be looking for opinions on:
- django vs DRF
- react (or next.js) vs vue vs something else for frontend
- what other tooling should I use for modern front end development
- what else is in your stack for CI/CD, rollbacks, db migrations, deployments, etc.
- do you use any other templates or themes for the frontend to make the UI look good?
- how you handle security / auth when using a front end framework with django as the backend -- are there any tricks? I've heard JWT can cause headaches
- any django packages they recommend using, or other front end packages that are popular
- Huey for background jobs
- Whitenoise for statics handling
- Gunicorn for running the application
- Honcho for process management (also for development, for spawning js and css bundling)
- Slippers for django template components
- django-all-auth for signup/password recovery/etc.
- django-rest-framework if you need an API
Frontend:
- Unpoly for ajaxy SPAish feel (just the good parts of that feeling)
- Tailwind for styling
- esbuild (CLI) for bundling js
- npm for frontend dependencies
- postcss (CLI) for building/packaging css
Deployment/infrastructure:
- Dokku for hobbie or side projects
- Heroku for getting started with production apps, AppEngine if shit gets real.
- No need for Websockets, polling is good enough most of the time.
- Sqlite for side projects
- PostgresQL for serios stuff
- Maybe redis
It's not a "vs"; DRF is merely a library for Django thus requires Django. It gives you building blocks to create REST APIs and handles the boring stuff such as validation & serialization, pagination, rate-limiting, authentication, URL routing (in a RESTful manner), etc. If you need more than a few read-only API endpoints I suggest you go with DRF; it takes some time to learn the way it wants to do things but it's worth it in the end.
If all you need is to expose your models as a REST API, DRF will do that with pretty much zero code required, and in most cases that's enough. You can of course decouple your API structure from your models and write custom viewsets, serializers, etc but in this case you lose a lot of the built-in functionality of DRF and have to implement it all manually.
> react (or next.js) vs vue vs something else for frontend
Look at it from the perspective of the problem you're trying to solve as opposed to hype or frontend developers' career prospects.
Good old server-side rendered HTML (with ad-hoc JS or even jQuery where needed) can be good enough. The main advantage is that there's no barrier to entry, you grow as you go. Since you render the pages directly on the server you don't have to worry about providing API endpoints, then parsing that data back in JS, you just directly output it in the HTML.
If you do choose a single-page application approach, I suggest serving the SPA as static files from your Django app so that it is served from the same domain from the browser's point of view. This also allows you to mix and match SPA and server-side-rendered views (you can have your main app an SPA, but the login page - which is just a form and has no need for SPA features - can be server-side rendered for example).
In terms of which frontend framework to choose, I would advise against React as it's just a UI library meaning you need to (poorly) reinvent everything else such as URL routing, REST API clients, etc manually. Every React project I've been involved in had a custom, different, and differently flawed way of doing basic things like hitting a REST API even though it should be a solved problem (there's DRF on the server that can generate RESTful endpoint based on just a model, why is there no "JRF or JavaScript REST Framework" as a client-side equivalent?). I think Ember.js might be worth exploring as it's an MVC framework and should take care of the boring stuff so you can focus on business logic. It looks like it has the concept of "data adapters" so provided your API has a consistent structure it looks like you can have a data adapter that'll automatically map it all back to JS replicas of your model classes and then be able to use it as an ORM of sorts (and have the network communication, serialization, pagination, etc abstracted away by the data adapter).
> what else is in your stack for CI/CD, rollbacks, db migrations, deployments, etc.
Django now has built-in migrations which is pretty good. Deployments and stuff is highly dependent on whatever target you're deploying to. One thing I could suggest is Dokku (https://dokku.com) coupled with a bare-metal server from OVH or Hetzner. It's basically Heroku but orders of magnitude cheaper and more powerful. Depending on your uptime requirements this might be good enough for production, but if not it's at least a good option for a staging/test environment.
Use Celery with RabbitMQ for background & scheduled tasks. Do not use the Redis backend for it even if it's technically supported - it can fail catastrophically with the workers thrashing on the same task over and over again (speaking from previous experience having had to use the fire extinguisher more than once).
Use Sentry (cloud or self-hosted) for exception tracking.
> how you handle security / auth when using a front end framework with django as the backend -- are there any tricks?
Built-in Django auth is good if all you need is username/password auth. The django-registration package provides server-side-rendered views for registration/login. If you need social/federated login I suggest django-allauth which also provides its own views. For API endpoints you'll most likely need to roll your own login endpoint (with serializer) which takes a username/password, calls the auth backend(s) to check those credentials and then returns some kind of reusable token to the user (see below).
> I've heard JWT can cause headaches
JWTs (or other auth token formats) only come in play once you've already authenticated the user somehow (by them providing some secret info - most likely a username and password) and just want to give their client a way to reauthenticate without having to resubmit the same secret info again and again. You still need to actually handle that first part (see above). Once you actually authed the user and want to give them a token, then JWTs or similar come into play.
JWT's only selling point is that they can be validated statelessly without hitting a DB (they carry all the information you need along with a signature). This is mainly a benefit in microservices environments where some services might be completely stateless. In most Django projects you'd be hitting the DB to access the user model anyway (by accessing request.user for example) thus nullifying the sole benefit of JWTs. The rest of JWTs' features are just drawbacks is that it's extremely extensible meaning it's more complex than most projects need, and easy to screw up their validation or issuance, either from your side or even from the JWT library developer's side, thus I don't recommend them.
If you need to provide an API to browser-based clients, serve your frontend from the same domain as the API and use Django's built-in session authentication. On the login API endpoint, initialize the user's session (which will send a cookie back to the browser) and the browser will automatically send the session cookie for subsequent requests without your frontend ever having to do anything - all it needs to do is check for 401s and handle those by displaying the login view. As a bonus, any server-side-rendered views will also work out of the box.
For mobile apps, technically cookie-based sessions can also work, but common practice is to use some kind of bearer token. In this case, DRF has a very simple "Token" auth mechanism that just stores tokens in the DB and is good enough to get started. I suggest however extending that and making those "Device" models instead where you also store some device-specific metadata such as the push notification token, that way if you delete the auth token you implicitly delete the push token too and don't end up sending pushes to devices that are no longer authorized (seen that in production multiple times sadly). This also gives you a way to display to the user all the devices logged into their account and revoke them.
For the logins themselves something I recommend is to actually make your login page always server-side-rendered even if you use a SPA or a mobile app and render that page in a webview/iframe. This will allow you to easily change the login requirements later on (such as responding to brute-force attacks by adding a captcha, etc) without having to update the clients at all. That way you can swiftly react to new & constantly evolving security threats.
Something I've been thinking about recently is that there should be a site that only takes the "good" products from amazon and displays them for different categories - maybe only the top 20 or so. If i'm looking for an oven mitt / phone case / whatever the amount of junk is overwhelming. I envision it would essentially be a semi-curated list of amazon items. Anyone know of something like this? It's on my backlist of projects to build if I can't find any substitutes.
This is nice in theory, but Amazon have a big problem right now where merchants have figured out how to reuse Amazon listing to game the review rating. You'll often find you're looking at a product but the reviews are talking about something completely different.
Here's an example - third product in the search results on Amazon.
Lazy Prices [1] is a paper I was looking at for a while (I work on a Quant Research team) that does this in a simple way and the authors had good results. It takes the changes in language in some key sections of financial fillings as signals.
Thanks for pointing to the paper, haven't read it yet.
But it sounds pretty interesting, especially if you correlate it with things like insider's tradings (both buy and sell) and of course whatever external events there may be.
Talking of quant, would you mind sharing what is your typical stack, especially non-proprietary, non-confidential, stuff such as languages and libraries.
I don't do anything low latency so its mostly the common scientific python stack -- numpy, pandas, sklearn, etc. Mostly linux, lots of Spark, some R and C#. We're really flexible on what we can use to solve our problems but the team has pretty much settled on python as both developers and analysts can work together more easily.
I agree, it seems like no help just to list a formula to memorize. If someone knows enough linear algebra to understand what the formula represents, they can do the derivation. This link [1] is a good one if anyone is interested.
I asked an AWS solutions architect about this specifically and they said they had no plans to implement it yet (that they knew of). I’d like to see someone offer this too
I've seen a few well know prop trading firms that are trading cryptocurrencies as well (Jump, DRW, probably others) and there doesn't seem to be too much talk of that. Goldman lends credibility and pretty much everyone has heard of them, so I get this is bigger news, but to me the more interesting news is in the trading firms.
Prop trading firms have no reason not to manipulate the cryptomarkets (it seems legal) and have the capital and knowledge to do so. Or am I missing something and manipulating these markets is illegal?
I think it might be more accurate to say the laws are not enforced now as the markets are not regulated. That's not the same as legal. I'm not sure market manipulation / insider trading laws are that specific as to which asset you are manipulating as to make crypto ok. There's some talk now about Coinbase insiders trading ahead on bitcoin cash and some debate as to whether they could be done for that. I'd be curious how that plays out.