Python is the language of choice for many ML teams — it’s flexible, expressive, and has the right ergonomics for real-time workflows. But it’s also slow.
To make Python faster, I built a Symbolic Python Interpreter to make Chalk’s “Python resolvers” into optimized Velox-native expressions — enabling high-performance execution without sacrificing developer experience.
This post explains the problem I encountered, why it’s so challenging, and how we solved for it.
Arbitrary example: If you're using SHA256 you're emitting 256bit hashes. Hopefully the distribution of SHA256 outputs is indistinguishable from random (or close enough) or else we've got big problems in other domains.
If you're drawing at random from a pool of size k you need approximately sqrt(k) draws until you reach a ~50% chance of a collision[0].
With 256 bits, there are 2^256 possibilities, so following the rule-of-thumb you'd need 2^128 draws until you had a 50% chance of a collision.
2^128 > # of atoms in the universe.
If you adjust your risk tolerance you'll have different numbers come out, but the chance of a collision in any realistic scenario is negligible.
If you're using typescript and emacs, https://github.com/ananthakumaran/tide is a very good integration with the built-in language server for semantic auto-completion and "jump-to" support. I use it every day on large projects.
I think that vscode uses the typescript language server behind-the-scenes for plain javascript analysis.
I attempted to use clang-format on a mixed typescript/tsx codebase and found that it mangled jsx expressions. This wasn't surprising but I suspect that typescript users probably overlap heavily with react users so it's not a great solution as is.
It's not hard to automate with raw gulp or webpack, but it does add a significant amount of time to compilation. It roughly doubled our incremental compile times. My team decided it wasn't worth it, but it's possible that a cleverer person could decrease the overhead.
I ask because I'm debating between trialing TypeScript or Elm for a personal project (I primarily work in Rust and Go on the backend/infrastructure), and iteration times are important when trying to get a frontend UI "just right".
You could just use TypeScripts ES6 output directly during development, using a browser that supports ES6 natively, then add Babel to the build chain for public builds that need to work on older browsers.
TypeScript has dealt with this in the past by adapting to the introduction of `<Foo>` from jsx, which clashed with typescript's casting operator. If this ends up being a problem I'm sure that the TypeScript team will just pick a different syntax.
In my experience, the overhead caused by TypeScript is far outweighed by benefits in tooling and confidence that my code works (mostly) after refactors.
Lodash has many features that underscore does not have. Also, I am told that Lodash has been implemented in such a way that functions are largely independent of each other so that you can build a custom lodash package that only includes the subset that you use.
In practice, I'm not sure how truly valuable this is... it's far easier to just accept it as a whole, and with min/merge versions in the browser it's not so big at all.
It's not a macro framework like jQuery... the kitchen sink isn't in there... though, I do think that probably the string templating could/should become a sub-module... the rest belong together more.
I TA for this class. The class is optional for non-engineering majors but has enrollment >= 700 people per quarter. So many people at Stanford take the class that it's a bit of a cultural phenomenon.
There are three "intro" programming classes at Stanford. CS106A is "intro to java", CS106B is "intro to C++", CS106X is CS106A + CS106B + a bit more in one quarter. People who already know how to program are encouraged to take one of the latter two, although there is always a contingent of people with /some/ background in 106A.
For non-engineering people, Java seems like a slightly odd choice, is it not? So much overhead in learning, plus overhead if they did want to integrate coding into their lives in a light way. Why not Racket or a scripting language? As much as it pains me, even JS might be of better use.
Or is it just the case that these people just wanna take one course from the "real" CS for a non-coding reason?
Because the assignments, section material, grading infrastructure, handouts, teacher training material, and institutional knowledge for this class are all built around Java when the class switched over in the early 2000s. In the past, couple years the process of changing just one assignment out of the six or seven assignments was difficult for everyone involved (teaching staff and students). Changing the entire class would require monumental effort, and it would have to be driven by a majority contingent of the half dozen professors / lecturers involved in the introductory CS community at Stanford.
Given that the class has been successful at teaching programming methodology to tens of thousands of beginner students over the past decade there's no pressing need to restructure the entire course, even though everyone does acknowledge that Python or some other language could be better for new students. It's basically like any other large software project, yeah it could be rewritten from scratch and modernized to make small gains, but if it ain't broke, why fix it, especially when the professors and TAs have other projects that have far more interest in, like this paper.
Also one thing that many people don't know is that CS106A actually changes the language environment as well as providing a modified version of Eclipse to remove some of the less useful overhead in Java (for example removing main, and the beginner confusion surrounding keyword static, arrays and arguments). Also there's a much simpler introductory class called CS105/101 that uses Python.
Source: I taught sections and managed the TAs for the introductory classes over the course of four years while I was a student at Stanford.
The analytical gradient is actually a general tool used in multivariable calculus. It's vector-valued, in that if you take the gradient of a function of three variables: f(x, y, z), then you'll get a 3-vector back. Vectors are defined by two characteristics: a direction and a magnitude. The gradient's direction is the direction of greatest increase and the magnitude of the gradient is the instantaneous rate of change.
The gradient is being put to work here in order to optimize a function using a process called gradient ascent. Intuitively it makes sense that in order to optimize your function you'd want to "move" in the direction of greatest increase at every step. However, the size of the step that you take is tricky. As you point out, in this case, we can increase the objective function value more if we double the step size. However, you're not actually doubling /the gradient/.
If you look at the expression that you wrote you should see:
x + 2 * dx * step, y * 2 * dy * step.
What you've done is double the step-multiplier, not the gradient itself (<dx, dy>). This means that the optimization process jumps further at each step. However, the step size that OP chose is somewhat arbitrary to begin with, so it's not clear why any particular choice would be better or worse. The reason that the step size matters is that if your step size is too large your optimization process might jump over a minimum location or something similarly bad -- there are pathological functions like the Rosenbrock Function [1] which are notoriously hard to optimize with gradient ascent. In practice, you'll often choose your step size more intelligently based on a few different tests, or vary it as the optimization process progresses.
In this particular instance, the surface that you're trying to optimize is pretty simple so basically any step value will do the trick. It may take a different number of steps in order to compute the global maximum, but most reasonable step sizes will get there.
To make Python faster, I built a Symbolic Python Interpreter to make Chalk’s “Python resolvers” into optimized Velox-native expressions — enabling high-performance execution without sacrificing developer experience.
This post explains the problem I encountered, why it’s so challenging, and how we solved for it.