if i need a toposort in my proprietary task scheduling system, or fisher–yates shuffle in my proprietary ranking system, what do you recommend? this isn't hypothetical - i've had these things come up in the last half at work.
Hi! Please refer to the HN guidelines, especially the following section:
> Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community.
I don't think anyone is saying never implement an algorithm. There's a time and place. I've seen code bases where these algorithms had ten different implementations, some of the with subtle bugs, some of them with some performance issues. Your code will also need to be maintained down the line. There's more to software engineering than writing toposort.
Obviously(?) my Boost proposal was made in jest. Anyways, maybe I'm a little bit jealous that you get to do this all the time ;) I just hope I'm not the one that has to maintain it down the road...
If you're doing things in C++ you're already doing very specialized stuff, nobody can advise without actually knowing the app well. Especially if you have an app that rubs in all those different situations.
But... boost seems pretty common. Everyone seems to complain about it, but if it replaces a whole pile of hand written things, it looks like it could really bring some consistency to a project. I've never done serious C++ outside of embedded though, so I don't have Boost experience.
In a dynamic language like JS, you'd search for "Graph theory algorithms library" and probably find one with ten million downloads, investigate the maintainer a bit, and use it.
> In a dynamic language like JS, you'd search for "Graph theory algorithms library" and probably find one with ten million downloads, investigate the maintainer a bit, and use it.
Left-pad was as far as I know written by one person. Had they used underscore or lodash or any of those bazillions of utility libraries out there, that have real code review, and are actually making money, this would be way less likely.
That's why I'd look for a graph algorithms library first, not a topo sort specifically, I'd be looking for larger one size fits all libs likely to be maintained by a larger group and have more eyeballs on them.
I have said this before: it's group therapy for entitled devs that feel they're entitled to oversized salaries just because they have tenure. Young grads accept the process as is. People that are either happy with their comp or comfortable putting in the work do not lament the process - they just accept it and proceed. Only devs that have been doing it a while, whose salaries have plateaued, that either never learned data structures or can't muster the will to review, bemoan the LC grind.
The most utterly absurd instantiation is the dev that's 100% sure that FAANG is going to fail soon because they're passing on people like him/her that are just so talented that they don't need to prove it with any assessment. There's also the MGTOW-esque subgroup that eschews such interviews and will proclaim it proudly, along with their salary.
Personally I'd be embarrassed if I was making a less than obscene dev salary, which is still upper middle class anywhere in the world, and all the while complaining that I deserve more for no added effort.
What's funny is there are other professions where the same sort of multimodal distribution in compensation exists (big law vs everyone else, competitive specialties vs general practice in medicine, IB and PE vs everyone else in finance) and I have never seen members of those communities complain as much as developers do.
Edit:
I have a friend that's trying to break into FAANG (as a DS) who I've been coaching. He has a stats PhD and basically no software training whatsoever, outside of R. Since DS loops still have LC questions it's been very hard for him. I have never heard him once express feelings of resentment over the process - he fails an interview and just goes back to practicing. He's also a first gen immigrant from a very poor part of the middle east. Admittedly it's hard to assign/attribute his perseverance to any particular thing but naive intuition dictates that some of it must be his humility - something that software as an industry seriously lacks.
> Only devs that have been doing it a while, whose salaries have plateaued, that either never learned data structures or can't muster the will to review, bemoan the LC grind.
As someone who mostly agrees with you, I still bemoan the LC grind.
This is mostly because I think the software market has grown to the point where there’s a distinct market for “plumbers” vs “engineers”. I consider myself a “plumber” who understands how to piece together various frameworks and languages in the most efficient matter, but “engineer are the one who actually invent the new frameworks and who need to actually understand DSA to accomplish their jobs. It feels like distinct skills since I’ve tackled what I consider engineering problems multiple times and failed but I keep getting rewarded with higher and higher salaries for doing what I consider plumbing.
Unfortunately there seems to be an ego component and no company will admit that they need software “plumbers” more than they need software “engineers” or it makes them/their employees look bad.
I just wish I could be ranked and rewarded off my “plumbing” skills and only ranked on my “engineering” skills if I was looking for a career change
From friends and acquaintances in other industries with similar income distributions, "knock out 4-to-6 programming problems in a short amount of time" is one of the least-arbitrary and most accessible ways to jump from one group to the more highly paid one. Not to mention that if you can knock out those problems, you may not even be required to have a degree or other formal credentials.
That's not even getting into how hilariously unrealistic "trial periods", for instance, are. Why am I going to leave a job for just a trial period without any more thorough attempt to check fit before hand? Why would I invest that much time in side work if I'm doing it before quitting, when I could get a FAANG offer instead for single-digit-days-amount of brushing up?
>least-arbitrary and most accessible ways to jump from one group to the more highly paid one.
exactly. i mean like try to apply to IB at a BB and see if anyone even skims your resume without an ivy+ on it. they can't fathom how good they have it because they've never done anything else (which is, of course, what breeds entitlement).
For the F in that list, at least, definitely. Though that was because they used a different metric to sift/choose me for an interview (and a decade of experience) despite my lack of tertiary education.
How did you get the interview? They must get a lot of junk applications, what did you write to make them actually look at you instead of direct to trash, and then decide to interview?
easiest way to shortcircuit the process is to reach out to someone through linked in and ask for a referral. if you're really who you claim to be in your profile here then you shouldn't have any trouble getting an interview.
I mean, recruiters get paid sizable commissions for each hire they make, and they really dont care where you went to school as long as they can sell you to the company. If you can sell yourself to a recruiter (low bar) then you can get an interview.
of course they need credentials but undergrad at decent state school is much more widely available credential than ivy+. hell FB hires high school grads into their FBU program (for the gap summer between high school and uni).
> I have never seen members of those communities complain as much as developers do.
Other professions have guilds (not called that, but effectively that) who qualify membership and enforce various rules and standards on their members.
Maybe one day we'll have a Guild of Software Developers that provides this service and we won't need to deal with shitty interview tests. But until that day, maybe we shouldn't be comparing ourselves to professions that do have this and therefore don't face the same problems.
I think you are referring to Professional Orders (https://en.wikipedia.org/wiki/Professional_order) and, depending on where you live, there may actually be one.
In my country, computer engineers have a professional order they can join after graduation, in the same exact way all the other kinds of engineers they've studied with do. Unlike some other engineering fields, the order does not enforce nor require active membership in any order to work.
There've been a few attempts to get this going. So far none of them have worked (see also: any kind of union specifically for tech workers).
The other professions have a "closed shop" - you cannot work in that profession without joining that association.
e.g. in the Anglosphere, you cannot practice as a lawyer without being a member of the relevant Bar Association, and to join that you must "pass the bar" which is an exam.
If we had this in Dev, then we wouldn't be able to write code for money until/unless we were a member of the Software Dev Association, and they wouldn't accept us until/unless we'd passed a rigorous exam (passing the bar is something that takes years of study) to prove that we could code. Then we wouldn't be facing ridiculous "but can you actually code?" tests during interviews.
But creating that closed shop has always failed (so far). Employers don't want it, and new coders don't want it. It's only us old hands who expect to be grandfathered into it that kinda like the idea.
When the status quo is objectively stupid, it is right and just to complain about it. In real life, the ones who will do so are mostly just the ones who it hurts. So what do you want to happen here, if it's not just for devs to shut up about the insanity that's making them miserable?
While we might be able to argue that alternative interview strategies could be better under some set of assumptions, the status quo is clearly not stupid. These companies are hiring thousands of people and pulling in stupid amounts of money. The system works. Could it be better? Maybe, maybe not. But it clearly does work.
That's funny with your use of L I figured you'd be at Google and that that tracks with my experience at FB but no one calls it meta except people at meta so I guess I'm just saying that tracks lol
>you REALLY CAN do what you want! Real freedom; just be useful.
exactly. high-level ICs at FAANG set their own schedules/roadmaps and salaries as long as they pushing expanding the forefront of an area that brings value. the truth is academia is a shelter for a lot of people who can't do useful things.
> the truth is academia is a shelter for a lot of people who can't do useful things.
And that's a good thing, because often times it's not obvious immediately how a particular thing is useful or brings value. Sometimes a particular area of inquiry doesn't appear promising until a subsequent invention many years later.
For example, driverless cars were incubated in academia. Now corporations are using those same techniques. A lot of researchers crashed a lot of cars before a winning combo was discovered. You don't get positive results without a ton of failures and not-useful ideas. It's worth noting that industry was not the one footing the bill here to get all of these negative results. There were no high-level ICs at Google doing this research when it was being done at CMU. Google only started doing it only once the the techniques were proven at the DARPA grand challenges.
>Sometimes a particular area of inquiry doesn't appear promising until a subsequent invention many years later.
this could be the tagline of every single application to NSF. i'm not sure if you're in academia but there are hordes of projects that are funded that are clearly useless.
>For example, driverless cars were incubated in academia.
are you really claiming that driverless cars weren't immediately obvious as valuable? just because CMU got to it first doesn't mean industry wouldn't have gotten to it.
> hordes of projects that are funded that are clearly useless
I don't understand what your point is. It's not like industry is immune to funding ideas that are clearly useless (Juicero) or even fraudulent (Theranos). If you have a way to fund only good ideas, I think that would be quite a breakthrough. I mean, look around you: this community is set up around a good-idea selection engine that itself struggles to consistently identify good ideas. Bad ideas are a part of getting to good ideas. If you really had a 100% fool-proof way if identifying bad/good ideas at the pre-funding stage you wouldn't be here talking about it -- you'd be using it to make a ton of money.
Also, I would challenge you to point to a NSF funded idea that is 100% clearly, objectively useless. Something so clearly has no plausible utility whatsoever. I think this would be very hard to do.
> are you really claiming that driverless cars weren't immediately obvious as valuable?
It's true that industry could have but they didn't, and that's the point. It only became attractive to industry after the public sector dumped a massive amount of R&D into it and basically proved how to do it to industry, and that's not a coincidence. Before the the DARPA grand challenges it was absolutely a question as to how viable the idea of driverless cars were. This story is not an uncommon thing.
No it's I that don't understand your implicit point that we should let academics fiddle. Is it just because there's some grand narrative around science being objectively and absolutely for the betterment of mankind? The ideological devotion to science as some kind of monastic pursuit blinds you to the very real facts (on the ground) about how much funding is wasted on one off papers, one off projects, whole conferences devoted to areas that will never improve anyone's life or generate absolutely any return on investment.
>Also, I would challenge you to point to a NSF funded idea that is 100% clearly, objectively useless
It's very easy: pick any nsf grant going back as far as you'd like (such that there's enough lead time) and look at the number of citations on the papers generated from the grant. I don't even have to do this because everyone knows that probably 50% of all papers get absolutely zero citations (https://www.science.org/content/blog-post/not-so-many-uncite...).
>It's true that industry could have but they didn't, and that's the point
Again I have no clue what you're claiming here. You're trying to make some kind of case for academic science shouldering the burden of some fraction of fundamental (pie in the sky) research by citing self-driving cars which is positively laughable as an example of such research.
> whole conferences devoted to areas... pick any nsf grant... and look at the number of citations
Do you see the contradiction here? I don't think bibliometrics are useful for measuring utility.
> I don't understand your implicit point that we should let academics fiddle
I'm actually totally okay with letting academics fiddle. They are super cheap and you're mostly just paying them for small amounts of their time.
But, it should be the faculty who are allowed to fiddle. They shouldn't be given resources to direct other people's fiddling time. In particular, I think we should massively reform the graduate student system to invert the power relationship between faculty and students on any highly exploratory projects.
Specifically, for any grant whose purpose is "fundamental science" and/or training (e.g., ALL NSF money as a starter):
1. the agency funds students instead of faculty. So 100% of the money that goes to graduate students on NSF grants should be redirected to a GRFP-like funding model. This means that NSF grants to faculty should only fund PI summer salaries & shared department resources. Never students. Want a student? Recruit them to collaborate with you.
2. NSF should put a hard upper bound on the number of funded teaching hours permissible or required and funded through any sort of stipend.
NB: students can still teach more hours! But then they will be normal W2 employees who are paid prevailing rates, are included in faculty+staff retirement/pension/benefits, get FICA benefits, etc. The point: if your uni takes a single dollar from NSF, then student stipends can only be actual stipends, not back doors for tax-advantaged ad junct labor that excludes universities from paying FICA taxes on behalf of their teaching staff (who happen to be grad students).
MIT is one step ahead of you, they already ding faculty with NSF GRF students extra overhead because ostensibly the NSF fellow is there for MIT's brand and not the professor's research.
Yeah, places like MIT are where I got the model. Seeing how my peers were treating at places that are not like MIT is how I got the motivation to care.
Tippy-top programs in any particular field already operate in a way that treats grad students more like students than itinerant labor.
It's the other 99% of institutions that are gutting scientific human capital in the name of empowering incompetent middle management. (Professors are middle management. Most of them are incompetent at that job.)
>Do you see the contradiction here? I don't think bibliometrics are useful for measuring utility.
that's not a contradiction that's like just your opinion man. it's also the no true scotsman fallacy on your part.
>we should massively
>NSF grants should be redirected to a GRFP-like funding model
>NSF should put a hard upper bound
cool when you replace panchanathan you can institute all of the policies and only then will academia be more egalitarian and possibly more useful to society as a whole. until then you're just playing bait and switch wrt the current state of things.
> that's not a contradiction that's like just your opinion man. it's also the no true scotsman fallacy on your part.
I don't follow.
Useless work gets cited all the time. I'm not suggesting that there should be less accountability to be useful to society. I'm just pointing out that bibliometrics is a particularly bad way of measuring utility.
> until then you're just playing bait and switch wrt the current state of things.
Again, I don't follow. How am I playing bait and switch? I don't work in academia.
> replace panchanathan
Fortunately, that's not necessary. You don't put pressure on agencies like NSF by joining the civil service. Even high-ranking civil servants are... well, servants. You put pressure on federal agencies by having tons of money and free time. Which many lapsed academics in CS have in spades ;-)
it depends on what your skillset really is and what areas you're interested in for your phd.
pure theory (finite automata, complexity, computational geometry, optimization)? yea you're probably locked out by virtue of competing against students that have the relevant theory fresher in their heads. note i didn't list ML there because i personally don't consider ML theory heavy (no matter what the wannbes will claim wrt to TDA or whatever).
basically in any other area, if you have true hard skills (e.g., deep systems knowledge, networks, graphics) then you're competitive as long as you have a decent ugrad GPA and can get a decent GRE and some decent letters of rec. the path looks like reaching out to a potential advisor and starting a conversation with them about whatever research of theirs you find interesting.
Man some hn posts bring the cringiest comments; please do tell us which school has functional analysis as a "typical" topic for juniors taking real analysis. Even if you're getting a second helping of real analysis by then, you're probably looking at Lebesgue integration on R and such, rather than general topological spaces.
I'll never understand why some people try to flex on an anonymous forum.
This is from PMATH 351 at University of Waterloo. Every pure math student takes the course in 3rd year. Lebesgue integration isn't covered until 4th year, though it is not restricted to R at that point.
I'm sorry you think my comment was intended to be a "flex". I was trying to make a point about university mathematics which is this: at university level students should be going beyond solving simple computational problems. Synthesizing a proof requires a higher level of understanding than the application of a standard problem-solving technique. See Bloom's taxonomy [1] for details.
How does it remain impressive? Is this just deference to the institutions of the authors? Or the general aura around AI/ML/DL?
A couple of years ago when FB trained some kind of architecture to compute derivatives by presenting it with problem,answer pairs I called it out as "just wrote memorization" and got promptly downvoted. Now here we have people using GPT3 (a network widely recognized to be just memorizing) for a similar task and people are still impressed? Like you said, their only insight was to augment the data and translate to a computable (parsable) form.
I'm guessing people don't understand what a papermill is, especially at these schools that have access to the gobs of compute that you need to make these projects go through. It's junk science - probably not reproducible, definitely not extensible, doesn't transfer out of the sample set - purely for the sake of incrementing publication count for all involved (cf. the number of authors on the paper). And before people label me a hater: I speak from experience as someone at one of these types of schools that has their name on several of the same sorts of "turn the crank" papers.
They probably weren't judging the unique contribution here but rather the system as a whole. If you set out to solve this from scratch it would be very difficult. It's just recognizing what has been achieved not just by this group but the entire field which made this possible. This isn't an academic conference paper review session.
This is so charitable thank you. But please tell me are you this charitable when some derivative node package gets published? Or python library? Or when the 100th data science startup is announced?
this is like some kind of reverse occam's razor argument but i feel like your point is blunted by the fact that there are lots of companies making tons of money off of viagra derivatives.
I'm bookmarking this and showing it to everyone that claims Julia is the next big thing. Kudos for really exercising the language despite all of these painful shortcomings (you must be a very patient person).
