The power of FORTH is that creating a variation tailored to your specific problem is almost effortless.
In most languages, building a compiler or interpreter is a major project. With FORTH, if you’re the only user, you can have something working in minutes.
I’ve lost count of how many times I’ve thrown together a simple stack-based interpreter. Whenever I need to encode non-trivial behavior in an app, I know I can spin up a quick FORTH and get it done.
It's often used in bootloader/firmware type stuff (or was) because it can be a bit higher level than assembly and not subject to the same types of vulnerabilities and crashes... but written in a few KB and even easily hand written in assembly. And often run pretty fast, depending on the architecture.
I did. And when I went open with it, I did find out that plenty of people whom I know had breakdowns too.
The hardest thing was to admit I needed help (used to be a high performer: I solve my own problems!). It was very late, and by then it took a couple years, therapy and medication to recover. My doctor was very optimistic though. These sort of breakdowns happens to be very typical, and he was convinced that I would do a full recovery. I was skeptical. He was right.
The second hardest thing for me was to understand that it wasn't my fault. Athletes break their muscles often due to overwork. Brains can also break. In both cases recovery requires time, and often some treatment.
>So HN, what has your experience been with depression?
I am sharing my own experience, the topic being a taboo does not help people who suffer.
I was also shit scared of medication, and too apprehensive to go to a medical professional... until it was too late and it all collapsed.
I visited 4 different emergency psychiatrists. Only one of them managed to identify the condition and treat me accordingly. His goal: keeping me alive for 18 months. It worked. I did need strong medication, and it had to be carefully adjusted several times. This kicked me out of depression in about six to eight months, but did not solve underlying issues. Once the emergency psychiatrist rated that I was in a stable route for recovery, I switched to a regular psychiatrist that basically checks my medication once every two months.
In parallel, I started weekly therapy with a psychologist. Also I tried two doctors and ended up with the second one. This one is needed to solve underlying issues. It took long time to get it really working, but it ended up great. It's been a year and I am still attending weekly therapy.
Its been 18 months since I started treatment. I am better now than I've been the last 10 years, but change did not come overnight, and I am still working on it with full energy. I wouldn't have made it so well without any off:
a) sport & good food & sleep.
b) amazing doctors.
c) medication.
> Have you tried the drugs?
Yes, quite a few actually.
> What worked or didn't?
What worked: medication + full trust in the doctor + lots of patience.
What it did not work: expecting to be back to "normal" fast.
> Have you been able to triumph without chemical assistance and what did that look like?
I really doubt I would have triumphed without chemical assistance.
> Is my utter terror of these drugs warranted or should I just bite the bullet and try them?
Terror is due to depression being taboo, and fear of the unknown. Medication is not to be underestimated. It has side effects, for me it was mostly feeling even less energy than normal. So I would not take them unless you have under close surveillance and you trust fully your doctor. On the other hand, they are very very effective. I am weaning-off medication slowly and I am experiencing no side effects.
I can relate. I just read this at 3am while trying to stop wondering what to do with my life. I had the childhood dream of being a professor, and after a short stint in industry, when someone gave me the chance to do a PhD I took it. Life got in the way and got my PhD at 36 years old. Nobody guided me, and did not optimize my publication schedule, so it is impossible to land a job in academia. Now I am postdoc 39, non hirable bc I'm old and my cv compares badly to anyone elses. No clue how to provide for my family.
Last month we helped clean the room of a colleague after he "passed away" after 10 years of post doc.
Another colleague got a job at 36 as a bare programmer after a very successful PhD where he wrote books and was invited to conferences. At 40 he rage quit this job after a bout of frustration and we haven't heard from him since.
Luckily they didn't have kids.
Wouldn't I have kids I would definitely follow their path.
I can understand it too - I finished my PhD with 34. In the end, I could have seen it coming: my supervisor had 0 interest in supervising, so it took a huge mental strain on me.
I've also had immense luck - I've been a computer nerd for a very long time and I was programming a lot in my field (neuroscience). So all it took was a word from one guy I knew at a big software company and I was hired. I also have to say, that a PhD, or better said, a Dr. in my country still means something, especially if you have customer contact.
But I've seen things... bright people in their 40s who have to drop out in their 40s to be hired as a labor assistent at pharma. Associate professors who would run out of money and that's the end of the career. Doing a PhD was easily the dumbest and most risky decision of my life and I was extremely lucky to get away with only some mental scars. The only positive things I can think is the friendship with other PhD students (because you went through hell together!) and the confidence in my abitity to process and dissect huge piles of information. In the end, the latter is sole reason companies are willing to give you a shot. Dont undervalue it and sell it accordingly. If you have a PhD you most likely are very persistant and very capable of self-learning.
I'm curious, what advice would you give to a person that is in that position you were when the opportunity of PhD was given? (and have a childhood dream of being a professor)
I met one person at my Master's degree lab who went on to PhD, he said his childhood dream was to be a professor (and his father was a professor). I also have a father that is a professor (and have been very curious about PhD - although that window is getting narrower as I'm getting closer to 30:s) but he have multiple times told me that "it's not worth it", "it's a waste of time, the opportunity cost lost is too high", yet in my field I often see job post that requires a PhD, so I get mixed signals.
Doing a PhD is definitely not the same thing as becoming an academic. If anything, the PhD is a direct career boost: it's a recognised, high-level qualification that is known the world over (and, incidentally, if you go outside the US, you can probably get it done an awful lot quicker...). If you love the subject -- and you have to love the subject -- do it. It's like being paid to play.
Going from PhD to Prof though, is a difficult, unlikely path of awkward postings: for every 100 PhDs expect ~1 successful academic. Oh, and once you are a professor, expect a salary....less than that you'd get in a starting job straight out of your PhD.
Thank you for the reply :) I'll add it to my note of "pros/cons & advice to take in to account before important decision" that I've started to log after my repeated mistakes..!
> I'm curious, what advice would you give to a person that is in that position you were when the opportunity of PhD was given? (and have a childhood dream of being a professor)
Bluntly, if your goal is to become a professor, don't. With 99.99% certainty, you will not become a professor and the opportunity cost is extremely high.
Almost nobody with a PhD who dreams of being a professor ever actually gets to be. Most quit or only ever become adjuncts, and, in the US at least, adjuncts earn less than minimum wage and get zero respect from anyone. Most likely you will be abused by institution after institution who will keep telling you how important your dream is while stringing you along and paying you next to nothing. Or you will quit. Or you will have a mental breakdown.
If you can see yourself being happy doing literally anything else, do literally anything else. If you can't see yourself being happy doing literally anything else, spend some more time thinking about it.
If you're independently wealthy and don't really need to succeed at the goal to live a happy life of luxury, then definitely go for it.
>Another colleague got a job at 36 as a bare programmer after a very successful PhD where he wrote books and was invited to conferences.
I can relate, as this basically describes me - though I've been lucky enough to find bare programing jobs that I quite like doing.
One of the toughest things about leaving academia is discovering that (i) no-one who doesn't have a PhD has any clue what a PhD is and (ii) everyone who has a PhD has a healthy disregard for the intellectual capacities of their fellow doctors.
If you have any resources on how to manage anxiety please bring them on. My old son has a mild development impairment, I am the sole bread earner, and we have no family support (we live abroad).
Honestly, one of the biggest tools I think is in-person school. Which lets the kids get their own space and cultivate their own lives away from parents in a supportive place. Sadly that's been taken from us until Covid is resolved. It multiplies the anxiety quite a bit for everyone: kids & parents. It's hard for us not to be overbearing to our 9 year old about virtual school. It's hard on him, and he's frustrated. It's hard on us because it's time consuming...
Other than that, I strongly believe in paying well for good childcare and babysitting so you can take breaks.
I also believe in sharing your passions with your kids, and focusing on quality time rather than quantity or things that don't interest either of you. This can be surprising. My kids love camping, which surprised me. I love camping - my wife doesn't. My kids are meh on coding, they do it, but it doesn't grab them like it grabs me...
Focus your career choices carefully as your time on them is limited. I have found having kids, due to time crunch, has caused me to really focus more on what I really care about from work. Don't let others take advantage of you to do BS work, focus on what you want to do on a day-to-day basis that's fulfilling for you if you're able. I think sometimes parents fall into the trap of only focusing on $$ when really it's a good idea to focus on how much value you yourself are getting out of your more limited time investment
* A dose of magnesium l-threonate in the morning and before bed
* A dose of magnesium glycinate in the morning and before bed
* A dose of black seed oil in the morning and at night
If you did a hair mineral analysis test, I assure you, you would most likely be magnesium deficient. The body needs magnesium in order to deal with stress. Anxiety is stress.
The deficiency wreaks havoc on the nervous system. Take a HMAT from Analytical Research Labs if you want to be 100%. You'll have to find a practitioner.
I've done years of therapy. But getting my basic nutrients in order, made an immediate impact and has had a much bigger difference in treating my anxiety.
Talk to somebody. A licensed professional would be best, but a significant other or trusted friend will work too. You have to get your thoughts out into the real world so you can process them. Journaling helps with this. Also, focus on your well-being. Make sure you're getting enough exercise, take up yoga, find a productive hobby, etc. If you have a significant other, make sure you stay communicative. As a parent it's easy to sacrifice your well-being for the sake of the family, but the family will work best if you're healthy.
Hang in there! It's absolutely worth all the effort, and things will likely turn out far better than you may think.
I strongly encourage you to talk to a therapist or counselor of some sort - even after just a few chats they can arm you with some really great tools.
For managing the anxiety of life generally and parenting specifically, religion has been very helpful for me. If you're not into that, maybe find something that strengthens the big picture & long term view of things.
I'm tying (unsuccessfully) to get sleep, and I'll try magnesium supplements too. I already spoke about this but never got solutions, and at this point it feels almost even worse every time.
While the roots have been known for a long time, my impression is that the key paper that started this line of thought was Marco Cuturi's NIPS 2013 paper "Sinkhorn Distances", which is, IMHO, a very nice read.
Certainly I may be missing something, but it seems like the advance in this series of papers is that they figured out a way to calculate a differentiable solution to the sorting problem quickly, whereas it was already known that the a differentiable solution already existed, no?
IMHO the coronavirus was used as an excuse to kill a fair that was already producing negative ROI.
Due to the size and fame of the fair, most big companies were forced to participate to "look better" than their competitors, even if they didn't have any great novelty to show there: it was a marketing arms race where each company had to invest more than the previous year.
Nice video, however instead of riding the hype train of arxiv, could we wait until peer review analyzes the paper?
If someone other than Hinton presented a YADLA (Yet Another Deep Learning Architecture) that does not achieve state of the art level of performance in the basic datasets, it would not be very well received.
> Over the last three years at Google I have put a huge amount of work into trying to get an impressive result with capsule-based neural networks. I haven't yet succeeded. That's the problem with basic research. There is no guarantee that ideas will work even if they seem very promising. Probably the best results so far are in Tijmen Tieleman's PhD thesis. But it took 17 years after Terry Sejnowski and I invented the Boltzmann machine learning algorithm before I found a version of it that worked efficiently. If you really believe in an idea you just have to keep trying.
Most of the deep learning papers published are just exploring and incrementally building upon the ideas 'Canadian Mafia' (Hinton, LeCun and Bengio) discovered years ago. At some point this 'idea space' is explored and understood and we hit the wall just like before. Let's hope that people doing basic research can find new breakthroughs in less than 17 years.
I'm not saying to discard this research. I'm suggesting to wait until it is peer-reviewed and published before jumping on it.
To me, the capsule concept seems reasonable, and I have my personal opinion about its strengths and flaws. But my opinion hardly matters.
I expect peer reviewers from NIPS to have a better understanding that I have, and I trust them to filter and clean this idea, instead of trusting the research just because of the name that signs the paper.
To me, although it has its flaws, the _double-blind_ _peer-reviewed_ processes is important.
Wait for what exactly? Talking about this? Implementing and testing it? Experimenting with it, trying to replicate results, trying to extend the ideas? Etc? I'd argue that all of this is peer-review, if not in the traditional / formal sense. And CS (especially ML/AI) seems to be moving in this direction over the past few years and that's not necessarily a bad thing.
Also, keep in mind that peer-review or not, if you look at this from a Bayesian point-of-view, the prior on this work being important / meaningful is going to be pretty high for a lot of people - just because it is Hinton. And that's a reasonable position given his past work.
I feel like you're missing my point. Bayesian reasoning of that nature is totally reasonable and isn't something to be avoided just for the sake of avoiding it. What is is, is useful as a guide for where to direct energy and focus. And what it is is faster than sitting around playing with your pud waiting for review for a journal submission.
Again, what's going on now is a form of peer-review. Double blind? No, but that's not really relevant in this context anyway.
ML is really more of an empirical field in this day and age and people are going to read pre-prints on ArXiv, and use various Bayesian weighting schemes to decide what to direct time and energy towards. This process complements, not replaces, the kind of formal peer review you're demanding. There will still be plenty of room, and time, for that stuff, but there's no real reason to wait for all that to happen before starting to look into something.
Is peer review generally done double blind? Who in the field has not heard of hinton s capsule proposal that would be a blind reviewer? Hell I'm not even in the field and I've heard of it.
It was already reviewed when they published the arxiv preprint, so there would be no bias in this paper in particular. And as now, everyone and his/her cat has heard about capsules, you'd expect that some others than Hinton et al might write about capsules, so you shouldn't be able to say "hey, it's Hinton because capsules!".
I think if you have good natural instincts about artificial intelligence you realize that Hinton really is the god in this space. If you don't, you get distracted by LeCunn and Schmidhuber and whatever lesser minds. The capsule theory here is maybe not the best implementation, but the intuition behind it is still leading the way. The undifferentiated mass of neurons is not how evolution has solved our problems. 3d geometry is intrinsic to the low level design. It shouldn't be learned. It should be assumed.
I reject your generic devotion to process. The real leadership and the process are far different. The process of peer review might do a good job of rejecting bad ideas, but it does a lousy job of accepting revolutionary ideas. I bet you don't understand the difference.
While I agree wholeheartedly that peer review is fundamentally a risk-averse, conservative process, I bristled when I read the statement that LeCun as a "lesser mind." That's quite rude, and uncalled for.
I agree, it still needs a strong use case. IMHO, new network architectures are generally overhyped. Even AlexNet performs wonderfully well in most problems once you add nice initialization and batch normalization.
In most languages, building a compiler or interpreter is a major project. With FORTH, if you’re the only user, you can have something working in minutes.
I’ve lost count of how many times I’ve thrown together a simple stack-based interpreter. Whenever I need to encode non-trivial behavior in an app, I know I can spin up a quick FORTH and get it done.