Author of the presentation here. While you are right that sharding is an incredible tool that makes our lives much easier, not everything in our system is/can be sharded.
Also, we do handle more than a billion requests/day, that was just for giving the title a nice ring.
Author of the talk here. Thank you, you made my day by sharing my presentation. :)
One thing I would like to add after reading some of the comments here. The "1 billion requests/day" is actually an understatement so the title has a nicer ring. Last time I checked we were around 2B -and that's according the most conservative approximation-. Those are requests that hit the application, excluding the CDN caching layer.
For Zendesk, what are the business consequences of using a monolith? Is this a mono- or multi-repos? Are tests simpler? Does the system require lots of special-purpose tooling?
Spaniard here. Over 50k is definitely possible, particularly now with so many remote opportunities available. Unfortunately, those positions are highly-coveted, so prepare for long interviews with multiple stages.
I think becoming a software engineer is totally possible for you, you are only in your mid-30s. However, I think you need to like the field, at least a bit. Do you enjoy it?
I do think so, yeah! I only seems a vast field and a daunting task to build stuff that I like. I can do some front-end stuff with no so much complexity all by myself, but I have to put quite a lot of work because I forget some syntax and I spend time debugging.
When I see experienced people in videos etc they do everything so much quicker. I know this is normal but it feels that it will take me a lot of time to get there.
I'm currently in a self-paced bootcamp (open-bootcamp) and so far I'm happy. I went with beginner knowledge on Git, CSS, JS and Python, and I improved a lot of my OOP understanding which was very lacking, and their full-stack path touches also Java. I plan to take some compiled languages too, purely out of interest.
They also have projects, although I'm not sure how they structure that, but I hope that takes me up to speed to be productive.
I'm sorry, but exactly which "medium-sized European city" has more expensive housing than SF?
I'll be moving back to Europe this year, and I find your comment quite funny, given that I am moving to a city with significantly cheaper housing and better weather than where I live now (NYC).
I doubt that it can be taken seriously as a medical question, as some of the American "races" don't cluster genetically. Hispanic is probably the worst offender, but Asian is pretty bad too.
Hispanic isn't a race according to the Census Bureau.
> The United States Census Bureau uses Hispanic or Latino to refer to a person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture or origin regardless of race
The way it's used in popular imagination is as a "race" though. My mom doesn't see Pope Francis as hispanic because he isn't mestizo.
I am a "Hispanic" that lives in the US. I am also European. In my experience what qualifies as an Hispanic varies incredibly from institution to institution, from person to person. A few months ago I was admitted briefly to the hospital, my race (not ethnicity) was marked as "Hispanic". Other times, in other places, I have been assumed to be "white".
Also, you wouldn't believe how much better I am treated when I am perceived as "white".
I read that book when I arrived to the US: really interesting. IMO, the activists that pushed the idea of the "Hispanic" race/ethnicity into the American mindset made a huge mistake. Racializing yourself is a bad thing, even if the government gives you a few pennies in exchange.
Well, I think you’ve missed important context here. Darker skinned Latinos didn’t have a choice to not be racialized; “Hispanic” was introduced as an alternative to the prevailing practice of just using the Spanish-speaking country with the most local expats as an informal race designator. White Latinos could in principle have defected and chosen to identify as just white, but I’m glad we didn’t.
I think that the racialization of Hispanics as a group has been pretty bad for "non-white" Latinos too.
1. You are right that someone that doesn't visually appear "white" will be racialized in the US. However, the magnitude of this racialization can differ a lot. Incessantly repeating that "Hispanics" are a very consistent group of people makes the "otherization" of all Hispanics worse, including for "non-white" ones.
2. Including people of European descent in a group that is going to benefit from affirmative action opens an obvious loophole. Unconscious bias will provoke that the opportunities created by AA end up with people who look/are white, not to Latinos "of color". Hollywood is particularly terrible on that: with most of Hispanic actors looking European (Ana de Armas is a recent example) or coming directly from Europe. I guess they can't find non-white Latinos in LA. ¯\_(ツ)_/¯
3. Finally, the idea of Hispanics forming a genetic cluster will be pernicious for all Hispanics, particularly when applied to medicine. It puts our lives in unnecessary danger.
Do any of them really cluster genetically? Compare a person whose ancestors come from Senegal to a person whose ancestors come from Mozambique – Americans will call both persons "Black", yet I would not expect those two people to have many recent ancestors in common, or much "genetic clustering" at all.
You are thinking too literally. It's just a question that opens the door to more discussion. It's another piece of information, not the end of the discussion.
But does it always "open the door to more discussion", or do some medical professionals just make judgements based on which box was ticked on the patient form? Presented with an African immigrant patient, is a US doctor going to realise that the patient may have rather different genetics from most African-Americans, and hence information about health risks for African-Americans may not be relevant to them? I'm sure an above-average doctor would realise that and take it into account, but would a below-average doctor (of whom there are very many) do it?
If more information is better, isn't that a good argument for replacing "race" questions which expect coarse-grained answers with "ethnicity"/"ancestry" questions which expect more fine-grained answers? (i.e. "Chinese" or "Filipino" not "Asian", "African-American" or "Kenyan" not "Black", "English" or "Polish" not "White", etc.)
And yet Spain (and Italy) are relatively stable first world countries. Damn, I live in the US, and I would say there is way more hustlery here than back in Spain.
For me it's a mixed bag, and we will be probably moving back to Europe next year, after spending 6 years in between San Francisco and New York.
On the positive side, money. Software engineers make much, much more in the US than anywhere in Europe, including Nordic countries. My income tripled in the US, and my income was considered pretty high Denmark. Another positive side of the US is that there seems to be more opportunities, particularly if you are white or Asian and work in tech. My wife couldn't get a job in Europe, here she joined multiple FAANGs.
On the negative side, the lifestyle doesn't really fit us: too much focus on work. The problem is not -only- longer working hours, but that a lot people center their life around their job. I also find the social issues of the US (unequality, racism) more disturbing than the ones there are in Europe.
> I also find the social issues of the US (unequality, racism) more disturbing than the ones there are in Europe.
On the contrary, when I lived in Europe (Rome) I found it disturbing that the overt racism and inequality was not acknowledged by anyone. I got the impression Italians believed that racism only existed in America and that they themselves couldn't possibly be racist (even though moments earlier they were disparaging east European immigrants)
With all the talk about the USA and Europe being such horribly racist places I do have to wonder why so many people who should expect to be subject to said racism are literally dying to get to just those places. I suspect this is because they think life will be better in the USA and Europa than it is in the even more horrible places they came from. Seen in that light, and compared to the way things were in the USA and Europe not even half a century ago all the talk of these places being racist hellholes really does not seem to fit reality - and that is stating it mildly.
The more things are called racist, the less people will care. OK, so I'm racist for making rice pudding? Fine, so be it.
> I suspect this is because they think life will be better in the USA and Europa than it is in the even more horrible places they came from
Not always, but often times the reason their country is a horrible place to live is because of historical racism in the form of colonisation on the part of
European countries or the USA. And it goes without saying that just because something is worse in place A doesn't mean it isn't bad in place B.
What about the "historical racism" from, say, Zulu versus Xhosa? Toltec vs. Chichimec? Chichimec vs. what what was to become the Aztec? Aztec vs. just about any other group they managed to conquer (and sacrifice by pulling their hearts out of their living bodies)? Arabs versus Africans of many colours? Arabs versus whites? The slavers of the Barbary coast - that is, (Islamic) pirates who operated from the coasts of northern Africa - captured and sold millions of light-skinned Europeans on the slave markets in what now is Libya, Algeria, Tunisia and Morocco, why aren't they up in arms against this "historic racism"?. It took the French conquest of the area to finally put a stop to those slave raids - maybe the French were justified in conquering a region from where pirates had been raiding French (and Italian and Dutch and British and even Scandinavian) villages from where they took slaves to the markets of Tripoli? The Ottoman empire was rife with slave trade but not much is made of that for some reason, why? What happened to the Picts who inhabited the British islands? The Irish and Italians were subject to racism and enslavement but I have yet to see any of them call for retributions, why is this?
Do you know the etymology of the word "slave", that is comes from the Slavic tribes who inhabited central Europe and were enslaved by Arabs: 'The Slav was the most prized of human goods. With increased strength outside his marshy land of origin, hardened to the utmost against all privation, industrious, content with little, good-humoured, and cheerful, he filled the slave markets of Europe, Asia, and Africa. It must be remembered that for every Slavonic slave who reached his destination, at least ten succumbed to inhuman treatment during transport and to the heat of the climate. Indeed Ibrāhīm (tenth century), himself in all probability a slave dealer, says: "And the Slavs cannot travel to Lombardy on account of the heat which is fatal to them." Hence their high price.' (The Cambridge Medieval History, Vol. II, 1913).
I can go on forever for the history of humanity is indeed a sordid affair, riddled with accounts of cruelty and - seen through modern western eyes steeped in Judeo-Christian morality - truly proof of the fallen nature of the human race. I won't since it doesn't change the fact that societies and cultures rise and fall through conquest, then and now. Claiming, as you do, that "...the reason their country is a horrible place to live is because of historical racism in the form of colonisation on the part of European countries or the USA" is myopic, it stems from a vision of the world as a European/American-centric place where all roads lead to Europe and America.
This self-flagellation which is so popular in many western cultures needs to stop or those same cultures will succumb to the process of history which gave rise to them in that other cultures will take over. Many of those cultures don't care about racism - they are openly racist - and have no qualms about slavery.
I guess because "racism" is a probable misnomer. Especially in countries where most people are of the same race, but there are multiple different ethnicities or cultures.
If you mean Slavs, the Coalition of Communities of Colour actually formally recognises us as people of colour.
European everyday racism feels a bit different than American racism. It's the end result of many cultures interacting over centuries and accruing generational memories of when X group screwed them over. Meaning, everyone is equally racist to one another. Also, it's not based so much on physical race, as on group belonging. For example, often people from within a single country would be prejudiced to one another, based on the specific city one is born in. Depending on how the country borders get redrawn and what the government decides to call the ethnicity of people within them, those relations can technically flip between being racist and not.
Basically, we can be casually racist with each other, because it's between equals. Americans can't, because it's between two groups, one of which used to inflict horrible human rights abuses over the other.
They probably would recognize me too, but I would decline association to be honest, regardless if it is the US or British variant. Also racism to me has a very specific definition. To believe people to be inferior because of extrinsic properties or ethnicity. Not every form of prejudice is racism of course.
The only person I have ever met that though I was too brown was my dermatologist. Sure, others face racism, but it is also used to justify discrimination and I think that is moving backwards. The US has better understanding on multiculturalism but they changed course recently. It is progress as that is inevitable, but I don't see it improving anything.
America has many races. The majority are white or hispanic not black. The opportunities for someone who is black are so much more in America vs Europe. America is a big place.
When you judge someone by birth location you are also judging someone based on race. Would you accept an Indian person born in Moscow as Russian or would you treat them as Indian? This is harmful..
"Basically, we can be casually racist with each other, because it's between equals. Americans can't, because it's between two groups, one of which used to inflict horrible human rights abuses over the other."
I really enjoy the guilty/shame we put upon White Christians about slavery.
Even being the *OnlY* people who fought to end the slavery.
Black merchants/kings? Jesuit Company (Converted Jews), Islamic republics, etc.
All did not care about slavery.
But then these pesky White Christians came and said NO to slavery.
I hope they learn the lesson and be more cultural aware in future.
>>Use of the word arose during the Early Medieval Period, when Slavs from Central and Eastern Europe (Saqaliba) were frequently enslaved by Moors from the Iberian Peninsula and North Africa.
So brown/black peoples enslaved white peoples, but hey the US only thinks about sold black's by black peoples only, a slave can only be black..end of discussion.
Well, let's not put all Blacks under the same denominator, that sounds a bit racist. As a Slav, I hold no ill will towards them acting in a way that was appropriate for the time period.
That said, I do find it ironic, that we had to remove all mentions of master/slave from our software products, so as to not offend our American customers. Again, I don't mind, but it's funny when you think about it.
You didn't have to. You wanted to. Lots of people wanted to. For no real reason other than to fit in or appease a mob. There's no "have to". You don't "have to" do much these days.
I'm not letting people's stupid actions be marked as a "have to" when they're most definitely still stupid and still neither required nor appropriate.
Italians are not racists, Italians are chauvinists. They are acting against other Italians as well. If you're not local (from the village, town, municipality), you're _not local_ and that's about it.
It is part of their culture. The problem runs deep and is shared by countries in the North border (Switzerland, Austria, France).
The history of institutional racism in Europe is different. The levels of institutional racism and violence the US went through in the 20th century are extreme to whatever happened in Europe, barring notable examples (Nazi Germany).
A football hooligan a racist I wouldn't believe it till I see it /s
There is a difference between having racist people in population and having whole system set up rules to discriminate negatively against part of population.
While I might agree with the part that you said they act against other Italians as well, I can not agree with your 1st statement after 11 years of living here in Italy as a foreigner of non EU origin.
> The levels of institutional racism and violence the US went through in the 20th century are extreme to whatever happened in Europe, barring notable examples (Nazi Germany).
Ask a typical European what they think of Roma ("gypsies"), then get back to me.
IMO The problem people have with gypsies is that they're distinctly different culture. They have found ways to benefit from the social system without giving anything in. That naturally frustrates many taxpayers.
I blame the faulty social system for that. For example, if you never work a day in your life you get higher pension payout as in that case it is based on nationwide average wage. It basically penalizes getting a job.
Yeah, it boggles my mind how many people (here in the UK) who are otherwise very tolerant of other races are openly and unapologetically racist against travelling peoples.
The level of institutional racism is much higher in Europe. It is based on location and genetic group. The stuff being addressed in the US isn't on Europe's radar yet. The US is more progressive than Europe on race.
> I also find the social issues of the US (unequality, racism) more disturbing than the ones there are in Europe.
I grew up in the US but have lived in Barcelona for several years now.
I don't believe that racism is worse in the US than Spain; in fact much the opposite. I live in a mixed-income neighborhood in Barcelona and see far more casual discrimination against low-income immigrants and the local Roma subculture here than I've seen in any major US city. The US has historically had really bad racial policies and there are definitely still parts of the country with a backwards view on race and class, but most of the US (and certainly the parts of the country where a high-skilled immigrant might move to) just isn't like that anymore. I can't speak to the rest of Europe, but Barcelona at least seems behind here.
That said, I do agree there's a lot more inequality in the US than Spain. (It seems to me though that's more about the middle and upper class in the US being much richer—the poor in Spain don't seem to have an easier life than the poor in the US.)
Oh, I'm from Barcelona myself. While I agree with you that the upper class in the US is much richer than in Barcelona, I strongly disagree with you about the poor. I think very often about my particular situation, and what would have happened to me if I would have been born in the US. I come from a family that would be on the lower side of the middle class; unfortunately for us, there is a genetic illness running on my father's side of the family. In Spain, to get healthcare was never a problem; I'm not sure if that wouldn't have been an issue in the US.
I see the same problem with education: while we were not poor, I am not sure if I would have been able to go to one of the best schools in my country, as I did for free in Spain. Free education and healthcare are very important.
On racism, I don't wanna sound harsh, but I believe you only see less because the US is much more segregated. I am typically perceived as Hispanic by most Americans, and believe me, racism is an issue. I don't want to imagine how it is for an immigrant from Africa or Latin America.
> In Spain, to get healthcare was never a problem; I'm not sure if that wouldn't have been an issue in the US.
Yes, the Spanish healthcare system works quite well; I'm a fan! It does a much better job of ensuring coverage than the US system, particularly for the lower middle class. In the US, the poorest ~25% have good public coverage through Medicaid or Medicare and the top ~40% do ok with the employer-subsidized system, but there's a giant donut hole in the middle with no good options and the system sucks for them.
> I see the same problem with education: while we were not poor, I am not sure if I would have been able to go to one of the best schools in my country, as I did for free in Spain.
I admire Spain's commitment to free university education and I think it's a good deal for some folks, the lower middle class in particular. But for helping the very poorest advance, I'm not sure it works in practice. The Spanish high-school graduation rate is much lower than the US one, and anecdotally at least I feel like the children from low-income families I know here in Spain are less likely to consider university or a professional career an option at all compared to their peers in the US, despite it being free. I'm not sure how to fix this.
> On racism, I don't wanna sound harsh, but I believe you only see less because the US is much more segregated. I am typically perceived as Hispanic by most Americans, and believe me, racism is an issue.
It's possible that US cities are more segregated on average, but I don't think that's driving my personal observations. I've lived in low-income neighborhoods with 20-90% minority populations my entire adult life, so I've had a lot of exposure to race relations to calibrate against. More concretely, my wife happens to be hispanic, and feels she's experienced more negative racism in Barcelona than she ever did in the US.
> But for helping the very poorest advance, I'm not sure it works in practice.
I think it does. I studied in the south of Spain (I am Spaniard too), and some of my fellow students comes from very poor sides of the city.
It is very frequent now to see teachers, lawyers and other careers that requires University level studies that are Gypsies, that were born and raised inside those communities.
IMHO free quality education is the perfect equalizer, but it is true that there is much to do there.
> That said, I don't believe that racism/social issues are worse in the US than Spain
My current theory is that the difference between Europe and the US right now is that discrimination is about the same but that in the US racism is openly discussed and criticized. This leads to an illusion that racism is worse in the US because we hear about it more. In this theory Europe like the US in the 60s in which racism was prevalent but sort of more hidden in the dominant white public consciousness.
If I’ve learned anything in my decade plus in tech on both sides of the hiring table it’s almost no one cares about the race/gender of candidates, and that lack of care generally favors white men.
Not through bad intent from folks doing the hiring just through human things: white men are more likely to know what to expect from a tech interview, or even to know someone who can give them an inside scoop. Without even getting into how off putting some job listings can be to many folks not extremely steeped in nerd culture.
The actual interview process everywhere I’ve worked has been pretty rigid and without doing some form of blinding on the code parts could not be more neutral to the candidates identity.
Which is to say: if you’re not getting interviews as an experienced software engineer you must have something glaringly wrong with your resume. Maybe have a trusted friend or former coworker take a look. If you’re not getting offers from at least some of those interviews you probably need to brush up on your algorithms, and make sure you’re actually conversational in all the things listed on your resume.
>If I’ve learned anything in my decade plus in tech on both sides of the hiring table it’s almost no one cares about the race/gender of candidates, and that lack of care generally favors white men.
I don't think it "favors" white men. Go into any university Computer Science or engineering class and what will you find? Mostly White men and foreign students. Of course there will be other genders and ethnic groups but the majority of students are white men and foreign students. Therefore most of the candidates for jobs right out of school for most engineering positions at tech companies are white men because it is hard for foreign students to obtain work permits.
This would say the issue with diversity needs to be solved elsewhere. I think businesses should be judged on the diversity of the departments based on the diversity of people with the proper credentials. It is as asinine to think a tech company's engineering department should match the demographics of society when the demographics of the people obtaining the necessary skills to work in that department are so far off.
I kind of agree here, but the problem is that big tech is not even matching the "demographics of the people obtaining the necessary skills to work in that department".
According to your own data, Hispanics got 8.16% of CS degrees; African Americans got 3.81%. Facebook tech workers are 4.6% Hispanics and 2.1% Black. Google tech workers are 5.3% Hispanic and 2.9% Black. Both companies are far away from even matching the percentages of the candidate field.
I’m an associate-level engineer. Why would a company, many of which are increasingly proud of their diverse team, pick a white engineer when there are hundreds of equally skilled junior applicants that won’t bring down their recruiting metrics?
I’ve literally seen recruiting KPIs at previous startups involve how many black or woman engineers they hire in a quarter.
My current attempts at breaking into the industry is currently being made more difficult by the previous decades of inequity in software engineering teams.
As an immigrant to USA, I think inequality is valid but I have never faced any racism or seen any in engineering jobs. It is overblown. There might be some racism in blue collar work (probably same as EU) but my observation in tech/engineering jobs is always professional.
Maybe you have been lucky. I have seen racism in engineering jobs, most typically in the form of bias or preconceived ideas of how people of certain "races" are supposed to behave, or what are their perspectives in life.
I’ve never observed that having worked in several Fortune 50 companies. It is quite the opposite. Whites in US tech scene are being impartially treated and open-reverse-racism is more commonplace (witnessed it several times in hiring committees. It was quite shocking and blatant). I can imagine racism being rampant maybe 30 years ago in the tech scene.
I am quite frustrated that being colored means I get to be put on posters, literally got a photo taken of me and the diversity/inclusion HR put it up on the company website to further their agenda which I completely disagree with. It always felt condescending to me. I want to be judged by merit and not my skin color.
I completely agree with you. Pity is a particularly disturbing version of contempt. But if you think that this contempt is only reflected in the diversity circus, you are fooling yourself.
High-caste Americans (sorry Wilkerson, the simile is too good to let it go) want to virtue-signal and maybe give some breadcrumbs to the minorities they feel pity for, not put them at their level.
I want to be totally clear - I don’t think “they feel pity for, not put them at their level”. That’s not what I was trying to say.
Every white leader I’ve met have given me opportunities and often their higher ups are not white. The problem is the corporate HR wokeness and their agenda feels condescending to me. That’s the distinction I want to clarify.
Thank you for the perspective!
At the moment, establishing a social life has proved so difficult due to constant relocation and general difficulty of socialising in Sweden, that I'd prefer to go somewhere where at least I'd have better career opportunities in tech.
The Indian numbers are "inflated" as their backlog is high (i.e. they have to petition to renew their H-1B multiple times) - so for example if it takes 12 years for an Indian to get the GC, they petitioned 4-5 times and the "equivalent" percentage is really about a quarter of the figure shown. Whereas a South Korean likely will get the GC without petitioning a second time, so their numbers remain small.
Put another way, if the wait times for Indians was under 3 years, then about 75% or more of the people listed in the Indian count would have gotten their GC a while ago, and would not be counted.
When you adjust for this inflation, it's not obvious that India + China is > 50%. Nevertheless, I agree it is a significant percentage.
Also, we do handle more than a billion requests/day, that was just for giving the title a nice ring.