Hacker Newsnew | past | comments | ask | show | jobs | submit | mynameisasdf's commentslogin

By "roll our own" do you mean you just used the twitter streaming API? Did that have enough data?


Yes, Twitter streaming API plus several other data sources (Facebook, Reddit, forums, etc.). It's enough for us since we're always filtering by keywords or users. We were also granted elevated access.


What was the process like for elevated access? I'm deciding between using Gnip or Twitter's streaming API today actually.


It was pretty straightforward in our case. We had to show them that we had use cases that weren't feasible with the default access level and also demonstrate that we weren't competing with their user experience at all. Not sure how much it mattered but we had been using their streaming API for almost as long as it existed.


I had the same question when I read this. The streaming API is only a fraction of the data offered by the firehose, but that's not as much of a problem in every use case.


Who says tech companies do better when there are more women?


I have a hard time imagining that there are many companies who benefit from a monoculture. Diversity equals truth, or more accurately, increases the likelihood of finding truth.

If your business makes any decisions, like "how do we capture this market," or "what sort of features should we add," or "how should this thing work," then you are embarking on a truth-seeking endeavor.

Unless your product aims to solve a problem that only affects white guys in the city (which, to be fair, is certainly a non-negligible number of products), then you're going to have a hard time expanding your base beyond white guys in the city if your team consists of only white guys in the city.


I don't think that is necessarily true; it implies that we cannot empathize with people that do not share at least racial and location characteristics with us.

I think a white man can make a product for black women, and I think a black woman can make products for white men. For me, it's more about acknowledging that the world exists outside of white men in the city and less about making sure your team is made up of every racial and socio-economic variable you aim to market to.


I do not disagree with that at all, and clearly, we've seen monocultures succeed at doing exactly that. I wasn't attempting to decry the methodology of every company with a monoculture, as much as to point out that it's just easier with actual diversity.

As Vezzy-Fnord points out, a single individual is able to empathize with other types of people, but that empathy is finite, and also, empathy does not equal understanding.


My main problem with the way many people handle the diversity issue has been exactly this: their major insistence on phenotypes as the pinnacle of diversity.

It turns out that "white guys" are not a homogenous group. In fact, chances are that white guys from (e.g.) Finland, Serbia and Nebraska will have little in common besides their skin color and sex. Their opinions and worldviews will differ greatly.

Diversity of opinion is just as important, if not more so. Sure, on the outside everyone looks like a pasty-faced white dude, but jumping to conclusions based solely on that, is... misguided, to put it euphemistically.

I'm not saying we can't benefit from having a wide variety of phenotypes as well, after all different phenotypes equal different experiences. But not every white guy is a Bay Area caricature.


I can't tell if you're being serious, so I'll assume that you are. One google search led me to this study: "gender diversity generates significant gains in high-tech/ knowledge intensive sectors".

http://ftp.iza.org/dp7350.pdf


This was only made for us to feel guilty about the status quo. People need to stop focusing intra-industry and look at the larger symptoms: this is a societal problem. This is how families at large are raising their children. If we want more women developers/entrepreneurs, we need to take a serious look at the gender roles we're setting up for girls as early as preschool. We need to stop the incessant "We need to fix the software industry blah blah blah" crap.


Except even feminist societies are finding that the women and men prefer the gender roles.

Maybe "we" should feel guilty about socially engineering something that is proven to make people unhappy. Ironically, the happiness index in women is down in Western societies whereas the same measure for men, is up! And we can hardly argue that women are treated as less equal than they were in the 70s.

People have a knee jerk reaction and say "oh yeah, that's because women have to do even more work now" where that isn't actually the case. Married women with traditional gender roles are actually happier than their married and equal counterparts.

Now, I know I'm not providing any statistics or references but I would urge you to be on the lookout. They show up quite regularly and are easy to find once you are open to the concept. One good place to start is "Brainwash", a "documentary" by a comedian that actually resulted in a loss of funds to a gender studies institute.

All this being said, I do not think a single one of us would say that a woman should not be encouraged to be a programmer. But to deny the joy that a woman can get from being a "traditional" mother as well as a programmer is unfair to an entire gender and unfairly creates a generation of latchkey children. I do not know a single mother or father who says "Yeah, I wish I spent less time with my kids."


You aren't wrong, but this viewpoint is incomplete. That approach does nothing to stem the well-documented repelling forces that exist in the industry today and the anecdotal yet overwhelming accounts of women who are interested in these things feeling unwelcome and leaving. Asking someone who feels unwelcome to suck it up instead of introspecting on how we can do better now is ridiculous and morally questionable.


Exactly. This is stupid. There was some bogus Gallup "study" that says that companies performed 15% better when there were more women, and I call bullshit on this. They didn't say which companies, and in which fields these companies activate in.


Source?


Exactly what I was thinking. Its quite insulting to suggest that girls are so myopic they can't possibly be inspired unless they meet someone of the same gender.

For the record, I'm a female engineer who moved to the USA from Ghana when I was a smaller. I can't stand this ridiculous patronizing attitude there is towards female engineers, as if we need "extra help" or something. Even worse are the feminists creating us-vs-them politicization of the issue.

When I was learning to code, ride a bike, tie my shoes or something, I never thought "gosh I can't do that since none of the role models are female". In fact, if there are no role models of the same gender or race, its usually even more inspiring as you want to be the first one.


> I can't stand this ridiculous patronizing attitude there is towards female engineers, as if we need "extra help" or something. Even worse are the feminists creating us-vs-them politicization of the issue.

Nobody is saying women engineers need extra help. Wanting to meet someone who you identify with as a woman in STEM is not necessarily easy and it is totally legitimate to seek someone out. As for feminists and feminism, I don't know of anything that would suggest an us vs. them attitude, generally feminist leaning groups look to increase participation of women in STEM through a variety of ways.

> When I was learning to code, ride a bike, tie my shoes or something, I never thought "gosh I can't do that since none of the role models are female". In fact, if there are no role models of the same gender or race, its usually even more inspiring as you want to be the first one.

This may be true for yourself and others, but we know that this is not true on a larger scale. People can and do internalize messages about the kind of people that exist in certain professions and this can be self limiting in both conscious and unconscious ways. I'm not speaking for any one person about that, but it is legitimate to find a role model that you identify with esp. when you are a minority status in a group.

You can read more about stereotype threat here: http://www.reducingstereotypethreat.org/definition.html


> People can and do internalize messages

Thanks for this. That's why I said she could use some inspiration. And not all the messages come from men. Women and friends provide powerful messages as well.


I used to feel that way. Then, I realized I had no one to talk to when I saw blatant sexism affect my friends because there were no tenured woman on the faculty whom I knew. I've been lucky that it hasn't hit me, but it's out there and it is insidious.

Also, I'll add another +1 to the people who are saying it's nice to see someone like you. There are questions that it is safer to ask someone who has faced similar struggles in the past. For example, I would love to meet more female professors who have managed to find a balance between family/R01-level research/service.


I am glad to know there are others who see it the way I do.

I draw great inspiration from people who do not have my gender, my race, or several other role model limiting demographic factors.

I feel very sad for people who are set up to not have the advantage of looking up to everyone of value.

===========

Edit: why is this being downvoted? Is it disagreeable to say "I have learned from people unlike me?" Are we now saying it's bad to argue for inclusion?

It's a polite, friendly comment, stating a positive opinion that says "I think people of every gender, race, etc are of value and should be available."

I feel like people are using downvote to show personal frustration.


The reason why you are being downvoted is that the notion of "I think people of every gender, race, etc are of value and should be available." doesn't really address the issue of why someone who is in a minority in STEM would seek out someone like themselves as a role model.

People have value, but we can't deny that for some people finding someone like themselves in a particular field is very difficult. Those people certainly can and do learn from others, but finding someone that has faced the same issue and challenges as themselves is very helpful in not only exploring an issue, but also in counteracting stereotype threat.


"The reason why you are being downvoted is that the notion of "I think people of every gender, race, etc are of value and should be available." doesn't really address the issue of why someone who is in a minority in STEM would seek out someone like themselves as a role model."

This reads to me like

"You are being downvoted because your opinion doesn't address the original question you asked."

I have been polite, friendly, and reasonable. You guys are punishing an opinion about increasing inclusiveness because it goes against your norm.


> "You are being downvoted because your opinion doesn't address the original question you asked."

Except your original post said:

"I don't understand why we all talk about the gender problem in engineering, then continue it by isolating female students from male teachers."

The gender problem in STEM isn't that women have a hard time finding men as peers and role models, its that women have a hard time finding women as peers and role models. You started out by misidentifying the problem in the first place, this is why your subsequent posts are downvoted because you are creating a straw man out of the OP's request to find a woman in STEM for their daughter to meet.


I can't believe that no one has spotted that she is exactly the same annoying feminist that forced the company to get rid of a rug because it used the term "meritocracy". http://readwrite.com/2014/01/24/github-meritocracy-rug#awesm... Tbfh, she sounds like Adria Richards v2.0


I found that episode terribly confused, and I never understood the rationale behind it. Isn't "meritocracy" a value and environment that feminists have told us over and over again is a positive development for women? Don't women want to work in a "meritocracy" precisely because their work will be valued and appreciated not because they are women but because excellence at work knows no sex, colour, or creed?


The reasoning goes like this:

"Meritocracy is all well and good as a theory. It's all about who decides what merit is. If it's a privileged group of people who decide the merit, then it's going to be biased. Thus championing meritocracy in this organisation means upholding a hierarchy which is unfair, biased and oppressive to those outside of the people at the top".

In other words, meritocracy = an aristocracy of white males, where if you do good according to white male values you progress. Therefore meritocracy is not progression based on good work.

In a less gender explanation - removing the rug equated to a statement of a lack of trust in the employers. The employers agreed to it being removed in an attempt to gather back some of that trust.

What the whole issue ignores is that the rug was about the platform - meritocracy - because all people see is code, where the better projects get the more stars. Now there is a valid argument here that popularity doesn't equal merit - but it does not negate the concept of meritocracy.

Anyhow, I'm just the messenger - I think that there are some serious problems with this reasoning. It's horrible to twist something good to something bad.


When I look for libraries to use on github I don't know or care about the race, ethnicity, gender, sexual orientation, religious beliefs of the person who wrote the code...

I'm amazed the company wouldn't respond to her by saying: while the tech industry is a very imperfect meritocracy, and our company is still an imperfect one, the service we provide attempts to be a true meritocracy. The rug is about the goal, not the status quo.


It's so sad to see things like this twisted around by people with too much time and an axe to grind. I don't think anyone expects to put down a mat and claim 'Mission Accomplished' on building a meritocracy. Or any other vision statement from a company or person.

The whole purpose of these statements is to represent an ideal to strive for. Things like this are exactly what give militant feminists and other PC groups a bad name.


Steve Klabnik explains it well: https://news.ycombinator.com/item?id=7405881

In short, it’s a not-really-present ideal that’s often used to mask the existing power relationships that are really responsible for people being promoted/demoted to where they are.


The assumption underlying this analysis is that no merit exists outside of subjective value judgement. Furthermore, subjective value judgments are biased in favor of the existing privileged groups. Do I have that about right?


Sort of. “Merit” is more a measure of those groups’ definitions of success. Calling it a meritocracy overly simplifies the circumstances for that success, often reinforcing the power relationships.


So, how does one tease apart what is meant by the original definition of merit, let's call it 'accomplishment', from these subjective definitions of success? Or is that even possible?


> So, how does one tease apart what is meant by the original definition of merit, let's call it 'accomplishment', from these subjective definitions of success? Or is that even possible?

You can't. Merit -- including "success" or "accomplishment" -- is always a subjective value judgement. Even if there is an objective measure, the evaluation of the measure as something meaningful to measure (i.e., that the measure is one of merit or success) is a subjective value judgement.


We're talking about software development here. Writing code that works, and implementing features that make it to the website/product are easily quantifiable metrics (someone who contributes a lot of good code is judged as more worthy).


>And the dominant group in society is, pretty much by definition, the one whose judgement is most influential.

Then what is the viable alternative to achieving merit in the judgement of that group?


So, is there any way to make an achievement or do something of merit outside of the dominant power group's judgement?


Nothing is an achievement or thing of merit outside of the judgement of some individual or set of individuals. Merit/achievement isn't something that exists independently.

And the dominant group in society is, pretty much by definition, the one whose judgement is most influential.


I love this line of reasoning. X isn't perfectly Y, so let's get rid of it in favor of something that's even farther from Y!


It’s not “get rid of meritocracy” but rather “recognize that it isn’t really meritocracy and it’s preserving problems”.


"X isn't perfectly Y, so let's stop working towards it in favor of working towards something that's even farther from Y!" is equally absurd.


No, it’s “X isn’t really Y, despite what we say, and it’s actually harmful, so let’s stop reinforcing problems by pretending that it is Y”. As being discussed elsewhere in this thread, the problem with meritocracy is that it’s dependent on value judgements by those already in power. Simply, it is a fine ideal, but in practice it is unachievable, utopic. Establishing an organization or community as “meritocratic” means ignoring the role of existing dynamics.


Everything you've said is equally true of any hiring system. It makes sense why employers are drawn to the one that provides them with the most value, while also carrying the added benefit of also being the one that isn't systematically sexist/racist.


Meritocracy as an ideal may not be intrinsically sexist or racist, but declaring an organization a meritocracy doesn’t automatically eliminate existing sexism, racism, etc, and instead masks it. That’s the problem. It’s not unlike the “structureless organization”. It’s not really structureless — there are always informal social dynamics in play — and acting as if it is structureless results in avoiding problems instead of confronting them. Everything was all rainbows and unicorns at GitHub until the informal structure apparently resulted in an institutional inability to deal with certain issues. Valve has seen similar problems.

Organizing people is the fourth hard problem.


Are you suggesting that hiring less qualified women for the sake of diversity would help to dispel the notion that women are less qualified, and hired for the sake of diversity?


Of course not. But don’t use meritocracy as an excuse to not be proactive about diversity.


It's certainly something to aspire to. But think about it this way: if someone is claiming that you are already a meritocracy, but their upper management are almost entirely white and male, what subsidiary claim does that seem to be making?

Essentially, use of the word as a description (rather than an aspiration) packages up a whole bundle of problematic claims of the form "we're not sexist, we would have more women rise to the top if only they {tried harder | were smarter | had the technical ability | ...}" (and similarly for minorities).


So by removing the rug she already told the management what she thinks of them. No wonder that love didn't really grow between them...

I also don't agree the inverse conclusion (upper management must be all male because women don't have merit) really follows. What if the women in upper management simply work elsewhere? Where there even any women who complained that they weren't in upper management at GitHub? Do they even have a better/worse hierarchy for people so that people not in upper management should feel like losers?


I bet that rug really tied together the room too...

</Lebowski>

All joking aside, that rug thing is pretty silly. "Meritocracy" is not code for "Straight White Males only." Nevertheless Horvath being feminist does not invalidate any unfair treatment she may have experienced should her claims be substantiated.


Let's go ahead and assume GitHub is a meritocracy.

If you "succeed" and do "well" on GitHub (what metric can we use? Lots of a stars? Getting your name out there?), what does that actually prove?

If GitHub is a meritocracy, what have the "winners" done to set them apart from the rest, and how does that translate into the professional world?


Hadn't even heard of the rug incident until I read about it in this thread, but I'm pretty sure their idea of meritocracy here only applied to GitHub the company, not GitHub the website, and came out of their culture of "Managers and chain-of-command? Not here. You can work on whatever you want at any time and at any location, as long as good work comes out of it."


Making something new that customers like and pay for with a minimal amount of resources?


Either way, this highlights that the founders of GitHub had every intent to run a conducive and inclusive environment. I mean, obviously they failed, but I doubt it was for lack of trying.


Holy shit, that is fucking hilarious!! It's like you know it's going to be her before you open the article, in the back of your mind, "Wasnt there some other sexist nonsense generated by Github a few months back with crazy feminists involved?". Yes mind, yes there was, and it's the same fucking attention whore.

So, as with these outrageous stories there is always a highly commented /g/ thread.

http://boards.4chan.org/g/res/40840868

I'd say it supplies much needed contrast to this thread's aversion to calling this chick out on her flawed character.

Githubs image is being crushed. Schadenfreude. They only need to grow a backbone.


That rug was ridiculous.


wow, thanks for posting this. throws some cold water on the situation.

not saying she is right or wrong about how things went down with the founder's wife etc.

but she clearly is insane.


This article is incredibly badly written and lacks any real thought about the matter. So he had one interview with some douche CEO and now jumps to the conclusion that:

>"Sadly, sexism is alive and well in my industry. "

>"Now I really empathized with all the women who were victims of assault or rape."

Fucking really?

I've been a developer for now over 11 years and must have met thousands of other devs from countless conferences and meetups. Only once, ONCE, have I ever met someone who made a women-bashing comment similar to what he is talking about. This is both infront of women and behind their backs.

The only real sexism prevalent in the tech industry are the groups of militant feminists setting up endless, patronizing, segregated "coding for girls" meetups.


"The only real sexism prevalent in the tech industry are the groups of militant feminists setting up endless, patronizing, segregated "coding for girls" meetups."

ahahahahaha no try again


"The only real sexism prevalent in the tech industry are the groups of militant feminists setting up endless, patronizing, segregated "coding for girls" meetups."

It's easy for this kind of thing to be completely invisible when this is your attitude. Sexism in the tech industry is alive and well, and there are a lot of good reasons to have meet-ups for both girls and women.

When you immediately start lashing out about "militant feminists" the second someone suggests that sexism exists, you're saying that our experiences are invalid and that you know better than us how welcome we are in the industry. It should be completely unsurprising that this does not make us feel welcome.


I don't understand his post the same way. He said that the only sexism prevalent in this industry is the "girls code"-type sexism. Sure, sexism exists everywhere and always, as does racism and other -isms. Simply said, people will be jerks. However, it is unreasonable to draw conclusions for the whole industry based on the behaviour of one such jerk, that just happens to be a male sexist (I'm sure you'd find female sexists and black racists as well).


I don't need to base my opinion off of one guy's account of an incident in a blog post - I have a laundry list of my own very unpleasant experiences as a female developer to base my opinions on. I've also spoken to a great deal of other women who have a very similar list of very similar experiences. A lot of these experiences have aspects that are specific to the tech industry.

If you only have insight into industry sexism as far as one or two jerks, it's likely because you aren't a part of the affected demographic.


"A lot of these experiences have aspects that are specific to the tech industry."

Huh - how can the tech industry be more sexist than other industries? Please explain? I mean in what way can such an experience be tech specific? "Woman can't code" comments perhaps, but what else?


Apologies; I fat-fingered the down button when I meant to +1 your comment. Might someone reading along at home please help remedy that for me?


Only once, ONCE, have I ever met someone who made a women-bashing comment similar to what he is talking about.

I'm guessing you're not a woman. Now, I'm not trying to make judgement on the article either way, but I don't think you can really say "I've only ever seen sexism in tech once" when you aren't someone that actually experiences it.


Why not? That's like saying, "You have to have been shot by a gun to be able to say 'I've only seen someone get shot one time.'"


It's absolutely nothing like that. I'm not even sure where to start on breaking down how wrong that comparison is.

To attempt to mirror your example, it's actually like an Army general sitting in HQ saying "Well, I've only ever seen our guys get shot at once!" when his units are being decimated on the battlefield.

(Please don't read into that comparison. I am not actually attempting to compare women to battle-ready soldiers and men to lazy generals)


I get what you're trying to say now. My point is, if the OP has only seen it happen once, then they've only seen it once. You're denying that they've only observed it one time. That's what I have a problem with, not your sentiment.

You're trying to say, "just because you haven't seen it, doesn't mean it doesn't happen," right?


> Only once, ONCE, have I ever met someone who made a women-bashing comment […] The only real sexism prevalent in the tech industry are the groups of militant feminists

Next up, Hiram Wesley Evans explains he's only ever once seen a lynching as a child and how MLK is the real racist.


Something here doesn't make sense:

- They can afford to spend $900m on a shitty app like this that no one uses

but...

- They pay the average 31 year old engineer in Tokyo (arguably the most expensive city in the world) a measly $66K http://jobtalk.jp/company/1527_earns.html

- None of their international subsidiaries have any real traction (buy.com, play.com, kobo)

Where is this money coming from?


Ok, so you don't use so no one uses it.

Tech salaries suck almost everywhere in Tokyo.

Doesn't matter if their international subsidiaries don't have great traction because domestically they do pretty well, that's where the money is coming from.


>anti-intellectualism that is so pervasive in the software development scene

Errrr....


How else do you characterize an industry that seems content to re-learn lessons of the past over and over?


1) Industries don't learn; people do. We have a lot of turnover and growth in this industry, so a lot of new people. 2) Although there are high points of genius we may never reach again (Turing, etc), we are making progress. Software today can do things hardly imagined a few decades ago. 3) Part of how we make progress is to try "unlearning" things; throw out conventional wisdom and try something "crazy". Maybe it will fail the same way as before, or maybe this time it will work. Maybe the constraints that produced the conventional wisdom have changed. 4) People problems will always be hard, in every industry. Productivity is a people problem.


I am against a general ignorance of where the industry's been, which manifests as a refusal to learn basic software engineering concepts that are laid out in a book like the Mythical Man Month. Seems like every new medium (such as the web) starts with masses rallying around a figurehead who proclaims "this time it's different! We don't need any of that engineering stuff!" A few years later, they do.

This trend of anti-intellectualism is worrying. I doubt it's new -- Dijkstra had similar sentiments -- but the worst part is the developers who seem to enjoy being ignorant.


It seems like almost all professions have a body of knowledge. Software Engineering doesn't.

Imagine getting open heart surgery, your chest is open, and doctors start arguing over the best methodology to do it.


This is not a fair analogy. Open heart surgery is a repetitive and well-defined procedure with clear goals and context. Software Engineering is a complex and creative endeavour where individual talent makes a big difference. There is no methodology that will help a mediocre writer produce a great novel. It takes skill and talent.


This is exactly why software engineering needs to professionalize.

I don't see developers always having more leverage than their business counterparts. Really, I'd love to see developers wholly accountable to their peers, along with some sort of entrance exam. I think the world at large would take developers a bit more seriously.


It does, and it has for quite some time: http://www.computer.org/portal/web/swebok


OMG. I just hope you are being sarcastic.

That's everything we are complaining about right there on the index.


No sarcasm intended. You don't have to agree with SEI or IEEE, but the complaint was that there wasn't an organized body of knowledge. My response was to show that there is.

I'd be interested in knowing what you find so objectionable about SWEBOK.


Actually, there are surprisingly large and persistent differences between hospitals in their complication rates for various surgeries. It is fueling a debate about how to evaluate them and how to promote the methodologies of the more effective places. Not really all that different from the software industry.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: