As an Indian immigrant I had noticed this very early on too. It made me very uncomfortable when some of us would crack jokes in non-English language while there were others around who didn't understand the language. After first few years I stopped working for companies that had mostly Indians in IT. I must say I have been lucky since then to always find a good mix of diverse backgrounds in my team that it has not been a problem.
As a fellow Indian immigrant, I actually share your discomfort, and always try to be inclusive and diverse in my social groups. Most of my Indian coworkers and friends behave this way — we came to the US to be a part of a mixed culture, so our social groups should be diverse.
What makes me uncomfortable is that this inclusivity is increasingly being taken for granted to the point where not having it starts conversations about how to "fight back".
Even though it's not my preference I don't think there is anything wrong with Indians cracking jokes in a non-English language, or going to a restaurant by themselves. You will find that Americans will mysteriously be far more tolerant of, say, a group of French people talking among themselves in French; and going to a French restaurant as a group.
In public, yeah people probably wouldn't care about french speakers, but in a shared environment I definitely would prefer they speak English. My brother use to have to tell his Dutch-Canadian in-laws to speak English when we were at his house. I'd feel the same way at work.
I am from slavic lands, and found out that I prefer not to work within mostly slavic teams, as any other teams with high genetic and cultural cohesions.