>You can not test intelligence, you can only test knowledge.

You absolutely can test for a combination of intelligence and knowledge.

>But everyone wants to know intelligence.

Seldom do people actually care about intelligence in isolation. What use is intelligence without the discipline and ability to apply yourself towards a goal for a considerable length of time?

>As the people who study for a long time keep increasing their studying, the scores naturally improve

That simply does not work. Even with the coaching industry trying to lure parents into sending kids to them since Kindergarten, year after year it turns out that the students who do best in the JEE have only had two years of dedicated preparation. Because that is all the time that is needed for a talented student to prepare for the material that is being tested.

> and therefore they need to make the test harder to keep the same pass rate

If anything, the JEE has gotten easier with time.

Honestly, your entire argument seems to be based on dogma and not evidence. Selecting for the top 1% students by their aptitude for math/engineering, within an acceptable error margin, is not really as difficult a problem as you are making it out to be.

I know more about the South Korean system.

They grade to a curve. When the students get better, the test gets harder. Their students spend 16 hours a day studying for their university exams, and start at the age of 12. 46% of students are depressed. Suicide is the leading cause of death in both 10-19 and 20-29 population, mostly due to the stress.

The goal of all students is to pass the test so that they can make it into Seoul University so that Samsung will hire them. The bottleneck is the test. Life afterwards is easy.

Rigorous testing does not give you the best 1% of the population, it filters the people willing to go through the system down to the top 1%. Some of them will be of the .1% best, and others will be those who worked harder as they did not have to help their little sister get home, or their parents clean the restaurant after school.

You can not know your false positives. You can not know your false negatives. Every Einstein born to a poor family who does not get to study is wasted potential to humanity. Every Einstein whose passion is in Machine Learning and spends all of his time on that topic will fail this test.

>The goal of all students is to pass the test so that they can make it into Seoul University so that Samsung will hire them. The bottleneck is the test. Life afterwards is easy.

That sounds quite similar to the JEE tbh. Won't you say that a significant cause of the stress is the cultural expectation that all parents have that their should be able to get into Seoul University and eventually a job at Samsung?

In India there is a good reason for this mindset, because we do not have many good universities and base salaries are very low and of course there is not much of a social security net. I wonder what causes a similar mindset in a developed country like South Korea.

> Every Einstein whose passion is in Machine Learning and spends all of his time on that topic will fail this test.

Any person who does not have a basic understanding of his background and social situation is not an Einstein.

Regardless, that is a very strange example indeed. I really don't understand what kind of a person has the background and social circle that enables him to do top quality research in a highly specialized field of engineering (that also requires considerable funds to run any sort of experiments) at the age of 17 but also doesn't have the connections that will help him get into a decent university.

While I largely agree with your argument, I'm curious if you have a solution in mind that could work in the Indian context.

Use the start-up model whenever possible.

1. Take as many bets as you can.

2. Pay a great deal of attention to those bets. Guide them.

3. As soon as you know the bet failed, let it fail.

Entrance exams are an anti-pattern. They commit to much to early. This means they have to be correct, so they get more strict. This squeezes out exactly who they were hoping to find. Same problem with interviews. Same problem everywhere.

I'm not sure I follow how this would work for students applying for college. In the specific example of the JEE, you have ~ 13K really prestigious seats that ~ 2.2 Million students are competing for, with about ~ 50K seats of decreasing desirability up for grabs. Similarly for the top medical colleges, ~ 340K applicants are competing for ~ 1200 seats. More than half of those seats are reserved for students from traditionally marginalized & oppressed communities burdened with severe economic & societal hardships. I do not understand how a startup model helps in this scenario; ie, college admissions where desirable seats are a severely scarce resource.

You're still thinking of it in one big sort. That's the centralized method. That relies on metrics.

You want several gradual sorts from the bottom up where each sortation relies on personal accountability between the different teachers and administrations. It would probably look much closer to the European football leagues. You would have gradual promotions and relegations every year.

I admit I'm quite baffled by your recommendation.

Is there an example where this system is employed for the purpose of college admissions?

> each sortation relies on personal accountability between the different teachers and administrations

How would such a system actually work? As an illustrative example, the year I graduated High School I was one among about ~ 300K graduating students in my province. There were only ~ 35K college seats across all undergraduate courses (including Women's Colleges, Evening Colleges, etc) in my province up for grabs. If I was interested in medicine, there were ~ 400 seats, of which ~ 150 are female only. How would 'personal accountability' between teachers & administrations work in this scenario? I still fail to grasp how you suggest one kid should go to college vs another. School grades? I had the highest grades in the science exam nationwide, but so did 10 other kids just in my school. All of us could have gotten glowing letters of recommendation from our teachers, and I think that's a terribly subjective way to evaluate large numbers of people.

If your football league analogy is meant to suggest that students should move into and between colleges based on performance, that still doesn't resolve how one gets into a college in the first place without an exam. Even with continual evaluation at the school level, you're still left with many more eligible students (100X) than you have capacity for.

You can not rely on metrics. A letter of recommendation from someone you have never met is just another metric. You keep thinking of this from the top down perspective. India might have such scale that perhaps nothing else is possible. But the more scarce the resources are, the more important that they are used properly.

By 3rd grade you would be filtered into the best class in your city. By the 9th grade you would be filtered into the top province level school. By college, you would be filtered into the position that you already earned. In America we call this system the magnet schools. All I am suggesting is to make this more fluid, and with digital schools coming to be as a result of CO-VID, is the perfect time to experiment with such a system.

More importantly than attempting to fix the Indian education system is what can you do to help against metricification. Do not rely on others scores. If you are hiring and get a resume from someone who did not go to a prestigious school, do not throw it away without looking. People are more than a number. The best often do not look like the best.

>Even with the coaching industry trying to lure parents into sending kids to them since Kindergarten, year after year it turns out that the students who do best in the JEE have only had two years of dedicated preparation. Because that is all the time that is needed for a talented student to prepare for the material that is being tested.

Do we have any figures (anecdotal or otherwise) of how many students in the top rankings have had coaching?

I am of the opinion that the JEE (and every other competitive exam: AIIMS, etc) should be much much harder, and designed with a specific eye to defeat coaching. I'd go so far as to draw an analogy with crypto algorithms designed to be ASIC - resistant.

>Do we have any figures (anecdotal or otherwise) of how many students in the top rankings have had coaching?

I would say 90%. Probably even higher. That's also because pretty much anyone who is serious about taking the exam gets some coaching. However, I know at least a few people that cracked JEE without any coaching. Even a guy who got rank 1 without coaching (Piyush Srivastava).

> I am of the opinion that the JEE (and every other competitive exam: AIIMS, etc) should be much much harder, and designed with a specific eye to defeat coaching

I am not sure how making the exam harder would defeat coaching. The value of the hard problems is that it helps you distinguish better between the very top of the top students. At least at my time there were always a number of such problems thrown in for that purpose.

I also don't quite understand what you hope to achieve here. The whole "coaching centers teach you tricks to solve problems" idea is way exaggerated. (Honest to God, I have a terrible memory and I actually derived half the formulae I needed during the exam itself.) There are many other ways in which coaching centers add value. The most important of which being that you surround yourself with and compete against other talented, motivated students.

The goal isn't for the exam to be hard for the sake of being hard: it's to increase the discriminative power of the exam, increasing the gap between the top student and the bottom student to be selected. The fact that 13 students this year have the top score tells me the test cannot meaningfully discriminate between them, and that is likely true of raw score as well - probably hundreds or thousands of students share the same raw score, with the need to break ties. We're talking about a total point score spread of 375 to pick ~ 50K students, and that's not even including the requirements for minimum qualifying scores. If 13 students are able to get the same raw score, and still can't be meaningfully separated based on the established tie breaking criteria (presumably even more students got the same raw score but could be assigned a lower percentile based on tie breaking), then the issue is likely many magnitudes larger at lower raw scores where the distribution is much more crowded. I'm advocating a system where perhaps 10 points separates even the students at the very top. Of course, this system has its own problems, but that's besides the point: no system is perfect, and I think its still better than the current system.

Sure, the very top students are likely to be both talented & hard working, and would perhaps have still gotten on the list without coaching (though perhaps not high enough to have their pick). It's in the minutiae of small differences in raw scores where the benefits of coaching likely help. If coaching gives me an additional 10 points, that could mean the difference between getting a seat vs not for a bright student.

I think you're suggesting that joining a coaching centre is a strong signal of seriousness, rather than a material benefit from the actual classes. Possible, but as others in this subthread have pointed out, access to coaching is something out of reach for many students, irregardless of their seriousness. To pose the question another way: how many of the 50K people who got a seat would not if coaching (in its current form) did not exist? Perhaps the top 100 names would be redistributed within the top 5000, but they'd still have a seat. There's likely a lot more students in the long tail whose fortunes were determined by access to coaching. My concern is less with the top 100 and more with students at the precarious end.

Raw scores! (Btw what are the un-raw scores called? cooked?!)

13 students getting top scores!!

My God, it has been a while since I wrote the exam, has the situation changed that much since then? At my time, there was a 25 marks difference between the top two ranks.

Is there any chance you are talking about JEE Mains and not the JEE Advanced? JEE Mains is supposed to be the easier test that serves as a "prelims" to the more difficult Advanced exam. For Mains itself, it is not all that important to distinguish between the top students.

>I think you're suggesting that joining a coaching centre is a strong signal of seriousness, rather than a material benefit from the actual classes

I am also suggesting that there are other benefits from joining coaching that are probably more important than the much hyped "tricks" for solving problems. For one, joining a coaching class means that you are part of a community of other students with similar goals and aptitude.

Otherwise, I agree with your overall argument (although not in its entirety). Yes, at the lower end of the ladder, small difference in scores can result in big jumps in ranks and yes, coaching would definitely have a significant impact on the prospects of these students.

The point I do not agree with is lack of access to coaching. Even middle class families of Bihar, the poorest state of India, find the resources to send their kid to Kota for a couple of years. So, it is not that inaccessible. Plus there is apparently free material available on youtube now, created by the IITs itself.

I would also argue that people much below this threshold have bigger worries in life. We as a society definitely need to ask ourselves why it is not possible to have a dignified, livable wage for the majority of people in our country when many other countries seem to have solved this problem. Solving such hard socio-economic problems is a ridiculous expectation from a measly entrance exam.

I was talking about the JEE Mains, because that's the only score the majority of students will be able to use for all the other (non-IIT) colleges. Even if we consider the Advanced, the difference between the top two ranks was only 6 points, and the top 8 were within the 25 point spread that separated the top two in your year (I'm using 2019 for a non pandemic perturbed year). The top 10 were separated by an average of less than 4 points each.

> Even middle class families of Bihar, the poorest state of India, find the resources to send their kid to Kota for a couple of years. So, it is not that inaccessible.

I wonder, though. Quora answers about the cost of coaching in Kota from 2018 yields figures in the range of 2.6 - 3.3 lakhs annually (ie, coaching + accommodation + food, etc). That's a median additional expenditure of 25K per month, for two years. Yes, some families from Bihar are able to muster up the amount, but I suspect there is some survivorship bias here. Even if Kota were to be filled by Bihari students, it doesn't necessarily follow that it is accessible to the average bright motivated Bihari student. My guess it is not.

> I would also argue that people much below this threshold have bigger worries in life.

No argument there.

>Yes, some families from Bihar are able to muster up the amount

Not "some". It is literally lakhs of families every year. And if you look at the data of total number of JEE candidates from Bihar, that is actually a very significant percentage.

Even if you believe that coaching is not accessible to the average bright motivated Bihari student, the very fact that so many people from the poorest, most backward part of the country look at this exam as an instrument to better their lives should tell you something.

> Not "some". It is literally lakhs of families every year. And if you look at the data of total number of JEE candidates from Bihar, that is actually a very significant percentage.

Across all the coaching centres in Kota, you have about ~ 150K students annually, so about 1.5 lakhs in total from all states. Even if the overwhelming majority were Bihari, it doesn't generalize. That's tells us more about the network effect operating in Bihar rather than something about dedicated students in general. That's a lot of kids for a tiny town like Kota, but a drop in the ocean when dealing with the scale of the JEE.

> the very fact that so many people from the poorest, most backward part of the country look at this exam as an instrument to better their lives should tell you something.

Yes, that the exam needs reform. The fact that so many students who 'succeed' at the exams benefited from enrolling in coaching classes suggests that the exam needs to be reformulated to eliminate that variable. It's not the only thing in the education system that needs reform, but it is one among many. If as you suggest serious students benefit from the community of other like minded kids, then self organizing study groups can happen and still save 25K/month.

According to me, the goal of the JEE is to identify ~ 10K kids with the right combination of aptitude & dedication, in a manner that's not blind to their wide ranging social & economic backgrounds. Attending a coaching centre is a signal for their family's desire, but I think we can find much better signals for aptitude & dedication.

> the very fact that so many people from the poorest, most backward part of the country

You keep repeating this misdirection everywhere. Students studying in Kota from the poorest and most backward states are not themselves from poor and backward families. India has tremendous wealth inequality as does Bihar. If they were, they wouldn't be able to afford 2 years of coaching and lodging in Kota.

Several of the candidates qualifying for the exam have appeared multiple times for the exam. Very often the students have been studying for more than 2 years.