Universities discuss this all the time. It's incredibly hard to prove.
This is one reason we keep exams -- while they have many problems (particularly in Computer Science), forcing someone to sit in a room and answer questions gives a way of checking the student really does understand the course.
While a grad student at TX A&M CS, I rolled out a grading rubric (for a couple of intro courses) which required the students to pass individually each of the homework, quiz, and exams, separately, to pass the course.
This nipped HW cheating in the bud by the 3rd semester, because the students knew that you couldn’t game the system by maximizing HW grades to bring up exam & quiz grades. Exams & quizzes were autogenerated from a set of parametric problems, and put in random order on the print out — made side-by-side and answer-sheet cheating basically impossible.
Faculty weren’t as charmed with the cratered grades, though.
Are there any downsides to removing the potential for students to compensate poor test and quiz scores with strong homework and take-home work?
I remember having some terrible graduate instructors at the University of Oregon who clearly had 0 desire to teach and routinely expressed frustration when first year students could not keep up with their doctorate level notation and language styles. Most students passed by a hair thanks to homework and quizzes, mixed with study groups and studying topics which weren't covered properly in the class. Had it not been for a heavy weight on HW (about ~35% IIRC) most of the class would have failed. And cynically that homework weight must've been intentional - they knew there was a problem with their course instruction and they covered up presumably high fail rates by bumping HW grading...
The problem with evaluating large groups of people is that many will fall through the cracks, not due to malice but because there is no one-size-fits all solution. Yeah you can improve cheating metrics but how many non-cheaters, good qualified students who put in the work, were also negatively affected by your grading rubric? 0? 1? 100? 1000? Unknown?
And as a corollary this bleeds right into tech recruiting where scores of qualified candidates are put through a song and dance routine which arbitrarily and sometimes biasely culls for no valid reason other then being a day that ends in y.
> Are there any downsides to removing the potential for students to compensate poor test and quiz scores with strong homework and take-home work?
It facilitates cheating in a way that's almost impossible to counteract.
The other way around works though, where you work hard to make sure your final exam isn't something they can cheat on, then allow that to overide or pull up the other grades. This isn't really helpful for people with high test anxiety but it does help.
You can't get this stuff perfect, but you can make sure the result is at least fair.
Once we realized that these same graduate assistants were treating some folks to a different standard, a group of us got together and basically forced them to grade us the same way.
It was bullshit, we were all neck deep in debt paying some incompetent to not teach us, and then haze us with capricious grading and classroom policy.
Weeeelllll... it depends on how you calculate it. Minimum course credit grade was a "C" and total A-C were unaffected; or, at least, in the noise. There was some motion from D-to-F, which affected total GPA average for the course. Honestly, I don't remember the magnitude of the change.
These courses had 600+ students in them.
I'm also certain they stopped a lot of this when I stopped teaching. The "cheaters" would get caught, later, anyways.
What's fascinating to me is about all of this is that techniques like the ones you implemented and others are all that is needed to eliminate cheating. It's the kind of stuff that is analogous to the protocols developed by the air industry in the mid 20th century that has almost eliminated entire classes of error that could result in loss of life from flying.
It is especially troubling because post-secondary education is a sort of cornerstone of all other human endeavours so the cascading effects of people cheating and not learning the material that they need to be better at their jobs influences lives just as much as a plane crashing.
The silly part is that post-secondary education is something that is far older than the flight industry yet no such standardization of controls over testing / grading have been implemented.
It seems like it's almost a choice.
> Faculty weren’t as charmed with the cratered grades, though.
The scientific research that goes on in post secondary education is the cornerstone. The academic setting is just the way we do it now and can, and I would argue should, change. Grad students could just as easily be junior employees if the same work were done in the private sector and would be financially much better off for it.
> It is especially troubling because post-secondary education is a sort of cornerstone of all other human endeavours so the cascading effects of people cheating and not learning the material that they need to be better at their jobs influences lives just as much as a plane crashing.
This is insane. Most people never attend university and do plenty of productive work. Most people who do attend university never work a day in the field they did their degree in. Most people who do work in the field they did their degree in never use the large majority of what they leaned there. On top of this a large majority of professionally relevant material is learned on the job.
The reason that there’s no real effort to check people aren’t cheating is because school isn’t primarily about learning, it’s about ranking and sorting.
That says more about the ruling class from which presidents are selected than it says about universities. No doubt the management of AT&T said similar things about Bell Labs, even after they invented the transistor.
The sad thing is that the person who said this actually cared. Being a student at uni makes little sense unless you know that you can afford to not earn any income
Lawyers, doctors, nurses, engineers, and teachers are all educated at universities.
These are the people who create our standards bodies that dictate how the rest of society functions. Whether it's the legal system, the standards regarding medical practices, the standards regarding electrical and mechanical safety our entire society is built on the education and credentials that these people receive from attending university.
Think about this: Nearly every single person if not every single one that you have ever met was taught to read by a university educated teacher. If they were home-schooled, their syllabus was likely prepared by a university educated teacher.
We are all literate and numerate because of university educated teachers. That fact alone proves that university is (for better or worse) is a cornerstone of all human endeavours.
> We are all literate and numerate because of university educated teachers. That fact alone proves that university is (for better or worse) is a cornerstone of all human endeavours.
No. People were literate and numerate for millennia before universities existed. Universities are not necessary preconditions for literacy or numeracy. At least 10% of children are able to read before they start school at all because their parents taught them. The first society with near universal literacy, Puritan New England did so without any universities and their teachers were in the large majority not university educated.
Then what are you trying to say? If we agree that universities are unnecessary for literacy or numeracy then university is no more a cornerstone of contemporary societies than supermarkets or libraries.
For most people, first years at university is also the first years of independent adulthood, with all its challenges. I sometimes wonder if universities would work better at educating people if everyone was given a year or two to focus on getting their shit together - interests, romances, whatnot - and only then the real education would start.
At my university in the U.K. evaluation was almost entirely done with exams at the end of the year. For undergrad, the exams had questions from all the courses so one could choose whichever questions one wanted to answer (this meant each course only turned up in the exams for one year. Also I did a subject which didn’t require anything like labs).
It meant homework was an opportunity for difficult questions to be used for teaching rather than a test for knowledge—the only thing preventing one from skipping homework was the shame one would experience turning up to go through the solutions and discussion without having made attempts. One could also choose how collaborative to be with the problems and discuss them with peers. The downside was that there was a lot of pressure on exams and if you were ill on exam week you’d be SOL (also no resits because why not). And some people just don’t perform to their true ability under exam conditions (though to be fair the university selected for it with entrance exams.)
This heavily biases in favor of lone wolves in college. I was one myself. I'd say we are rare. Most people are social and learn by collaborating with others. It's the more natural approach. Evolution has groomed us to be social after all.
Maybe 15 years ago, Georgia Tech rolled out a pretty great homework copy detector for the Java kids. The guy who wrote it was well aware that students knew to change variable names and such, so he just had it compare generated byte code instead. It caught something like 200 cheaters. It was a huge problem because Tech really, really didn't want to just expel or fail everyone like the academic rules required, so they created a sort of case-by-case comparison and punished students to various lesser degrees based on some rubric or other.
Anyway, the next year they adjusted by saying "it's absolutely fine to collaborate on homework and projects. Go nuts. Copy off each other all you want. Also, homework is just 25% of your grade now, quizzes and exams are everything." This made for pretty terrifying quizzes, but things worked out for everybody (except folks with a lot of test anxiety) because folks who actually did the homework tended to be the ones that did well on exams anyway.
Back when I was an undergrad studying Applied CS, we[0] had a really friendly, cooperative attitude - we would help each other learn, do homework together; people who turned out to be good at a particular topic would often compile guides and learning material and FAQs for everyone else. Late in my studies I discovered this was seen as a highly unusual thing about our sub-faculty[1]. Other faculties had much more competitive, every person for themselves attitude.
It turns out, the driving factor was that our sub-faculty had different rules about scolarships: you had to cross a threshold of high grades on a given year to be eligible for one, and the amount of money you got was purely determined by your grade average. Everywhere else, scolarships were limited to top % best students. Where everyone else competed and kept their hard-won knowledge to themselves, we'd routinely assist each other, so that everyone could get a shot at getting the scolarship.
I hear that after I graduated, they normalized the incentive scheme to % top students everywhere, and that new Applied CS groups got as unfriendly as everyone else was.
--
[0] - All years studying Applied CS in our sub-faculty[1], not just my year. Don't know what the proper English term for it is, in Poland we call it "kierunek" (literally: direction), vs. "rocznik" meaning a class of a particular year studying on a "kierunek".
[1] - Not sure what's the right term for this either. Our faculty had essentially two branches that dealt with overlapping fields of study; as I was graduating, they ended up splitting into two separate faculties.
Huh. Your comment makes me think. I wonder if some of the push-back on standardized testing is partly because students cannot chea— I mean rely on “social collaboration”? (Not ALL of the pushback, of course. But part of it..)
Not OP, but it could still be an average (either weighted or not) and have a requirement to pass certain things to be eligible to pass. One of my core CS classes had the requirement that you had to complete at least 1 project, complete all daily assignments, and pass at least 4 of the 6 lab tests. However the overall grade for the class once you met those requirements was just an average like it was for my other classes.
People love to complain about whiteboarding but it's extremely effective at assessing knowledge, far more than even most exams. Exams are used in an academic context b/c they're less resource intensive, though.
"To understand if coding interviews—as administered today—can induce stress that significantly hinders performance, we conducted a randomized controlled trial with 48 Computer Science students,comparing them in private and public whiteboard settings.
We found that performance is reduced by more than half, by simply being watched by an interviewer. We also observed that stress and cognitive load were significantly higher in a traditional technical interview when compared with our private interview.
Consequently, interviewers may be filtering out qualified candidates by confounding assessment of problem-solving ability with unnecessary stress."
This has been widely discussed on this site and is a pretty contentious topic.
For my view, consider a very bright problem solver, who bursts into tears every time a person watches or criticizes their code. Do you think that's relevant to performance?
Certainly is a balance, but studies like this usually wilfully ignore all of the other aspects of software work, that aren't sitting in a chair alone writing code.
The stakes are completely different, which is the main problem.
When being interviewed, an entire career is resting on your 45 min - 3 hr performance. And at certain places, that could potentially mean the difference between $50k/year, and $250k/year.
On the other hand, you're not risking your job every time you discuss something with your co-workers.
There will always be absolute edge cases (like the person you described, which by all means sound like an extreme case) - and I think it's fine that some tests manage to exclude these, but the problem is that it's also potentially removing candidates far away from the edge.
White-boarding or "oral exams" are effective in proportion to the skill of the person giving them. Most people have little natural skill in this even (perhaps especially) if they are expert in the domain being examined.
I learned this rather well studying abroad in Ireland. I got a final grade of 92 in a class on protein modeling, which in the UK/Irish grading system is really high.
The professor asked me if I had taken the class before.
I hadn't, but the exam was all short answer and multiple choice. Which was very different from the long-form work most of his students did, but absolutely in an American student's wheelhouse.
I was okay in the class. What I was really, really good at was the test.
I also personally appreciated the time limits of exams as well. They seemed like a good balance to projects whose outcome was primarily decided by how much time one could spend (there's a clear balance in the real world utility of performing well under unstrained time, and time management, etc).
Projects are great. But the best ones are time-consuming because, as you suggest, you can often iterate one more time. This is especially true if the problem space is at least somewhat unconstrained even if the result isn't. I probably got more out of project-oriented courses undergrad/masters (and my Masters thesis) but it doesn't scale to the whole curriculum.
This is one reason we keep exams -- while they have many problems (particularly in Computer Science), forcing someone to sit in a room and answer questions gives a way of checking the student really does understand the course.