This perfectly describes my experience as a TA in graduate school. At first I didn't understand why my advisor insisted on being so precise in assignment instructions. Then when TAing with him I saw how students could creatively misinterpret instructions, even when I could not imagine how to make them more precise. An exception for the new case would be added to the next iteration of the assignments. I only understood why we went to such lengths to prevent cheating because in my first year I watched my advisor spend two weeks of his time sitting down individually with each student and present evidence that they had cheated. Only about 10% of the students had cheated, but in a class of 1400, that's 140 students! I can't even imagine how much work that must of been on the head TA.
I thought the article was fairly strong except for in the two points you highlighted here. In the first case, I still don't understand why you don't just mark their answer from creatively misinterpreted instructions wrong and move on with life. And in the second case it seems like just not worrying about cheaters and letting it be their own funeral (or not) is optimal. I remember who the cheaters were in my classes and a couple decades later it's clear that to a one, I would much rather be in the shoes of the diligent hard workers than the cheaters.
Both questions were answered in the article. The reason for precise directions is because otherwise people will complain, and if you ignore their complaints, they will complain to your boss. At the end you'll win, but you'll waste a bunch of time defending yourself.
The reason for not allowing cheating is repetitional. If you get a reputation for allowing cheaters, then all the cheaters will want to take your class, and eventually you'll have so many that your testing will be worthless. And if word gets out that your institution allows cheating, then your students will not be respected when they leave, causing harm to the non-cheaters and your chance at keeping your job as fewer people want to attend a school known for allowing cheats.
There's a deeper reason for not allowing cheating: you are building cheaters. People who cheat in courses will cheat in industry, why wouldn't they? They normalize this behavior. So you end up with major corporations that steal, politicians that lie, etc.
If for example, Harvard and Yale's law schools stopped rampant cheating. Maybe so many of their graduates wouldn't go on to routinely lie to the public?
I don't teach because it's some sort of penance that I need to pay. I teach because I like it and I want to help build smart humans. Not contribute to our society degenerating.
I would be willing to bet that most of the politicians/ceos/etc that currently lie to everyone's face and went to harvard/yale didn't need to cheat their way through and didn't bother more often than not.
just out of curiosity, did they cheat more than the average cheater? I knew a few of people who cheated in college but it was infrequent and varied by class, friend group, etc.
> The reason for precise directions is because otherwise people will complain, and if you ignore their complaints, they will complain to your boss. At the end you'll win, but you'll waste a bunch of time defending yourself.
So instead you force all your students to do busy work, like signing a statement accepting no grade if they use the wrong size bread board or photographing the breadboard next to a compass to prove it's the alignment?
To me this sounds like lazy teachers punishing students rather than working to solve the problem.
> I still don't understand why you don't just mark their answer from creatively misinterpreted instructions wrong and move on with life
Because the actual incidents are often in fuzzy areas where it seems possible the teacher's instructions were confusing. You're stuck making a character judgment of your student instead of evaluating knowledge. Over a career, it becomes easier to cordon off fuzzy areas than it is to risk a moral challenge.
> it seems possible the teacher's instructions were confusing.
Yes; I've been on both sides. I've written assignments that I thought were clear and unambiguous, only to find that a significant number of students misunderstood what I meant. They weren't intentionally trying to make the problems easier, they just weren't sure what I wanted. (And, of course, who is going to interpret an ambiguous problem so as to make more work for themselves? A few students will do it both ways -- the easier interpretation and the harder one -- but most won't.)
And on the other side, I've taken continuing education classes taught by other teachers where the instructions were confusing, ambiguous, or sometimes just plain impossible to follow ("You'll find the answers to this quiz in the article you just read." but the article was revised and now uses different terminology from the quiz.)
I find that students talk to each other and spread interpretations of the assignment. They might be correct, they might not - either way the interpretation spreads (never through anything like 'official' course forums set up for students to ask about interpretations, of course). They've also gone through shared experiences in other courses beforehand and will often simply come up with the same incorrect interpretation. For 5 years the basic assignment was clear and easily understood, then the next year it's almost universally misinterpreted. Those shared misunderstandings have easily outnumbered creative interpretations to help grades in my experience.
> who is going to interpret an ambiguous problem so as to make more work for themselves?
I did.
In fact I always tried to find a unique or novel solution to my problem sets, ambiguous or not. (If the problem set contained a hint I tried mightily to not use the hint, I'd always try to replace a proof by contradiction with a constructive proof etc...)
My marks suffered for it. I even almost failed a first year exam cos I didn't want to perform a grody 4x4 matrix multiplication. Later the prof said: "Your exam was crap, but you came up with a better answer for problem four than I'd thought of."
It's still one of my most cherished memories from undergrad.
I always hated the: "Will this be on the test" type of attitude. Are you there to learn and break new ground or to just get marks? I had crappy marks but my work spoke for itself.
Students should put more effort into creating their own body of work. If they spent half the energy they put into finding tricks and gaming the system, they'd be much better off for it.
I was never one to game the system until I was failing 3 classes while on academic probation (2 Fs would have gotten me kicked out). Then I gamed the shit out of the system.
That was my breaking point. Others it's losing a scholarship; others, getting a B.
In the first case, they complain, and there's ~750 of them (in the course I TAed) so even a small number can take up a lot of time. The right way to think about it is for a small additional bit of time spent clarifying instructions you save yourself a larger amount of time later.
In the second case, it does depend upon how much the instructor feels it's their duty to uphold the integrity of the grades in their class. I'm not sure if I would have made the same choice in my advisor's shoes, but that is the decision he made.
> In the first case, I still don't understand why you don't just mark their answer from creatively misinterpreted instructions wrong and move on with life.
Because your job is to educate them. They also complain about the task which in effect waste your time or give you trouble.
"Creatively misinterpreting" instructions means to me that the students are intentionally doing this (to get away with doing less work, or whatever). I think marking them down and moving on is educating them: it very quickly tells them that sticking to the letter of the law but ignoring the spirit is not ok, and will not be tolerated. It's pretty good preparation for being in the real world, too.
Regardless, giving ridiculously over-specified assignments will not be good preparation for the real world, where many (most?) things are under-specified and ambiguous. Adults need to learn how to read between the lines, interpret things properly, be comfortable asking follow-up questions for things that are not clear, and just figure things out when such clarity doesn't exist.
> They also complain about the task which in effect waste your time or give you trouble.
That sounds annoying, but to me it feels like over-specifying tasks in this way is the opposite of education. And it feels like the time dealing with the misinterpreters wouldn't be wasted; it would be spent actively teaching students that the world is not black and white, there's often no instruction manual, and that getting out of doing work through "creative misinterpretation" will not get you far.
It seems like a good learning experience to get an answer wrong because you didn't succeed in interpreting the question. Nobody takes pains to describe things in minute detail in real life.
> Then when TAing with him I saw how students could creatively misinterpret instructions, even when I could not imagine how to make them more precise.
The best part is if you do make it more precise by specifying the problem in more detail, they will just not read it and ask questions that you answered explicitly in the assignment.
sometimes "precise" in the mind of the instructor is "unintelligibly technical" to the student. I'm tutoring an (ESL) friend through an intro to programming course right now, and every time she gets an assignment she sends me the full text of it just to ask me what the instructions mean. to me, the instructions are almost describing line-by-line exactly what to write. but to someone who isn't already at the level where they can just read and understand random pages on cppreference, it's basically impenetrable. this is a course designed for people who not only have zero programming experience, but also don't even intend to pursue a CS major/minor.
OTOH, I've definitely taken classes with years out-of-date syllabi. It is a funny thing, where some instructors consider it to be the fundamental contract between them and the student, and others consider it to be an annoying bit of extra busywork.
On the other hand, if you don't address the pathological cases in writing, 90% of your time will be taken up by the 10% of people who rules-lawyer their way through life: Pointing out the lack of written clarity, complaining about 'hidden rules', writing a letter to object, appealing to your boss, appealing to boss's boss, lodging a formal complaint with leadership implying discrimination, getting actual lawyers involved, and on and on and on.
There are a small number of people who just live for the thrill of taking advantage of poorly documented rules or process. They act disingenuously under the guise of sincerity. "I'm just trying to clarify: Nowhere is it written that [$obvious_bad_behavior] is not allowed, therefore how am I supposed to know??" People who spend more time scrutinizing their university's Policies, Rules and Regulations, and Code Of Conduct, looking for exploitable flaws, than they would ever spend actually reading their assignments. Happens in the business world too. I've seen salesmen who couldn't multiply two three-digit numbers together turn into Albert Einstein when the year's bonus structure got published.
> I've seen salesmen who couldn't multiply two three-digit numbers together turn into Albert Einstein when the year's bonus structure got published.
I kinda think you are arguing against your point, here. IMO these sorts of sales people are a result of over-specifying homework questions to this degree, because they haven't been shut down or washed out at the stage where you find out they can't deal with a reasonable (or even too-low) level of detail.
But the problems you talk about in your first paragraph are real problems, and the solution is that the entirety of the school's administration needs to take a zero-tolerance approach with this sort of behavior. Rules-lawyering should be shut down at every step of the way. Yes, that might result in some actual lawsuits, which will suck up time and money, but I think that's just the price of educating people. And might still end up being less trouble overall.
Yeah that's why you can have a catch-all rule like "TA is conferred final discretion on evaluations"
Though as I said, some things are good to have in writing, if it's an exception that happens with some frequency or some corner case that's not as rare as thought
After just starting a grad program after 12 years in industry. I'd have to disagree. While a large fraction of homework is busy work designed to give the illusion of challenge and rigor - tests simply estimate whether someone has memorized the material sufficiently for a short 1 hour exam.
In CS, a ~4-20 hour project is vastly more representative of how well someone understands the material and could apply it in a real world setting than a 40 minute multiple choice exam. At the advanced levels this is true for fields such as Physics, English, History or any others.
Maybe we should ask ourselves how to give better assignments in a class that aren't simply busy work?
Many engineering programs have their most challenging courses set up as semester long projects.
In chemical engineering the final boss is the process design class, a project where you are asked to produce a chemical substance with desired properties at scale without losing money. Almost everything you learned during the program has to be used to pull it off. Programming, numerical methods, CAD, Transport phenomena, kinetics, physical chemistry, thermodynamics. It really is the best all around test for a chemical engineer.
While this is feasible for the senior year, I am not sure if you can convert for example calculus 1 into a semester long project.
Calculus 1 is an interesting subject as there certainly is a degree of memorization required (you can't re-derive the derivative of x^n every time it comes up in your career). There is a similar to intro to Organic Chemistry, Algorithms and DataStructures, intro to programming etc. But the goal is to build detailed understanding of these methods more so than memorization.
On the other hand we live in a world where access to derivative rules is trivial. I'd imagine in 1800 mathematicians would assume that you would need to have multiplication tables to be productive and not reduced to pen and paper their entire career.
I wonder if there is an opportunity to push more challenging material into the earlier classes and make them more project like.
I am currently in the last few days before submitting my Chemical Engineering Design Project (I'm designing a packed bed methanol reactor), and yes I can confirm it is absolutely fucking brutal and hands down the hardest thing I've done in my life so far
I get what you're saying but I also disagree with it as a generalization, and say it would depend on the subject. For theoretical subjects, an exam is about the only way to test your understanding. Memorization is not going to help you solve math problems.
I was a physics undergrad who hopped into a few grad classes, and to be honest I was terrible at homework and great at exams (mostly due to some youthful obstinance on putting the time in on homework). At the time I believed that the exams showed who really knew the material and who applied time to solve the problem. With some time past I see that the larger/tougher problem sets were where the real challenge was.
I recall a few unique problem sets from Graduate QM such as
- Derive from first principles the color of the sky.
- Prove that charge must be Quantized if there is one magnetic mono-pole in the universe.
The exam questions were far simpler than the theory questions asked in the problem sets. The work for the first question easily totals > 20 hours of pen and paper time.
> The work for the first question easily totals > 20 hours of pen and paper time.
I guess grad students generally take less coursework than undergrads, but how could a professor expect students to have 20+ hours on hand to solve a single question, given other demands on a student's time?
> Memorization is not going to help you solve math problems.
On the contrary, memorization is the way most people I know got through most of their math classes, at least through calculus and linear algebra. You memorize the steps by rote repetition without really learning why they work, then the test is mostly an exercise in guessing which steps and formulas you should apply to the given problem.
Is that really memorization? Memorizing multiplication tables is one thing. Practicing the techniques over and over isn't memorization imo. In grad level maths, you are solving proofs pretty much, you can't just memorize facts in a textbook to do that.
It's memorization insofar as you can do all of that practice and become proficient at solving math problems without really knowing what they mean or why the steps work. You're regurgitating what you were taught, not making connections and using your understanding.
You used math as an example of a subject where tests are used to check understanding. I disagree, because most people that I know who did well in math did so by being good human computers, not by understanding anything.
I expect that doesn't continue to be true at the grad level, but most people don't get that far.
I’m someone who crammed their way through 4 years of computer engineering exams at a challenging university. It’s possible. It’s hard and the worst few weeks of life before exams, but it’s possible.
Can confirm. There's 0 retention. Maybe if I kept cramming over an extended period of time I could retain it. Typically though I stop after taking the exam so within about a week or two things I thought I understood disappeared.
>tests simply estimate whether someone has memorized the material sufficiently for a short 1 hour exam.
I feel a deep sadness reading this. Is your computer science curriculum more accurately described as a software engineering curriculum?
Memorization should be virtually irrelevant on most computer science exams. Proofs should be core to computer science exams; the ability to reason is the most fundamental skill to all scientists, especially for fields which are tightly coupled to mathematics.
> Is your computer science curriculum more accurately described as a software engineering curriculum?
Given that most CS students want to go into software engineering, it would surprise me if this isn't the case for most CS curriculums. In my experience CS students don't generally want to be scientists, so most CS classes are more application-oriented than proof-oriented.
Schools are starting to provide separate software engineering programs, but we're not all the way there yet.
I disagree, but at least you didn't use the word "regurgitate".
I always find it funny when people say that tests are just about "regurgitating" information. It's such a cliché that just gets regurgitated in every argument over testing, as though it's visceral imagery actually gives it any real weight.
Tests can assess whether the student learnt the material covered in class. They can also test problem solving abilities.
Assignments test conscientiousness, and the ability to make good design trade-offs when working with a single customer who is buying 100 different custom products and doesn't really care about any of them.
Graded assignments are useful to give feedback to students. And more importantly they force students to work regularly and not wait for the last minute to study.
I think it is mostly the latter. At least -- I rarely got useful feedback other than a little x (best case it would be on the error, more likely on the questions).
Personally, when grading I keep a file of all my feedback so I can easily copy-paste it into their feedback files (since everything is digital nowadays). For a given assignment, usually only a handful of mistakes are made (repeated by each student). If anything, having the file makes my grading more consistent -- same points for the same error.
I'm under the impression that this is a not-unpopular system, but try as I might, I cannot get anyone else to adopt it.
Personally I always preferred quizzes for that. I've always been a very strong autodidact though, there are probably people who prefer getting dragged through things by homework.
This is tough because it creates a strong incentive for them to make bad long-term decisions. Think of it from the perspective of a student: you're taking 6 other courses, all of them very demanding with graded assignments, except for this one class where the assignments are not graded. You have a limited budget of time over the week, and time is getting short. Do you: a) work really hard on your ungraded assignment and turn in your best effort for no impact on your grade or b) tell yourself that you'll make up the work at a later point in time, and then focus on your other graded assignments to make sure you optimize those grade. Then you will focus on the other course later on during spring break or something.
Sure everyone says they'll do a but really, this sets a lot of students up for a trap. They think they will have time to make all of this up later, but really what will happen is they will just fall behind in the class. The assignments from other courses keep piling up, so the free time never really materializes. In fact, the same scenario repeats: the student will forego a second assignment, having already done so once before. Then the deferred responsibilities pile up and you end up with a student who is failing your course (even though on paper the grade is undetermined (kind of like a wave function), in all actuality it's just waiting to collapse to a grade of F at test time.
Look at it this way: it's like a reinforcement learning problem. If your reward schedule is that you only give a reward to the agent when it achieves the end goal, sometimes training that agent takes a very long time; if the search space is too large, then the agent can go any which way and will take a long time to reach that goal. That's ungraded assignments.
Instead, if you give the agent little rewards along the way when it makes some significant progress, then the agent can converge to the goal state much faster, in a way that avoid a lot of unpleasantness for everyone. I don't like giving Fs, and they don't like receiving Fs. I feel like if I give an F that's really more on me than them. Part of my job is not just to put course content into student brains, but to also shape their ability to manage their time and juggle a variety of projects. It's the kind of thing I spend many semesters (4) instilling in my students and grades are one of the effective tools I use to do so.
You may say just do away with all grades and we can talk about that. There are different models we could use. But as long as others are using grades it's kind of a baked in assumption at this point. Very hard to change that kind of system.
It does sound like a pointless arms race (between different courses)
I majored in Law but took a couple CS courses on the side so I saw the contrast between traditions in different departments. CS courses had a constant stream of non-trivial graded homework. Even if I knew the materials it took me quite some time to complete them. Law courses usually one essay that counts for ~15-25% (or less frequently, a mid-term test), and the rest is the final exam.
Obviously, both methods work (I guess). But if you're already in an environment where courses give out lots of graded assignments, your concerns definitely make sense.
When I was a college student, I wasn't diligent enough to always do ungraded assignments. I'd do reading, but for actual questions I'd only do them maybe 50% of the time when preparing for a test on the material. And out of my peers I felt like even doing assigned reading and trying to do ungraded homework put me well ahead of the pack.
I think it's a maturity thing. Probably until I was ~24 I just didn't have the executive function to be able to do things like that. It seems beneficial to have graded assignments as a forcing function especially given some college students are literally teenagers.
Also, I took an accelerated math curriculum as a freshman where I went from never having written a proof/knowing how to prove something rigorously, to pretty good at it. The feedback from the assigned homework was absolutely crucial in helping me learn these skills. It's easy to follow a proof from the answer section, but since there are usually several ways to prove something, it doesn't always help just to see an answer, plus you don't know what kind of divergences/hand waves are acceptable or not without feedback.
You can't address the underlying problem that a difference in an A and a A- could very well have lasting effects on a person's life.
You can't address the underlying problem of someone making it to their late teens and being a little shit.
You can't address the underlying problem that some people don't even really want to be in your class but "have" to take it because they want a degree.
You can't address the underlying problem that some students have spent the last 19 years rules-lawyering their parents and always getting their way.
You can't address the underlying problem that any concessions you make for the 20 year old mother of two struggling with two full-time jobs on top of college will also be vehemently claimed by the stoner 20 year old with a parent on the Board and who thinks college is awesome except for the classes.
You can't address the underlying problem that the university gave you a class size three times what it would need to be for you to be able to provide each student with the requisite attention to really address anything other than "did they meet the criteria".
Seems like a trivial thing to say there is an underlying cause. Student should still be failed for cheating. Definitely not like a research physicists job to address a students personal issues
"class of 1400" just means a given course has an enrollment of 1400 in a given semester, not necessarily that they packed 1400 students into one lecture hall and taught them all at the same time.
1400 is huge. It's common at large state universities for the introductory classes to have somewhere between 200 and 300 students. The professor lectures in a large auditorium, and grading (and questions!) are delegated to a staff of TAs.
If you get a good TA and have some good classmates, it's totally fine. Unfortunately, it's common for your TA to be crap, at which point grading becomes a nightmare.
I avoided all of this by taking introductory classes at the community college, where they teach the same material to classes of 25 students.
