So far 3 of the 11 people we interviewed have been clearly using ChatGPT for the >>behavioral<< part of the interview (like, just chatting about background, answering questions about their experience). I find that absolutely insane, if you cannot hold a basic conversation about your life without using AI then something is terribly wrong.
We actually allow using AI in our in-person technical interviews, but our questions are worded to fail safety checks. We'll talk about smuggling nuclear weapons, violent uprising, staging a coup, manufacturing fentanyl, etc. (within the context of system design) and that gives us really good mileage on weeding out those who are just transcribing what we say into AI and reading the response.
> I find that absolutely insane, if you cannot hold a basic conversation about your life without using AI then something is terribly wrong.
I'm genuinely curious what questions you ask during the behavioral interview. Most companies ask questions like "recall a time when..." and I know people who struggle with these kinds of questions despite being good teammates, either because they find it difficult to explain the situation, or due to stress. And recruitment process is not a "basic conversation" — as a recruiter you're in far more comfortable position. I find it hard to believe anyone would use an LLM if you ask them question like "what were your responsibilities in your last role", and I do see how they might've primed the chat to help them communicate an answer to a question like "tell me about a situation when you had a conflict with your manager"
We usually just ask them to share their background, like the typical background exchange handshake at the beginning of any external call.
That normally prompts some follow ups about specific work, specific projects, if they know so-and-so moot at their old company. I call it behavioral because I don’t have another word but it’s not brainteasers and etc like consulting/finance interviews.
I think you (your company) and many other commenters here are just trying too hard.
I had just recently lead through several interview rounds for software engineering role and we have not had any issue with LLM use. What we do for the technical interview part is very simple - live whiteboarding design task where we try to identify what the candidate's focus is and might pivot at any time or dig deeper into particular topics. Sometimes, we will even go as detailed as talking about particular algorithms the candidate would use.
In general, I found that this type of interview is the most fun for both sides. The candidates don't feel pressure that they must do the only right thing as there is a lot of room for improvisation; the interviewers don't get bored with repetitive interviews over and over as new candidates come by with different perspectives. Also, there is no room for LLM use because the candidate has to be involved in drawing on the whiteboard and showing their technical presentation skills, which are very important for developers.
Unfortunately, we've noticed that candidates are on another call and their screen is fed by someone else using chatGPT and pasting the responses, as they can hear both the interviewer and the candidate
I saw a pretty impressive cheat tool that could apparently grab the screen from the live share, process text on the screen in response to an obscure keybind and then run it through OCR to solve (or just look up a LC solution).
At that point it seems like trying too hard, but be aware there are theoretical approaches which are extremely hard to detect (the inevitable evolution of sticky notes on the desk, or wall behind the monitor).
> if you cannot hold a basic conversation about your life without using AI then something is terribly wrong.
I wouldn’t be surprised if the effect of Google Docs and Gmail forcing full AI, is a generation of people who can’t even talk about themselves, and can’t articulate even a single email.
Is it necessary? Perhaps. Will it make the world boring? Yes.
what actually happens to the interviewee? Do they suddenly go blank when they realise the LLM has replied "I'm sorry I cannot assist you with this", or they try to make something up?
Yeah pretty much, they either go silent for 2-3 minutes or leave the call and claim their internet has cut out and need to reschedule.
Just one time someone got mad and yelled at the interviewer about nothing specific, just stuff like I’m not who you are looking for, you will never find anybody to hire.
We actually allow using AI in our in-person technical interviews, but our questions are worded to fail safety checks. We'll talk about smuggling nuclear weapons, violent uprising, staging a coup, manufacturing fentanyl, etc. (within the context of system design) and that gives us really good mileage on weeding out those who are just transcribing what we say into AI and reading the response.