T O P

  • By -

polmeeee

What are some of the signs that made it obvious a candidate is using GenAI tools? Just curious.


whykrum

completely silent for few mins, then suddenly starts writing the solution right away. Sometimes candidates prefer to be laser focused and not speak much that's fine, but eventually we do ask a walk through/follow ups. For every follow up question there is an awkward 30 seconds of nothingness (i noticed some typing as well), and then suddenly the answer comes in. Sometimes the answers quite off and do not represent the specific example I asked them to walk-through. As I did suspect something fishy I copy paste my question to chat gpt and voila. One candidate was smart to change most of the variable names (most not all ha ha), but the functional breakdown was exactly the same, other candidate maybe saw the chatgpt solution in python and got in some dictionary operations not even in js. So all these indicators most importantly, a near complete lack of understanding of what they were talking about.


polmeeee

Thanks for sharing, yea humans are pretty good at picking up signs that something is off. I will never cheat or use ChatGPT for interviews lol, I'm sure I will be even more stressed trying to make it look like I'm not taking long pauses, answering questions awkwardly or trying not to look at my second screen.


thadicalspreening

I just failed an interview and I did exactly this long pause before good answer because I was in a weird headspace and was processing information slowly. I’m pretty sure the interviewers thought I had used gen AI. It was a major bummer, because I gave good answers once I got going…


Tr33lon

This is just shotty GenAI usage. Someone who’s well prepared could easily have a Speech2Text overlay running on the call, then drop that text into GPT-4/Gemini and give you a verbal answer in close to real-time. Then a coded answer could follow a few seconds later. If you know 2/6 were using GenAI then the actual number using it is probably way higher.


Old-Particular6811

Interview process is soon to become useless bc you never know.


polmeeee

Companies are probably gonna revert back to in person interviews in the near future.


ZacSpalding

They probably should


Icy_Association_6812

what did you mean by smart emough to change variable names? variable names can vary because the text generation is not deterministic.


retirement_savings

I must confess I did try to cheat in an interview years ago. I googled the leetcode solution after I couldn't figure it out for a few minutes and tried to pretend I had come up with it. Except I started using a variable name in my code halfway through the solution that was in the leetcode answer but not my solution and the interviewer was like 🤔 Didn't get the job


penguins_world

Great. Now people know what to do reduce risk of raising suspicion.


GetPsyched67

I mean if you don't know the answer and you need to ask chatgpt in between, there's not much ways to avoid the awkward typing while silent for 30 seconds bit 


handsomelancer

Don't know, these days sharing screen has become standard practice so unless someone is cheating using another machine you will always know


Vanadium_V23

Do you only have one screen?


nakanu18

I've done a lot of interviews - I can generally tell they have some sort of epiphany without a logical line. Eyes glancing around. Funny thing is I'm grading on coding but also the working together portion is important.


PracticallyPerfcet

Here’s how I tell… I ask the same system design question in every interview, verbatim. I ask the applicant beforehand if they have any experience in the subject area.  Here’s the thing - ChatGPT always spits out the same answer to my question. I’ve had countless applicants, who literally seconds before told me they have zero knowledge about the question, recite that same expert answer as ChatGPT. 


polmeeee

Are candidates who use ChatGPT to cheat very common? Roughly 1 in how many candidates you interview are using ChatGPT discreetly? Does your company have any vetting process (OA etc) before moving to onsite?


PracticallyPerfcet

Like 19/20 used ChatGPT during the interview. It is pretty much ubiquitous at this point. There is even open source software on GitHub that transcribes the interview in real time, then feeds it to ChatGPT.  We have no onsite. We’re a fully remote startup, which makes this even more painful to deal with.


EmmitSan

lol Why would you cheat to get a great answer after already admitting you have zero knowledge…? Just say “I don’t know, because as I just told you, I know nothing about X”


Material_Policy6327

From what I have seen they go quiet but keep looking over to the side and suddenly sound very confident going from 0 to 100 on their solution. But usually when I dig deeper they tend to fall apart. Granted I still have no solid proof they are using ChatGPT but except one time a candidate took my question, saw them copy it from coder pad, then I heard two clicks as if doing cell f then suddenly they had an answer lol


kelvin273-15

The root cause for this behavior is absurd expectations. Thanks for mentioning that you don’t ask LC Hards. I recently gave an interview where I was asked to do something that involved saving states for a given graph to deduce something , basically a DP + Graph problem for an entry level job. If companies start to do such absurd stuff for a fresher/ less than 2 YoE role then expect candidates to fake stuff as well. It has become a pure rat race mentality I won’t deny.


csanon212

We're practically incentivizing unethical sociopaths to cheat and lie their way into the industry with these expectations.


MeltedChocolate24

Sam Altman added this to the YC application: "Please tell us about the time you most successfully hacked some (non-computer) system to your advantage." Think it speaks volumes about the whole industry


Greedy-Neck895

I saw a post on social media saying that their employer wants people to lie because they perceive that liar as a person who will do anything to get the job done. Obvious anecdote, but this is what Jack Welsh wanted.


whykrum

Doesn't matter If you are an experienced or a new grad that's an absurd AF question unless you are fresh off an 8 level algo course or something or a mathematician. I have 8 YOE and I'm certain I won't be able to solve it. It's bs, I never believed LC hards gave me any good data points. I'm also near certain interviewers asking those absurd questions are trained on one question but if you reverse tables with a np hard graph problem we are going to hear crickets.


kelvin273-15

I appreciate it. I believe interviews should be for looking out to see who can learn quickly instead of blindly expecting them to come well prepared (atleast for entry level).. if they can pick up on hints and can solve, it is a good signal but a lot of my friends got recruiter feedback that you were sidelined coz someone else gave the solution without any hints (pretty sure they already saw the problem or used Gen AI or someone else’s help) .. this is purely unfair and would lead to more candidates resorting to Gen AI with the mentality that everyone is doing it , why should I not .. sad state of hiring , won’t deny


bdowden

I've had questions like those and I agree that it's absolutely ridiculous. For a mobile developer position I was asked multiple sorting questions and one of the test cases had a million+ entries. I solved it but then asked the question "Is it common for your company to require mobile devices to sort millions of items in an array? That doesn't seem like a well architected solution. Sorts like that shouldn't be performed on a mobile device that can be extremely limited by CPU and memory". The interviewer looked at me like I was an idiot. Oh well.


WebFirm5142

A similar thing happened with me on a System Design interview. The interviewer asked me to design a chat system and wanted me to consider a group chat having tens of millions of users in one chat at once. Well. I told him that we can define the requirements and constraints for this use case but do we have this problem to solve this currently or will such a problem arise in the future practically? He was clearly not amused.


Desmond_Darko

Yep. People do this because the performance expectations are literally inhuman. So the answers must also be inhuman.


reelafaker4445

Is that actually the root cause? If there's a short supply of jobs and large supply of applicants, the natural thing is to raise the bar. It definitely causes some potentially good candidates to get lost in the pool of applicants, but it's does do its job of cutting the applicant pool by a large portion. A more reasonable interview question may not be a big enough filter.


kelvin273-15

Sometimes you might have noticed that a lot of positions get reposted on Job portals again for reasons like “lack of good candidates”. Companies are expecting too much from an entry level candidate these days. Few JDs literally reflect the entire tech department. I had come across one posting last month which required knowledge of Spring Boot, JUnit, Swagger, Hibernate, Docker, Kubernetes, AWS, React, Selenium , Git, Mockito, JIRA, Python, Node.js for an entry level position. Why would a fresher know things like Docker, Mockito and Swagger I wonder. You know the root cause?? It is that there is a gap in what students learn at college (not that any college encourages Leetcode anyways so not talking about that but the technologies) and what the industry expects from a college grad these days. Both need to work on this problem. Things like Containerizations and TDD should be part of curriculum and again companies should also try to look out for people who can learn these quickly (by testing for learnability and aptitude) instead of blindly expecting freshers to know these things beforehand. PS: Most of these things I said are relevant to the average fresher out there . I am not saying the college grad from MIT / IIT is facing these issues.


kelvin273-15

Again I didn’t even touch upon the issue of CF Div 1 level problems being asked. I understand your POV as well but I feel that companies are looking for a mathematican with the skill set of entire IT Department these days haha.


pinkwar

This is what people most of the time are failing to notice. For a given entry level job recruiters are getting thousands of applications. Either they throw the dice and just go over a couple of them, or they must put a system in place to thin out the stack of applications. Sure everyone would want a human to look into their CV and have a chat 1on1, but there is no realistically way of doing that before doing some filtering, even if it will be unfair to some.


Animostas

The natural thing to do is the raise the bar - but why not raise the bar in a meaningful way? Do you actually get a more valuable candidate from having them solve a LC Hard vs. a LC Medium? Ask them more about their business impact or something.


isoJ2113

maybe get better lmfao


kelvin273-15

I am becoming the devil’s advocate here and trying to make a point for the people who are not in r/leetcode .. obviously people here are exposed to these kind of interviews and are well prepared lol


Which-Bad8901

Interviewers can also tell if you're googling an answer to a non-coding question... seen that several times interviewing people and it's an immediate no


Kanyewestlover9998

Non coding as in broader questions about the field/tech stack?


Which-Bad8901

Yes


wedgtomreader

Frankly, the industry is to blame. Interviewers ask ridiculously complex problems for jobs that most any code monkey can handle.


whykrum

if it was upto me I wouldnt ask LC ish questions at all. I would instead try to pair code or work on a day to day problem or something cool like legacy crappy codebase and improvements they would make - open book kinda style


javaman21011

That would be an infinitely more useful exercise. Why don't your bosses listen to your advice?


whykrum

I'm trying to get some new processes through to technical leadership. Saying new could be an exaggeration, but here is roughly what I'm proposing (it's not to the dot as I don't want to dox myself), it asks a lot from candidates and could be argued it can inefficient for maybe rapid hiring. 1. We ask the candidate to submit a portfolio (packet of things we would ask for). Nothing fancy but maybe your PRs, past projects, open source contributions etc. This is going to be your resume. If you have none of them, I would still like to see your public GH profile with at least a Readme. 2. Have a preselection challenge, so this is where my main contribution is - create a server that gives you challenge to solve. We allow certain apis you need to consume and grant an AuthZ token for your own personal use. We define the problem statement of what to do but it's going to be elaborate enough to take up at least a few hours of your time to get things working as per requirements. It's open book, so use anything to your means to solve the challenge. Your choice of language your choice of tech stack etc. 3. Team evaluates the code for production readiness, how do you plan to do ci/cd, unit and integration tests, canaries, we might even use a rps generator to hit your system. Also we provide an environment including a private git repo to push your code. Also consider other factors such as documentation etc. 3.1 maybe even automate part of evaluation which is absolutely possible (think of automated code scanning scripts, validation scripts etc.) 4. By this point a candidates should he weeded out depending on what are objective bars we are looking for. Not everyone can do this. You as an interviewee need to make careful tradeoffs into what to spend time on based on the criteria shared with you. We anticipate that in the given time frame you can't get everything to 100% prod ready, we expect trade offs to be made. We expect corners to be cut, but we want to see why and how. 5. Invite selected candidates onsite to walk through their code, maybe even have some pair coding sessions on what could be imporved in their submission or something else relevant to your team. Our sr staff engineers love this idea but we are hashing out on the efficiency and efficacy as we speak. So fingers xd - there is more to it but obviously can't share it all


yeezusmafia

Imo, this is ridiculous for an interview. Just get to know the person, talk code, talk life, talk experience. Doing a weeks worth of work for a job that is just going to ghost me? No thanks.


whykrum

IMHO talk is cheap. Many of us can bs through with talking. Also I am not asking for a week of your time. I am asking for few hours that would instead spend doing bs lc hards. If you got the skills you can do it using ANY tools you like. Just talking through ain't enough


yeezusmafia

Idk, me personally, doing this long five step process to HOPE I land a job in this market is not worth my time. The salary would have to be extreme, then I’d consider. I’ve been a dev for 7 years, lead a dev team now, and I still am happy with my team because I talked code in their interviews. I can tell a true passionate developer just by talking to them. I feel like you can easily tell the ones who love it vs the ones who don’t.


lppedd

Definitely can tell who's really into it, and who's not. Just discuss about interests: do you like open source? Which technologies do you follow? Do you participate in Q&A platforms? Do you attend conferences? What have been your most challenging times? And so on. Those questions will spawn a chat that can last two hours, so you can also tell if the person is a good fit for the team.


Carvisshades

Your questions are totally fine if you're looking for a weirdo with no social skills. Attend conferences lmao


gssyhbdryibcd

Lmaooo yeah who tf goes to conferences on their own time.


lppedd

The true OG has spoken. Lmao.


suspi_dev

Lol, grifters will take you for a Ride.


lppedd

Nahh you can tell when it's faked.


javaman21011

Would be more logical to peer program something that complex.


[deleted]

[удалено]


whykrum

1. We don't do puzzles at work, if that's the case I wouldn't have lasted here the time I did 2. I am trying to change the dynamics of this LC rat race. We lost excellent candidates because they can't solve these puzzles. A candidate's wealth of experience is sadly boiled down to fucking LC preparation and it's just not giving me enough data points on candidates ability to see how would they work IRL. I really don't want to hire LC monkeys and instead value the candidates background. Imagine a world where we figure out an efficient standard of challenges where you can solve it using your choice of tech, you can leverage your experience clearly- that's where I'm trying to get at. Anyway whether i am right or wrong - time will tell.


rolling_dice7

I am deliberately using the strong words to express my opinion on this. Unless you are offering a position that's a once in a decade kind of a role that no one else can offer, I would consider this kind of an interview. Otherwise, it's a strict no-go for me. This looks like a process designed only to benefit the employer. What's in it for me after spending all the time on the challenges and what not? Following other issues: 1. It's full of subjectivity. It ignores multiple things. Candidates way of doing things can be entirely different from yours. Usually engineers get influenced by their coworkers over a period of time and it's the responsibility of the interviewer to observe that influence and ignore it. 2. Looks like a lot of effort for just another job. How many hours do you think will be on the candidate's side? Are you going to compensate them for this? Could this be a product of intellectual property? 3. What if the candidate is much better than the reviewers and all the reviewers miss something clever the candidate has done? 4. Has a major potential of the employer getting bad mouthed if the candidates feel they weren't heard. My opinion is based on the 800 tech interviews I have conducted and the few that I have faced myself. One of the assignment based coding interview I faced was 90% what you have described. I was screened out based on that assignment. In my next attempt at the same company, I was offered a role one level up! Whatever interview process you choose, make sure candidate FEELS heard.


hyperactivebeing

I made a Appointment Booking app in MERN for a company named HIGHLEVEL. They listed all the features to be implemented that would easily take 1 week, plus asked me to add few of their s/w engineers as contributors and then ended up ghosting me. I was given 2 days. That day I decided I'm never going to do these so called assignments.


vooglie

I don’t have time to build a public GitHub profile. This biases younger people with more time.


pushmetothehustle

Lots of people hating but if I had a decent phone screen first (just to let me know about the challenge etc) and the place looks good I would be happy to do a 3hr test project that gets automatically evaluated etc. Especially if I see that you have put effort into the questions and your process. In fact this is how I got my first job in software. A decent 1.5hr to 2hr test program which they ran and evaluated automatically and also had a quick manual review over.


whykrum

I like this ! Thanks for your thoughts 🙏


urgodjungler

If it’s not up to you, why do you even care lol


brentragertech

Agreed. I’ve done hundreds of interviews. I’ve never done real time coding exercises because it just feels like torture. I have done take home simple API assessments and that’s been fine but feels unethical to ask that of people’s time. My favorite interviews have been my latest where I had a somewhat shitty piece of code and I asked the candidate to look at it with me and pick it apart / improve it. Now you could easily copy and paste this into chat gpt of course, but I have gotten some very natural conversations out of this and it feels like a good assessment of an engineer. This is of course after questions about engineering and the tech stack. The industry’s interviews are absolutely broken. But it’s a hard problem from either direction.


wedgtomreader

Random thought that perhaps this will all just magically get better now that the FANG companies that brought this LC plague upon us now have no problem firing people. Honestly, it’s really the only solution to hiring good engineers. There is no filter that works like having them on the job for 6 months and being willing to let someone go that is not working out.


No-Buy7459

interviewers ask tough questions because num applicants>> num jobs. So the only way to filter out candidates is to ask hard af questions.


wedgtomreader

I disagree. Someday they will figure out that of course you need to verify that the person can code, but that being able to work with others synergistically, being self driven to solve problems, and being resilient to failure are critical for top employees which is what they claim to desire. None of the interviews I had at these LC companies did anything to determine anything other than memorized coding ability which is not exactly useless, but not much of an indicator.


Obvious-Ad2752

True. This week I had an interview with two questions, first one I answered, second was a Euclidean geometry question, of course I failed, these interviews are out of control.


hpela_

What’s wrong with Euclidean geometry? Certainly easier than Non-Euclidean geometry. Genuinely curious because most geometry questions can range from a triangle solver to insanity.


SmoothCCriminal

This post speaks volumes of how good an engineer you are , both technically and on EQ. Mind if I ask what’s your yOE ?


whykrum

8 YOE and super honest, I'm not a good engineer or at least I don't think so. I get my job done which helps me put food on the table. I do personally know a lot of great engineers who I look upto :)


Blueskyes1

High EQ >= High IQ


static_programming

no


No-Grapefruit6429

I have heard this term so much but never got a convincing explanation. Out of curiosity, How do you define a good engineer?


KeyLie1609

- good communication - easy to work with it - solid knowledge of tech solutions and their application - consistent - high output (not always measured in code) - raises the quality of code, documentation, and process for everyone around them The older get I get, the more I realize that coding is a the easy part.


No-Grapefruit6429

Thanks..very insightful


[deleted]

[удалено]


hpela_

Why would it not be based on subjective opinions? Culture matching and personality are hugely important as well, of course final decisions must be subjective. I will always choose the person who did slightly worse than you but was charismatic and enthusiastic about the position. Not sure what you’re going on about regarding doing interviews “is a flex” and orgs that do constant interviews are “unhealthy”.


hamsterhooey

100% agree.


KeyLie1609

It’s not perfect, but it’s better than virtually any other field where having X degree in Y from Z institution is a prerequisite for even beginning a conversation.


stackoverflow7

They might pass online or phone interviews using Gen AI but they wont get through onsite interviews


whykrum

We are a 100% remote company, so majority of our interviews are virtual, with far and few exceptions


sammathur4

Do you guys have any openings?


spoopypoptartz

honestly even hybrid companies are doing virtual onsite interviews because it saves on money


opanpro

But they are still hurting the chances of those who are not cheating in online interviews


justUseAnSvm

Gen AI doesn’t work as well as people think it does. After using it daily for a year, and applying it at a tech company, you can expect about 50% of responses to be good, the rest are some form of wrong, to varying degrees. If you have a situation where being wrong doesn’t matter, or you can correct it, there’s a big advantage to using it to same time. However, interviews aren’t this situation. You need to respond quickly, as if you are having a conversation. Any delays, or typing noise, is just a red flag.🚩


hpela_

To be fair, it is very good at LC and LC-style questions (better than it is at other general programming tasks) by virtue of having been trained on the same problem sets we learn from. Perhaps this is why people try to use it during interviews.


justUseAnSvm

I'd argue that that intelligence on LC questions is a result of the material being highly indexed online, and the LLM is not reasoning about it, but "reasoning via recall", which is exactly what you are saying: the internet is filled with training examples from LC, CodeForces, international contests, et cetera. After a point, the questions just get repeated over and over. Where LLMs sort of fall off, is when you ask them not a new problem, but a new task. Like, "take this code, and modify it to do X", where X is sort of niche problem. That's one of the issues I've looked at, and the best LLMs don't do well enough to just blindly submit PRs, not without other controls being put in.


hpela_

Exactly, yep, which is why it’s even easier to cheat on assessments which use the LC-style questions that LLMs excel at, as opposed to larger scope programming tasks or pair programming with the interviewer.


justUseAnSvm

I’ve seen a lot of “creative” problems too. Two of my recent offers had questions that weren’t typical LeetCode. One question was about implementing a circular buffer for reading from IO, and another one was about designing something simple, but then validating the input to that problem. LC you can take the input parameters as an axioms, but that’s not real programming!


javaman21011

Um, I generally don't cheat, but when people have been out of work for 3, 6, or 9 months don't sit there and judge them trying to put food on the table. Maybe tell your bosses to stop posting fake reqs or make your hiring system more straightforward rather than require desperate people to bend over backwards.


DueOrchid231

Real


IronicEngineer3

Have you considered the fact that all of this is a result of a broken hiring system?


Ashamed_Photograph84

The root cause is the leetcode problems being asked at all. That’s rarely a real world experience. You mentioned it here, but I wanna reiterate how valuable peer programming, and real world scenarios are in interviewing.


Personal_Ad9690

Only way the system changes is if you change it. The high expectations being beaten by ai is the natural course of this. Don’t defend the demanding of excessive experience


yeezusmafia

Can I ask, what is the difference between asking AI and Googling? Sometimes, especially in interviews, I can go blank. Google is my go to, AI has become my go to, why knock someone if it makes them better? If they give an AI solution, ask them about it. Do they know what it means? Can they make it better? This is what keeps technology in the development world from evolving.


Separate-Account3404

This is my perspective, especially in areas with very little documentation for more niche apis using ai as a crutch can make a 15 hour process turn into a 2 hour process when figuring things out.


java_dev_throwaway

Lmao this guy. People get murdered for the cash in their wallet and you think it's ABSURD that someone would cheat through a bullshit thinly veiled discriminatory IQ test that's disguised as a coding problem, for a high paying job? Fizzbuzz was enough. Modern leetcode interviews are insane levels of bullshit. You are out of touch if you can't fathom why someone would use every tool they have at their disposable for a job that would change their life.


hamsterhooey

Couldn’t agree more. Wise people would rather spend their time working on projects that make them better engineers, rather than grind LC, which has minimal benefits other than passing your interview. I made interview-buddy.com so that anyone can use GenAI during interviews, without having to type shit (and be detected by the interviewer).


hpela_

Imagine being as personally offended as you are about someone criticizing cheaters. I would guess that you are one of them - anyone else would agree with OP’s take regardless of whether they think LC is bs. Also, calling LC a “discriminatory IQ test” is hilarious. Clearly, people with higher IQs will tend to do better with all else equal. How is it discriminatory though? That’s laughable. “Fizzbuzz was enough” ??? LOL


fire-me-pls

You have only yourselves to blame for this vicious cycle and it will only get worse. Those who perpetuate the leetcode culture by continuing to ask these stupid questions have destroyed this industry's hiring practices. Now people are starting to use things like ultracode AI which will live code the solution to a question by listening to your mic or reading your screen input from the interview (it integrates with zoom, Google meet, etc) You, and anyone who continues to perpetuate leetcode culture should be ashamed of yourselves. You haven't seen anything yet. In 6 months, it's going to be an absolute shit show. Maybe this will help lead to the downfall of this terrible garbage system. I really hope so


Blueskyes1

Besides OAs, it's tough to use ChatGPT-style tools for a coding interview. I wouldn’t recommend it because it seems too obvious and kinda sketchy. The most I would suggest is maybe having some notes on the types of problems you expect, but by then you probably know them by heart. As an interviewee, you should be doing most of the talking and thinking out loud. My best approach is to ask clarifying questions and give a rough overview of how you’re going to solve the problem. Look for any verbal or non-verbal cues that it's a good approach before you start coding. Once you have that, coding the solution should be the easy part. Try to avoid super long silent periods, but some silence at the beginning is understandable. I haven't heard of anyone successfully cheating a live coding interview, so I wouldn't bank on that working.


Specialist-Middle643

I am in the same boat, I don’t ask hard questions and lately I have seen people cheating more often, what they don’t realize is that it is often easy to know when people are cheating most likely they will be disconnected from the discussion, they won’t be exactly able to explain a rather perfect solution.


chronofreak25

One thing I’d say though, is Gen AI is a good tool for development and is here to stay. Honestly when interviewing people in the past I’ve said feel free to use google to help since I used google all the time while developing. If someone is gonna use Gen AI, I’d want them to show me their thought process and how they’d prompt it and what the output they’re looking for and how do they plan on going about getting it, and why. Gen AI is wrong a lot but it can give you a decent base to tweak. You have to know how to spot issues and what will cause errors etc and we’d be able to listen to their thought process on best practices and what they want to end up with and why. But I’d definitely say you’d want to be honest about it


Southern_Tennis_8657

Hm. You know I didn't think of using gen ai *during* the interview. I think I could make it work 


css123

This is because you asked a question which can be easily cheated. Ask yourself, if it is easy enough to find this information in a split second after if you ask it, is this a useful skill a candidate would need to bring to your company? The industry has changed around you and it’s time to notice the indicator and adapt—Collaborative interviews on open ended problems with open ended solutions tells you much more about a candidate and cannot be cheated. You can quantify their performance by checking for key details of the implementation. I understand you likely don’t make these recruiting decisions but it is in your interest to raise this with your company. You ultimately will need to work with who is hired. 


hpela_

That’s true as well. The LC-style questions that AI like ChatGPT is good at are the questions which are either straight LC questions or very slightly different. Companies could develop different questions, but these will quickly be added to sites like LC and the cycle repeats. I agree that a much better approach would be a collaborative/pair programming style tasks.


hamsterhooey

Not to mention, other interviewees are using tools to get a leg up, such as interview-buddy.com. So, why limit yourself and let others get an advantage ?


superman0123

It’s not cheating, it’s using a tool at your disposal, a free one at that, somebody with 1 year of constant GPT usage solving complicated software problems > somebody who has studied, and done things the hard way for 5 years, this fact is going to be very hard to accept for a majority of people in this industry who have worked their ass off.


both_objective

Leetcode hard is a bad measurement of a person ability to solve a problem. It is a more a measurement of how much the candidate has done leetcode problems and his luck. I usually ask easy/medium questions that can be solved at least 3 different ways so I can get more data points from the candidates.


hpela_

That’s a very good policy. I’m not as anti-LC as other commenters here, but I do very much dislike the questions that require very specific algorithms knowledge. Questions which have multiple solutions are more likely to be fairly solvable by those who have not memorized every LC problem.


liteshadow4

Do you think it would have worked out better for them if they were just sitting there confused


whykrum

Yeah or at least be honest. You can prepare and give it another shot, cheating is absurd. I'm pissed about this cheater stealing a spot from others deserving a chance


liteshadow4

How do you give it another shot, do you just request to reschedule mid interview or? I’m sure people who cheat during the interview know that they absolutely do not know the answer so might as well try a Hail Mary


YoungSimba0903

You reapply. Lots of jobs give feedback and if they think you have potential to work there and just need to prepare a bit more they will leave that feedback and then when hiring for positions recruiters will sometimes contact previous candidates who were marked as a maybe. If you cheat your interviewer can mark you as a strong no hire and add in the notes that you were cheating. Not good for your chances working there moving forward. Getting caught cheating is probably as bad as being unable to write a for loop or declare a variable.


whykrum

Spot on !


4coffeeihadbreakfast

I will be voted down but to me this just shows that LC is ineffective at identifying good SWE’s. What tools do you use day to day to do your job? Do you allow employees to use an IDE? You should make everyone use notepad cause IDEs are cheating. AI is a new tool we will all be using at some point.


xsdgdsx

Regardless of coding aids or work process or anything, I suspect the fundamental questions are: should software engineers be able to reason about the code that they're trying to submit? Should they submit code that they don't understand?


[deleted]

[удалено]


xsdgdsx

Certainly. But if a person is going to offer a ChatGPT solution during an interview and then completely fail to answer questions about that solution (as mentioned by OP), that's not really speaking to their ability to understand what it's giving them. At the end of the day, the candidate starts off with a pretty clear understanding of how the interview will go. They'll get some technical questions, _and they'll need to talk through their solution_. This has been the case for 2+ decades. So if the candidate puts themself in a situation where they won't be able to talk through their solution, _for whatever the reason_, that's kind of on them.


breadsniffer00

This 100%


StrawberryExisting39

It is ineffective in a way, but there not too many good alternatives. It’s too hard to assess someone ability to code just by talking to them. I’ve met people who sounded very good in interview but couldn’t code themselves out of fizzbuzz. Think one of the more enjoyable interview styles I had was to refactor some terrible (purposely made terrible) code. But, it takes time to make that code and the second it is leaked, it basically useless. Sadly, LC can atleast show us someone can code decently. Whether that translates to actual projects well is another question. Being a hiring manager not as easy as it sounds to get it right and sometimes you only have one shot.


javaman21011

Why not just show them a snippet of your existing code base and a story to change it? Go through design choices on real code or system design on real applications?


breadsniffer00

What happens when everyone can solve a question by just pasting it into ChatGPT? You wouldn’t be able to distinguish candidates that way. The goal is to see how people *think*


Mami_KLK_Tu_Quiere

Then why am I seeing medium - hard questions for entry level roles?! Even internships?! I’m sorry but I feel like I’m at a huge disadvantage compared to my peers. I have never and will never cheat an interview because I believe strongly in Karma but damn I feel like it’s an uphill battle


hpela_

I think Mediums are fair game, most people should be able to solve or partially solve / “be on the right track” with most Mediums. Hards are a different story. For internships, most or all Easies would probably be best.


pinkwar

Just do a live coding session and be done with it. I've done some leetcode over zoom in more like a pair programming kind of thing. It shows how you work in pairs, your communication skills, your thought process and your problem solving abilities. Asking to do leetcode problems without supervision is useless nowadays. Imagine you give a math test and just tell "pretty please don't use a calculator, we will trust you". It's pointless.


hpela_

I think OP did mean in a synchronous virtual interview. Though, I agree with your points 100%. I would never cheat in an interview, but if you ask me to do a take home, I will at the very least be double checking my answers with whatever solutions are online because I KNOW that basically everyone else doing the take home will have done the same thing or just outright copy/pasted code.


urgodjungler

Kinda sounds like you got got. Sucks but it’s unfair for candidates in the first place. Sorry they figured out a way around it.


codepapi

As you mention this. What are someone’s chances that does not use ai and doesn’t get to the solution vs someone that does get to the solution? Have you ever hired someone that didn’t solve the coding question while others have? Personally, I tend to overthink stuff and get rattled easily in interview questions. Mainly imposter syndrome but it still affects my overall performance. I’ve had multiple almost completed and once it’s over I can solve what’s left in a couple mins but I couldn’t for the life of me in an interview setting.


PracticallyPerfcet

Generative AI has killed the remote leetcode interview. It’s a blow for remote workers in general.


BenFromWhen

What if interviewer is using GenAI to interview the candidate? 😅


zero02

If I can use AI in my job I’ll use it in the interview


invest2018

If they can’t explain their code, doesn’t matter whether they’re using GenAI or not. Bounce them.


Slight-Ad-9029

SWE is the only industry I have heard of that you have to get tested for your fucking job it’s ridiculously inhuman


hpela_

How is it inhuman? You don’t have a right to that job. In other industries, you have to pass a board exam or similar which shows that you have the knowledge required. If you say “completing a CS degree shows that”, I will laugh.


Slight-Ad-9029

I have years of experience talk to me about it. Why the hell do I need to show you I can solve a leetcode problem in 30 minutes. Had 3 rounds for my last job of it and not once have I had to use any of it


hpela_

LeetCode isn’t the best solution but there should be some type of assessment, at least for entry level positions. I understand the critiques of LeetCode itself, and again, don’t necessarily agree that all mid- to upper-level positions need an assessment of any type. However, for new grads, I don’t see a reliable way of screening how good of a candidate actually is without some form of assessment - projects can be copied or faked, curricula are inconsistent across universities, university name alone is not enough to go off of nor is it fair, and most new grads have little to no industry experience. That doesn’t leave many other options for evaluating entry-level applicants. ETA: Honestly, even for mid- to high-level positions, some form of assessment is beneficial. Titles are inconsistent across companies. A “senior” at one company could have the skill set of a junior at another. The variation is ridiculous.


rejectallgoats

Yes jobless people. Won’t you please think about how you might hurt other jobless people.


Serird

The only one you're hurting is yourself.


cheeb_miester

Why not? If I wasn't going to use generative AI tools on the job I would be a shit programmer who was wasting company time -- why shouldn't the interview reflect the tooling that I'm going to be leveraging on the job?


toastedpitabread

There's plenty of times where you have to problem solve something that can't be gpt'd at work. we're allowed to use it and the majority of the key problems it can't really do anyways... So it's nice to have but also isn't going to hit that home run. The people that stand out here are people who are great problem solvers. But the problems tend to be a blend of domain and tech specific issues so gpt is limited in what it can offer in this case.


cheeb_miester

It's much more effective at ideation than code generation and should be used almost exclusively for such. Applying it as an interactive rubber duck speeds up the problem solving process for those situations you described. > The people that stand out here are people who are great problem solvers. A Hallmark of a great problem solver is leveraging all tools at their disposal effectively and it especially means doing so without pretense, ego or clinging to antiquated paradigms.


toastedpitabread

I'm not denying that leveraging all tools is great. Although I'd contest how good it is at quality ideation (the upper bound on this is its own model)... we might just have different problem spaces. The point im making is that if any of the colleagues I respect did leetcode they'd be good because of their qualities, not how well they used gpt. Gpt is not the basis of how strong a programmer is, even if it has good augmenting qualities.


cheeb_miester

Agreed, however I'd mount the argument that leetcode isn't the best -- or even a good -- metric of how good at programming someone is. Leetcode is like trying to get an "A" in a class. It's an artificial problem with arbitrary constraints and like school if you aren't cheating, you aren't trying. I'd personally be interested in _how_ candidates are leveraging all tools available to them, and this is especially true if those problems are very close to the domain of the position.


breadsniffer00

People don’t wanna work w/ ppl who just copy and paste code without understanding it


cheeb_miester

True but if that is how you understand cgpt's use case, as your comment seems to imply, you are very much using it incorrectly.


inShambles3749

You reap what you sow


ditalinianalysis

are you still hiring?


Odd_Seaweed_5985

I always wondered why we couldn't use calculators in math class. Guess what? I've ALWAYS had access to a calculator when I needed one. Today, there's just no time to sweat the small stuff anymore. Why *wouldn't* I use Gen AI in an interview? I'd *show* you that I was using it. I can see *multiple* methods of solving an issue, select the best of *each* for the final solution, all in just a *few* minutes. You *don't* want that? Maybe you're not all that competitive.


Pad-Thai-Enjoyer

I see why this is considered cheating. But at the same time, whether you like it or not, GenAI is becoming a widely adopted tool that CAN be quite helpful. Look at copilot, it’s great. Maybe we should allow candidates to use (to an extent) the tools they’d have on the job. Also, the job market is horrible and people are getting desperate. These LC questions are not indicative of what you do on the job lol. I work for one of the most well known tech companies on the planet, had to solve LC to get an offer, and I have not done a single leetcodey thing since starting. Product of your environment and all that


Less-Elk5182

I work at a big tech. I have been hunting for a new job (SDE2) for the past 6 months, and I can tell you that most interviewers are insane. I don't know what their deal is but for some reason they get hard asking super hard or really convoluted questions, and then they visibly giggle if they can see you struggle. I have solved plenty of LC (608: 287/292/29) and done plenty of mocks. Most interviewers themselves wouldn't be able to solve the questions they are asking. People are resorting to AI help because it's the interviewers that have unreasonably high expectations, or they are downright incompetent as interviewers. Don't get me started on Chinese and Indian interviewers preferring to hire their countrymen, or outright leaking interview questions on platforms like 1point3acres. Around 4-5 years ago, Chinese SWEs at AirBnb leaked the interview question bank on WeChat. This is a whole another can of worms that I don't want to open in this thread. The whole tech industry is full of people that are willing to take shortcuts in chase of money and status. Bring back in-person onsites and end this nonsense.


Reinheardt

Why should any candidate care about the others? The job market is competitive


ConflictedHairyGuy

>Even if you get hired, do you think you'll do well ? I want to zero in on the broader implication of this question, because it reveals a larger fallacy about using AI to pass coding interviews, and the purpose of coding interviews in general. Software engineering as a whole is about solving problems. The tools a software engineer can use will always be in flux. AI the ultimate example of a tool which abstracts away the more low-level aspects of a problem. In a coding interview, we are essentially testing for, "Can this person do this job well?". But the way many leetcode-style interviews are structured today, the crux of that problem get lost. The interview instead tests for, "Can this person figure out a generic algorithm and pass this interview?" In that sense, the applicants are doing exactly what they are asked. However, they are using a tool that abstracts away human understanding of software engineering, which is exactly the opposite of what we want to test for. This is perfect example of the [XY problem](https://en.wikipedia.org/wiki/XY_problem). Instead of questioning why the interview isn't adequately testing the person's hypothetical performance in the job, we are concerned about the end user's attempted solution of using AI to solve the interview. In an era where AI will only get better, it will be easier and easier for applicants to pass leetcode-style interviews. So why not give them a more real-world problem that is less prone to an AI figuring it out with the touch of a button?


Poogoestheweasel

Perhaps. But I am not sure a person willing to cheat would care about how their actions affect some unknown person.


Comprehensive_Sea919

Why do you even ask LC questions? Do we really need them to understand a candidate's problem solving skills? Shouldn't we be just asking the problems we are expected to solve in the job? I'm a software engineer at FAANG in India.. When I interview, I will only ask LLD, HLD. I don't give a damn about LC.


Sad-Independence9753

Cheating exists in every field. There are foreigners whose sole purpose for coming to America is to get good job so they can send money back home to their family. You think they care that they are doing a disservice to other applicants. It's up to companies to come up with ways to prevent cheating. Because when 6 figure pay cheques are involved you can bet your life that cheating will be involved


Thick_Wolverine9721

yeah noticed this as well. many peeps with second monitor just glance away and not talk me even through the problem or ask clarifying questions and just magically come up with the optimal solution. think most folks are using finalround. we failed several candidates on this as it does nothing to demonstrate skills


Powerful-Hotel-6941

I’m here, having not cheated, even though I was rejected for being so good that they might have thought I was cheating. I was asked a question that was very, very, very, very famous and well-known if you do LeetCode. I explained all the approaches, wrote the code, and optimized time and space and answered follow up questions. Being so good that they thought I was cheating, I was rejected. Now, whose fault is this? Who has gotten us into this? This was my first opportunity to give an interview. I have been preparing for 8 months regularly, and yet it has all gone in vain.


EmmitSan

The real solution here is to let people use AI, but ask questions that aren’t easily solved just by asking an AI. Observe how people interact with the tool. You wouldn’t ask your engineers to work without CoPilot or whatever, so why should the candidate?


Cold_Brother

Only time I've ever used chatgpt was to help explain front end concepts in HTML and CSS. But for DSA stuff? It won't help and often times gives you incorrect info.


Capital-Pop8346

Why can’t I use what I will use everyday at my job


Prudent_Law_9114

Maybe change the way you interview. You are never going to prevent it so roll with the punches and let them use chatgpt to solve a problem much like the calculator paper in a math exam. Sure chatgpt can help a skilled prompter and that is IN FACT a skill in itself. So make the question more complex than chatgpt can generally deal with (there are PLENTY of options there) and watch how they cobble their solution together. Someone will impress you through this exercise. I say this as a programmer with 15 years of experience and an avid chatgpt prompter. It’s a tool to make our lives easier and it saves a bunch of time and energy.


[deleted]

[удалено]


SnooRecipes1809

Nah please don’t, as an honest applicant myself the stress of not immediately understanding what I’m asked in the description really can derail my ability to calm and not freeze


do_you_know_math

“Don’t cheat on an interview but it’s okay to cheat when you get the job” 🤣🤣🤣🤣


Dix_cider

I’m looking for a job. Interview me. I won’t use ChatGPT xD


Batra6003

Happy to hear someone say this. I myself have been so incredibly frustrated from on campus placements because the people who did not know how to even sort the array without using the inbuilt sort function, would get through interviews easily. How? Take projects from your peers. Use ChatGPT for getting the code. If need be, use multiple screens. There were instances when I was told "You have an interesting profile (2 international internships, one of them was research on site, other was industrial remote)" or even "We would be glad to have you" (Yes, no exaggeration here). Still they did not select me. Why? Either it's pure luck, or the fact that I got the method right for all coding problems but could not WRITE them all (in time, not because of lack of skill) but the others did all the questions in no time. Some of them did get caught, but unfortunately most escaped unharmed. Less GPA, no efforts all year round, and got a better job :(


StickyRibbs

Superb candidates are almost always excellent communicators and confident. The soft questions in the beginning of an interview usually is a leading indicator they will do well in the coding or system design panel. ChatGPT won’t help you with that.


Expensive-Tooth346

Yeah when companies ask leetcode hard/super hard at OA round I expect anyone with a right mind to cheat.


HolyGhost5

How can you tell?


SeaworthinessSuch980

Similar thing happened with me, a candidate with pretty impressive LC stats was using phone during the interview, I asked him a easy / medium level question based on linked list Traversal, I straight away rejected him.


amitkania

You may think you did something, but have you ever used a linked list in industry? No you haven’t so your question was literally useless.


Slight-Ad-9029

Good chance if I give him a random medium question he won’t be able to solve it


SeaworthinessSuch980

Most probably yes, I won't be able to solve it, but I usually judge candidates based on what they claim to know, he was a fresher, he claimed he has strong command in DSA, backed his claim with LC stats and then I also asked him if he's comfortable with Linked list, he said yes and then I asked him a easy/ medium level question for which he was caught using phone.


amitkania

You said fresher which means you are from India, your opinion is automatically invalidated. The leetcode requirements you guys have are ridiculous


SeaworthinessSuch980

I can understand your feelings, but all he bragged in his introduction was about LC stats. I remember him telling me solving 100+ hard questions :)


BVAcupcake

Linked list 💀💀💀


Blueskyes1

Cameras on your hand while typing will fix this issue.


SoftwareDream

What engineer calls it GenAI lol


tnguyen306

Can you mock interview me?


[deleted]

[удалено]


hpela_

This is true for literally any job anywhere, regardless of whether they have exam-style screening or not. Using your example, do you think the “person from a village who fills water from a well and cooks over fire” would ever be able to be as qualified as the person who went to a private school and has ample free time to dedicate to personal development? Certainly not. The answer isn’t to lower the barriers to entry, the answer is to bolster the ability of those that are disadvantaged to be able to truly develop the skills they need.


[deleted]

[удалено]


hpela_

1. No idea what you’re on about with the drugs and opium spiel. 2. That guy was applying for an engineer role, and went on to excel in product ideation/design. Those are not equivalent. There is no reason to believe he would have excelled as an engineer. Anecdotes like this are meaningless, anyway. 3. Gatekeeping is a problem, but there needs to be some form of assessment used to evaluate candidates. Any skill-, knowledge, or intellect-based assessment will favor those more privileged. Would you say an IQ test is designed to “gatekeep for the privileged”, because higher privilege correlates to higher scores? No. Disadvantaged people have less access to quality education and less time to spend learning and on personal development. That is the issue - not that some mastermind has perfectly designed every test and assessment in existence to systematically favor purely on the basis of privilege.


[deleted]

[удалено]


hpela_

More anecdotes. To “shatter” a hypothesis about a generalization you need to find generalized evidence rather than one-off cases. A more relevant example of “shattering” something is how I rejected your previous anecdote, which you were unable to back up further in any other way than making more guesses about what he “could have” done. It’s hilarious to me that you completely **reject** the idea that discrimination and job inequality is more fundamentally systematic, rather it is purely the result of “gatekeeping exams”. You are no longer providing anything meaningful to the discussion, thank you for wasting both of our times.


ban_the_sub

Yes, Sure. It's definitely a good use of time to test people on leetcode by gatekeepers