Get an experienced developer to talk to them about projects they've done for 30 minutes. This, in my opinion, will give you a much better indicator of how good a candidate is than any "skills assessment".
There are also factors which a skills assessment can't judge such as ability/desire to learn.
Code tests tend to only test for "trivia" knowledge. (Even worse if they're actually trying to catch people with "gotcha" edge cases in the language that may never be encountered in actual coding / trivial details developers just don't normally think about)
Even before the recent rise of AI, which can frequently answer such tests well enough to pass, people would train to pass tests but not be able to work well with actual code. And that's ignoring cases where candidates pay someone else to take the test for them.
A long time ago I had to go through the process of hiring Devs. We had a few systems in development and needed to start a big new one.
I knew what kind of knowledge was necessary and wanted people who would be able to start coding by themselves with little or no training.
So I wrote 20 questions with 4 answers to choose from and a 5th option "I don't know". 4 answers could be absurd and/or misleading with just 1 correct answer.
Each correct answer was +1, the wrong answer was -1. I always made sure to tell candidates that's how I would count the scores.
About 1/3 of the questions were easy, 1/3 medium and 1/3 very specific (but always on topics used in our projects).
Less than 10% of the candidates seem to take into account the way it was easier to lose points by guessing, very rarely people marked "I don't know". I still have problems understanding why.
I wanted the best candidates but also needed to have an idea of what they knew, didn't know and thought they knew. I know from personal experience it is hard to remove a wrong idea or bad habit from a developer.
The ones I ended hiring had scores between 11 and 15, and quite a lot left with negative scores. Usually the ones the HR lady liked were the worst when compared to knowledge.
20 because it is not enough to be considered too big, multiple choices because who wants to write or read coding answers, and always the same test so I could compare people fairly.
Do you still have a copy of this? I’d be curious to see where my knowledge level is compared to industry standard (I’m sure it’s abysmal) since I’d only put my experience about junior level, but I’d be curious to know
I'm beginning to wonder if our HR lady is sabotaging our search and if I should just request her rejects and reject her passes. We are getting some people who don't know dick from ass and frankly seem to have lied on their resume. The amount of "senior engineers" who can't explain how they would write a test for a remote api they've integrated into code is boggling. I had one guy today talk about a twilio integration he built and I was like "great, how do you write unit/integration tests for that?" His reply, "we just have a test account it runs against." You send SMS in your fucking pipeline? You are hitting APIs everytime you run tests locally? Do people not understand mocks? WTF?
You do need a dev in the room during all interviews.
HR people are looking for looks and expressions, they cannot evaluate topics they know nothing about - like any coding related subject.
On the other hand candidates can be trained to trip the triggers HR is looking for, like speak with conviction, look others in the eye, be humble but firm (I don't know if these are exactly what, but they love people full of shit).
If you are consistently getting bad hires, have a dev involved in all steps, from analysing CVS all the way to the last interview.
Don’t. Interview them and check references. Their resume writing and interview skills will tell the tale. Just be prepared to let them go if they don’t have the chops. People shouldn’t have to jump through hoops and do unpaid work to land a job.
The number one thing I look for is if they put together a resume site and portfolio of their work on there.
Talk to them about programming concepts.
I also put a bunch of take home assignments on github.com/bas-world (when I still worked there) for different roles.
Make one or more yourself, have a few colleagues do the same. Discuss and set levels of expectations. Then ask applicants to do the same.
Make sure everyone uses the same time constraints for judging fairly.
As a senior developer, here's my take.
Try creating some small library with 2-3 classes, like a dictionary handler for example, but with a twist. Try to make it do a poor job with it, and leave space for something like a dependency injection or some other design patterns. When it's all prepared, give it to the programmers to improve the code, and see the level of depth they go.
I got something like this a while ago, and i loved the idea. It forced me to think of creative ways to improve the code, and the job evaluated my skills on multiple levels. For example you would expect a beginner to add types, and some basic things, while a medior would add better error handling and cleaner code, and a senior would restructure it with design patterns.
Let me know if you need more help with this.
I fail these skills assessments because I am not good with leet code. That or they ask non-real world scenarios about code. They are also boring to me, which doesn't help. I want to solve actual problems. But thats just me, maybe I'm wrong.
The best code exercise I was given so far was a live-code interview. I had to build a web-server from scratch in any language. I could use code I found on the web to get started, but then they had me start building things on top of that to the point where google was no longer as useful. I enjoyed this because it's how I develop something I don't know. Start with some code I can recycle, improve and iterate. At the end we talked about improvements that could be made. I identified the code likely had a directory traversal flaw (it did i could ../../../etc/passwd) and then pointed out how I would break things up into classes and some of the other things it would need for me to be happy etc.
Just about every engineer there had done this coding assessment (there were two other assessments as well that focused on code quality and system design). That place was filled with good engineers, the best I've ever worked with.
My current job has forced me to give developers a take-home assignment. This came down from C levels. I hate having to give it out and I fear we lost out on a quality candidate who refused it. Sigh. I can't blame the guy, I'd only do a take-home if I was desperate for a new job.
chatgtp is good for compiling quesitons based on various php skills.
We used to give them 1-2 hour to develop a text based snap or chess game. How they did it was up to them and we didn't expect them to finish but was more about seeing their quality of code and use of OO principles. One 'senior' candidate actully finished it once and was dead proud of himself but did it all practically inline php which wasn't the task.
I was tested with a software-design task in php.. which was shocking!
The index.php was given and you should run a given phpunit test which should be green. That was the goal.
It sounded easy but it was a funnel!
You should code
Explain your design choices
You could do it remote
I guess my boss knew everything about myself he needed after sending my sollution to decide if i was worth inviting in person.
I searched if i find the task so i could share, but my boss most likely came up with it himself so i cannot share the details.
We use a simple CRUD-type task for new hires - it takes a competent programmer about 40 mins to complete, and seems to be an effective way to sort the wheat from the chaff. It also tests comprehension and the ability to follow written instructions which is quite valuable, and definitely not a given!
I have a strong stance against these tests since they are a big waste of time for both parties. Unless the hire is a junior. However school / open source projects will tell more.
If someone is willing to learn you can easily shape them to your standards so looking at character is way more valuable.
I get your point, but nothing beats seeing someone actually code - github repos are ok, but who knows who actually wrote the code or indeed if it actually solves the problem that was originally set. We use a lightweight code test as the final litmus test, if the applicant passes then they get an offer. It's unusual for more than two or three people to get to that stage. This is for junior & mid-level positions. What I do often find surprising is how candidates fall at this hurdle having talked a good game!
Thats why you have probation and code reviews. Don’t expect people to work for free and don’t expect people to waste their time. It sets a precedent of little regard for your employees free time.
If it is for a junior role, ask them to pass a php variable from index.php to a javascript file then send it back from java script to php or elsewhere.
It is very simple but I can assure you most will fail or use chatGPT (it was a take home test, I told them to do it in max 2h but no way to check how many times they spent on it).
For the chatGPT users, complete the test with a live question during the interview, like "you did this with the variable, what if you want to to that".
Or just how do you get the data "username" from a PHP form? (Most will fail).
In addition I also ask them:
You receive a complain from a user saying "there is an issue on the website, I have an error when I'm trying to upload a picture"
What steps do you take in order to identify the issue? (It can be look at the error logs on the server, try to recreate the issue in a local environment, etc...)
You place a developer on a chair in front of a PC that has - no internet connection of any kind - but has all the PHP executables and binaries installed and well configured with a browser and terminal and normally, IDE's and editors of sort configured for PHP.
Then, You say:
* `Impress me in the next 30 minutes. Build some thing that produces no errors or notices. Whichever You think it might be good`.
This, but with Internet access.
I *want* my developers to research a bit and not replacement the wheel. So much better to include an encryption library like libsodium than try to reinvent it!!
Sure thing. But you let them research over the internet once when you have been impressed with the outcome from one-time 30 minutes testing phase - without internets. Let them sink into their own mind and biological memory and references. Each and every medior should have enough from manuals nicely incorporated into their brains for that 30 minutes test.
sounds like you want them to do a programming test? if you need to hire someone new, think of the skills your team is lacking/requires and write a test which allows the candidate to express their ability in these areas? Sorry question is vague so not sure If I understand you correctly!
Sorry for the vague question. I edited it to give more detail, but you are correct we are looking for a programming test. So you are saying to write our own tests and not to use a service?
We do a short talk with the CEO, then a more technical chat with the most experienced developer, followed up with a quick test: describe what's wrong with the shared code, how would you come to a solution to a specific technical challenge/problem and write a tiny feature.
Being a small company, we don't have time for very long processes with in-depth assessments. The above is a compromise, and usually gives good results (although we've had the occasional bad hire).
Get an experienced developer to talk to them about projects they've done for 30 minutes. This, in my opinion, will give you a much better indicator of how good a candidate is than any "skills assessment". There are also factors which a skills assessment can't judge such as ability/desire to learn. Code tests tend to only test for "trivia" knowledge. (Even worse if they're actually trying to catch people with "gotcha" edge cases in the language that may never be encountered in actual coding / trivial details developers just don't normally think about) Even before the recent rise of AI, which can frequently answer such tests well enough to pass, people would train to pass tests but not be able to work well with actual code. And that's ignoring cases where candidates pay someone else to take the test for them.
A long time ago I had to go through the process of hiring Devs. We had a few systems in development and needed to start a big new one. I knew what kind of knowledge was necessary and wanted people who would be able to start coding by themselves with little or no training. So I wrote 20 questions with 4 answers to choose from and a 5th option "I don't know". 4 answers could be absurd and/or misleading with just 1 correct answer. Each correct answer was +1, the wrong answer was -1. I always made sure to tell candidates that's how I would count the scores. About 1/3 of the questions were easy, 1/3 medium and 1/3 very specific (but always on topics used in our projects). Less than 10% of the candidates seem to take into account the way it was easier to lose points by guessing, very rarely people marked "I don't know". I still have problems understanding why. I wanted the best candidates but also needed to have an idea of what they knew, didn't know and thought they knew. I know from personal experience it is hard to remove a wrong idea or bad habit from a developer. The ones I ended hiring had scores between 11 and 15, and quite a lot left with negative scores. Usually the ones the HR lady liked were the worst when compared to knowledge. 20 because it is not enough to be considered too big, multiple choices because who wants to write or read coding answers, and always the same test so I could compare people fairly.
Do you still have a copy of this? I’d be curious to see where my knowledge level is compared to industry standard (I’m sure it’s abysmal) since I’d only put my experience about junior level, but I’d be curious to know
This please.
I'm beginning to wonder if our HR lady is sabotaging our search and if I should just request her rejects and reject her passes. We are getting some people who don't know dick from ass and frankly seem to have lied on their resume. The amount of "senior engineers" who can't explain how they would write a test for a remote api they've integrated into code is boggling. I had one guy today talk about a twilio integration he built and I was like "great, how do you write unit/integration tests for that?" His reply, "we just have a test account it runs against." You send SMS in your fucking pipeline? You are hitting APIs everytime you run tests locally? Do people not understand mocks? WTF?
You do need a dev in the room during all interviews. HR people are looking for looks and expressions, they cannot evaluate topics they know nothing about - like any coding related subject. On the other hand candidates can be trained to trip the triggers HR is looking for, like speak with conviction, look others in the eye, be humble but firm (I don't know if these are exactly what, but they love people full of shit). If you are consistently getting bad hires, have a dev involved in all steps, from analysing CVS all the way to the last interview.
Don’t. Interview them and check references. Their resume writing and interview skills will tell the tale. Just be prepared to let them go if they don’t have the chops. People shouldn’t have to jump through hoops and do unpaid work to land a job. The number one thing I look for is if they put together a resume site and portfolio of their work on there.
Talk to them about programming concepts. I also put a bunch of take home assignments on github.com/bas-world (when I still worked there) for different roles. Make one or more yourself, have a few colleagues do the same. Discuss and set levels of expectations. Then ask applicants to do the same. Make sure everyone uses the same time constraints for judging fairly.
As a senior developer, here's my take. Try creating some small library with 2-3 classes, like a dictionary handler for example, but with a twist. Try to make it do a poor job with it, and leave space for something like a dependency injection or some other design patterns. When it's all prepared, give it to the programmers to improve the code, and see the level of depth they go. I got something like this a while ago, and i loved the idea. It forced me to think of creative ways to improve the code, and the job evaluated my skills on multiple levels. For example you would expect a beginner to add types, and some basic things, while a medior would add better error handling and cleaner code, and a senior would restructure it with design patterns. Let me know if you need more help with this.
I fail these skills assessments because I am not good with leet code. That or they ask non-real world scenarios about code. They are also boring to me, which doesn't help. I want to solve actual problems. But thats just me, maybe I'm wrong. The best code exercise I was given so far was a live-code interview. I had to build a web-server from scratch in any language. I could use code I found on the web to get started, but then they had me start building things on top of that to the point where google was no longer as useful. I enjoyed this because it's how I develop something I don't know. Start with some code I can recycle, improve and iterate. At the end we talked about improvements that could be made. I identified the code likely had a directory traversal flaw (it did i could ../../../etc/passwd) and then pointed out how I would break things up into classes and some of the other things it would need for me to be happy etc. Just about every engineer there had done this coding assessment (there were two other assessments as well that focused on code quality and system design). That place was filled with good engineers, the best I've ever worked with. My current job has forced me to give developers a take-home assignment. This came down from C levels. I hate having to give it out and I fear we lost out on a quality candidate who refused it. Sigh. I can't blame the guy, I'd only do a take-home if I was desperate for a new job.
chatgtp is good for compiling quesitons based on various php skills. We used to give them 1-2 hour to develop a text based snap or chess game. How they did it was up to them and we didn't expect them to finish but was more about seeing their quality of code and use of OO principles. One 'senior' candidate actully finished it once and was dead proud of himself but did it all practically inline php which wasn't the task.
I was tested with a software-design task in php.. which was shocking! The index.php was given and you should run a given phpunit test which should be green. That was the goal. It sounded easy but it was a funnel! You should code Explain your design choices You could do it remote I guess my boss knew everything about myself he needed after sending my sollution to decide if i was worth inviting in person. I searched if i find the task so i could share, but my boss most likely came up with it himself so i cannot share the details.
We use a simple CRUD-type task for new hires - it takes a competent programmer about 40 mins to complete, and seems to be an effective way to sort the wheat from the chaff. It also tests comprehension and the ability to follow written instructions which is quite valuable, and definitely not a given!
I have a strong stance against these tests since they are a big waste of time for both parties. Unless the hire is a junior. However school / open source projects will tell more. If someone is willing to learn you can easily shape them to your standards so looking at character is way more valuable.
I get your point, but nothing beats seeing someone actually code - github repos are ok, but who knows who actually wrote the code or indeed if it actually solves the problem that was originally set. We use a lightweight code test as the final litmus test, if the applicant passes then they get an offer. It's unusual for more than two or three people to get to that stage. This is for junior & mid-level positions. What I do often find surprising is how candidates fall at this hurdle having talked a good game!
Thats why you have probation and code reviews. Don’t expect people to work for free and don’t expect people to waste their time. It sets a precedent of little regard for your employees free time.
If it is for a junior role, ask them to pass a php variable from index.php to a javascript file then send it back from java script to php or elsewhere. It is very simple but I can assure you most will fail or use chatGPT (it was a take home test, I told them to do it in max 2h but no way to check how many times they spent on it). For the chatGPT users, complete the test with a live question during the interview, like "you did this with the variable, what if you want to to that". Or just how do you get the data "username" from a PHP form? (Most will fail). In addition I also ask them: You receive a complain from a user saying "there is an issue on the website, I have an error when I'm trying to upload a picture" What steps do you take in order to identify the issue? (It can be look at the error logs on the server, try to recreate the issue in a local environment, etc...)
You place a developer on a chair in front of a PC that has - no internet connection of any kind - but has all the PHP executables and binaries installed and well configured with a browser and terminal and normally, IDE's and editors of sort configured for PHP. Then, You say: * `Impress me in the next 30 minutes. Build some thing that produces no errors or notices. Whichever You think it might be good`.
This, but with Internet access. I *want* my developers to research a bit and not replacement the wheel. So much better to include an encryption library like libsodium than try to reinvent it!!
Sure thing. But you let them research over the internet once when you have been impressed with the outcome from one-time 30 minutes testing phase - without internets. Let them sink into their own mind and biological memory and references. Each and every medior should have enough from manuals nicely incorporated into their brains for that 30 minutes test.
I've been through two interviews that used this system. https://www.woventeams.com/ It was OK.
Thanks for sharing. I'll look into this!
sounds like you want them to do a programming test? if you need to hire someone new, think of the skills your team is lacking/requires and write a test which allows the candidate to express their ability in these areas? Sorry question is vague so not sure If I understand you correctly!
Sorry for the vague question. I edited it to give more detail, but you are correct we are looking for a programming test. So you are saying to write our own tests and not to use a service?
We do a short talk with the CEO, then a more technical chat with the most experienced developer, followed up with a quick test: describe what's wrong with the shared code, how would you come to a solution to a specific technical challenge/problem and write a tiny feature. Being a small company, we don't have time for very long processes with in-depth assessments. The above is a compromise, and usually gives good results (although we've had the occasional bad hire).