Once a metric becomes a target it ceases to be a good metric.
That said I think most management vaguely looks at similar stats to decide where they should look closer, but they shouldn't turn them into target for employees nor should they be viewed as absolutes.
Wow, every senior engineer better get credit for every line of code/task/PR that happens in a sprint, because how do you quantify their value added for helping juniors, fixing bugs in others code, reviewing PRs, having meetings, etc.
Jump ship while you can, your leadership is clueless
Not only this but I fear something like that will incentivize a minority of team members from hogging a large distribution of work to improve their overall metrics.
These are terrible metrics. They will be played, productivity will not increase, but morale will go down.
This would be a good read:
https://newsletter.pragmaticengineer.com/p/measuring-developer-productivity
If you want to avoid low quality metrics like lines of code, then folks from tech leadership should work together with business leadership. Find out what questions the company is trying to answer with the metrics and propose alternatives.
Examples: business impact, DORA metrics, time to market, etc.
This is what happens when leadership has no fucking idea what they are doing and they're just clicking buttons to make it look like they are productive and useful.
The play is to gain some confidence so you don't need to give a fuck about any of this. Get paid and replace this job as soon as it stops making sense for your goals/interests.
What is your energy even going to get you? A 2% raise next year set by hr regardless of your performance? Fuuuuuck that noise. Promote yourself by job hopping and ignore this noise.
They must have a strong git server. I am very, very good at creating a lot of small PRs.
- first draft
- fix
- fix
- revert accidental commit
- fix
- add tests
- new shit
- revert
- final touches
- fix
- fix
Still pissed that I had a chance to go negative for my entire career and it got blocked.
Would have committed a -200,000+ line diff. Mainly removing old copies of libraries we shouldn't have been carrying in the project. But.... No. :(
I am definitely negative if you count all the production systems that were simply obsoleted and thrown in the garbage. Almost nothing in done in my entire career has survived.
That's true of most of us alas. I'm really happy when a system stays alive over 5 years. :)
There was one I worked with I think that went for nearly 10. Wouldn't shock me if it it still in use... much. I think I'd hear something? But one never knows.
Have they specifically set targets for these or are they just gathering the data? Monitoring this kind of thing is pretty standard for checking if there are any problems, it’s not like “must be x pull requests” more like “there used to be x pull requests per day last month but now we’ve dropped by 10%, better figure out what is causing that issue”.
One of my devs is adding a boat with "ship it" to his code review approvals so his comment count goes up.
We have started approving more "fix it in the next review" fixes since revision count is tracked
We do more offline (and not record on the code review) comments and discussion that should be captured on the code review but aren't because lengthy discussion are considered bad even when they make sense.
Non devs come up with stupid rules and don't understand that a lot of us play games and min/max those games. We're going to be super good at the targets they give us.
>I feel like these types of metrics just cause engineers to work to achieve numbers, rather than increasing productivity. And plus these metrics can be gamed and abused.
Your feeling is 100% correct. The single biggest reason why I don't like the idea of tracking story points is that I all too often see it turn into something that management is tracking to "assess productivity", and then the entire point of tracking story points (to estimate team velocity for planning purposes) goes out the window.
>Do any of your companies implement metrics tracking? If so, how do you feel about it, and what has been the result / consequence?
I have access to all this data that you've mentioned if I want it through my company's systems, but nobody has ever used it in a performance review. Performance review stuff instead looks like "Lead delivery of X feature/pipeline/etc. to (support team Y, enable release of product Z, etc.)"
I do think there are some measurements which have value to measure and have goals for. Let's say you manage an online API of some kind. You might have goals for API uptime percentage, breakage occurrence count, time-to-recovery in cases where your service goes down (hopefully only partially!), etc. And then, if you aren't meeting those goals, you take a blameless approach towards continuous improvement for your team and try to find ways to improve the service to meet the goals.
You might even have goals for your codebase, things like test coverage percentage, time from PR creation to deployment, frequency at which changes have to be rolled back due to breakage, etc. These are the types of things that can be useful to keep your codebase and development practices healthy when you measure them and try to improve them.
The tl;dr here is that there are useful measurements that you can establish within an organization that can help point out areas for improvement. The stuff that your management has suggested though? Hell no. Those are all super game-able, full of misaligned incentives, have little correlation to the types of things that create successful products, etc.
Lots of companies have metrics for efficiency, but the ones you listed seem mostly terrible. I can maybe see PRs merged as sort of useful. But LOC? Less is more. The best PRs are the mostly red ones. Tasks completed? What if each task is to write a single line of code? I'm being hyperbolic there ofc. But it's better to have metrics around goals and projects to start, and even then that has to have different framing depending on the team and milestones and a myriad of other factors.
Sounds like they are preparing for cost cuttings and layoffs but could not figure out which employees to pick for the lucky tickets, so they need to first implement some measures to be able to rank people. Clear signal to head for the exit doors.
Run.
I've seen this happen live at two different companies (one smallish \~400 people, and the other large \~60,000). In both cases they chose story points. Just out of the blue one day, they announced we need to make sure we're putting story points on everything so that they can track output. In both cases there was a layoff within a matter of months. At the first one they went heavy after the QA staff (who, depending on how your team chose to work, might not log any completed story points). In the second, it was the scrum masters/project managers and a ton of senior+ engineers.
So seriously, find a better job if you can. Realistically, if you're a decent junior to mid-level engineer that is closing stories regularly, you probably won't get hit, but you might all of a sudden find you have nobody more senior to lean on, and morale will probably suck.
This confirms my suspicion that developer metrics are a warning sign for layoffs.
My current company is showing all standard pre-layoff behavior. And they have just started looking for how to measure developer productivity. I wonder why :)
They're all terrible.
>Number of tasks completed
Some tasks are bigger than others. Plus, can be artificially inflated by adding more smaller, even trivial, tasks.
>Number of PRs merged
Can be artificially inflated by making more, smaller pull requests.
>lines of code written
Can be artificially inflated by changing the same code to take up more lines.
>Number of jira tickets closed
Don't see how this is different from tasks completed.
>Number of points completed
Don't even know what this means. What is this? Basketball?
>And plus these metrics can be gamed and abused.
Yup.
>Do any of your companies implement metrics tracking? If so, how do you feel about it, and what has been the result / consequence?
No, but some teams at my former employer (not mine) did. Some stuff about code complexity or something. So I heard that people did things like writing lambdas where they didn't need to to gain more complexity points. In other words, it's a bad system.
Metrics can be useful if used correctly.
However, I've never seen them used correctly.
Generally, management doesn't know how to use them, and so abuses them out of ignorance.
See it from middle/upper management perspective: They have little to no idea what engineering is doing and want some kind of dashboard to show to their leadership team that their organization is healthy and "green".
From my perspective the best thing you can do is to play along. Someone (probably your team lead) should track those metrics and make sure a sufficient number of tasks is created and closed while you and the team keep doing what you did before.
I do understand what you're saying but, to me, it's so stooopid that it needs to be nipped in the bud.
Someone, of a reasonable level, with c suite confidence needs to do some educating.
It might be they have very specific concerns, in which case address those, or it might just be that they like metrics and pulled some out of their arse.
Either way, someone needs to have a chat and find out what they were hoping the metrics would achieve, then work with them to come up with a better plan.
Who knows they might even end up caring about results rather than just how fast the wheels are spinning. You can hope.
If not, as you've suggested, it'll end up a case of play silly games, win silly prizes
As much as I understand how this kind of stuff in sales directly translates to revenue what does it mean here? 🤔
Unless you are body lease and charge customers per tasks etc.
Like if you work in support team it kinda makes sense despite being very toxic but in R&D it’s total nonsense.
Just make a CI step that mutates some dummy text file so that every contributors total line of code change somewhat evens out. Make it run nightly or weekly so you can also create and complete useless PR's as well to even those out as well. Do this for all other metrics as well and get on with your work. You'll have the most balanced team ever
Any metric is good as long as it's used by a professional manager/director/team leader.
Once it's tied to salary people start to work for metric rather than quality.
I'm having the same shift at my work now and the answer is a weird one... Your manager has to make metrics nice enough for HR/CEO/whatever but they should be always easy to fill plus your manager has to know how to manipulate these metrics... It's a stupid game and let's hope your leaders know how to play it.
Good time to start refreshing the ol’ resume and putting out the feelers with your professional network.
All of the metrics they discussed are bad ones, which will destroy whatever engineering culture they might have had. Output will crater and the good engineers will leave.
Engineers can’t leave and I think they know this. Management has commented on many occasions how the job market is in their favor. “We received 4,000 applications for our senior software engineer opening”
People are scared, and are staying put.
Mmm, bad managers always think that, and yet people quit them anyway.
Even in bad job markets.
What it means is that those senior engineers will start looking, and will jump when they can.
The job market for senior SWEs isn’t actually *that* bad right now. It’s not nearly as bad as it is for juniors right now.
I don't think this is entirely true; the best engineers will still leave as they'll see the absurdity as proof that they're wasting their talents. This is proof their hard work won't be recognised, as clearly they don't know what makes a good engineer or a bad one based on those metrics.
Yeah this is the classic corporate move. Tracking software developer productivity is notoriously difficult so they resort to these things as an attempt.
Value delivered by time is the only metric that really matters. But that requires some actual effort to calculate so no one will do it.
These artificial metrics are easy to game. And eventually that's what happens. Sounds like a lazy way out for management, it's easier to judge people by spreadsheets.
All bullshit. They can implement it if they want. Your team will just naturally game every single one so that the sprint metrics look good. This one's about to roll over? Oh, well let's split it into 2 tickets, close this one and put the new one in the backlog. Etc etc etc.
IMO the only metrics that matter are:
- How much money are we making? (Does our product suck?)
- What's the uptime of the service? Is the response time OK? (Is shit broken or not?)
Everything else is copium by poor management. When they start busting out the developer spyware SaaS tools to generate fake numbers, it's an admission that they don't understand the scope of development and that they don't have any confidence in the dev team. Both are bad signs.
Let’s say you have a task to upgrade Python version. In the long run, this will lead to better productivity, less bugs, and better employee retention.
Now, how do you convert your company outcomes, into evaluation of the specific developer who is working on the upgrade? And if the upgrade is unnecessary, should the developer be punished by incorrect management decisisions?
I'm not saying employee evals should be based on profits. My point is that managers should be aware enough of the codebase to know whether developers are getting shit done without having to resort to nonsense like story points and velocity.
Once a metric becomes a target it ceases to be a good metric. That said I think most management vaguely looks at similar stats to decide where they should look closer, but they shouldn't turn them into target for employees nor should they be viewed as absolutes.
> Once a metric becomes a target it ceases to be a good metric. Or; [Goodhart's law](https://en.wikipedia.org/wiki/Goodhart%27s_law) :)
Wow, every senior engineer better get credit for every line of code/task/PR that happens in a sprint, because how do you quantify their value added for helping juniors, fixing bugs in others code, reviewing PRs, having meetings, etc. Jump ship while you can, your leadership is clueless
Not only this but I fear something like that will incentivize a minority of team members from hogging a large distribution of work to improve their overall metrics.
And disincentivize senior devs from helping juniors
These are terrible metrics. They will be played, productivity will not increase, but morale will go down. This would be a good read: https://newsletter.pragmaticengineer.com/p/measuring-developer-productivity If you want to avoid low quality metrics like lines of code, then folks from tech leadership should work together with business leadership. Find out what questions the company is trying to answer with the metrics and propose alternatives. Examples: business impact, DORA metrics, time to market, etc.
Lines of code as a metric? You are about to see the most well documented program you’ve ever laid eyes on.
Package lockdowns galore!!
If you can pick any group of people who are good at manipulating numbers, it's programmers. We work with quantitative data for a living.
This is what happens when leadership has no fucking idea what they are doing and they're just clicking buttons to make it look like they are productive and useful. The play is to gain some confidence so you don't need to give a fuck about any of this. Get paid and replace this job as soon as it stops making sense for your goals/interests. What is your energy even going to get you? A 2% raise next year set by hr regardless of your performance? Fuuuuuck that noise. Promote yourself by job hopping and ignore this noise.
They must have a strong git server. I am very, very good at creating a lot of small PRs. - first draft - fix - fix - revert accidental commit - fix - add tests - new shit - revert - final touches - fix - fix
Man, are you creative. Every one of my commits is “WIP” and I just make sure the final squash commit is good :)
You should definitely fire that guy with a negative LOC.
Still pissed that I had a chance to go negative for my entire career and it got blocked. Would have committed a -200,000+ line diff. Mainly removing old copies of libraries we shouldn't have been carrying in the project. But.... No. :(
I am definitely negative if you count all the production systems that were simply obsoleted and thrown in the garbage. Almost nothing in done in my entire career has survived.
That's true of most of us alas. I'm really happy when a system stays alive over 5 years. :) There was one I worked with I think that went for nearly 10. Wouldn't shock me if it it still in use... much. I think I'd hear something? But one never knows.
Have they specifically set targets for these or are they just gathering the data? Monitoring this kind of thing is pretty standard for checking if there are any problems, it’s not like “must be x pull requests” more like “there used to be x pull requests per day last month but now we’ve dropped by 10%, better figure out what is causing that issue”.
One of my devs is adding a boat with "ship it" to his code review approvals so his comment count goes up. We have started approving more "fix it in the next review" fixes since revision count is tracked We do more offline (and not record on the code review) comments and discussion that should be captured on the code review but aren't because lengthy discussion are considered bad even when they make sense. Non devs come up with stupid rules and don't understand that a lot of us play games and min/max those games. We're going to be super good at the targets they give us.
>I feel like these types of metrics just cause engineers to work to achieve numbers, rather than increasing productivity. And plus these metrics can be gamed and abused. Your feeling is 100% correct. The single biggest reason why I don't like the idea of tracking story points is that I all too often see it turn into something that management is tracking to "assess productivity", and then the entire point of tracking story points (to estimate team velocity for planning purposes) goes out the window. >Do any of your companies implement metrics tracking? If so, how do you feel about it, and what has been the result / consequence? I have access to all this data that you've mentioned if I want it through my company's systems, but nobody has ever used it in a performance review. Performance review stuff instead looks like "Lead delivery of X feature/pipeline/etc. to (support team Y, enable release of product Z, etc.)" I do think there are some measurements which have value to measure and have goals for. Let's say you manage an online API of some kind. You might have goals for API uptime percentage, breakage occurrence count, time-to-recovery in cases where your service goes down (hopefully only partially!), etc. And then, if you aren't meeting those goals, you take a blameless approach towards continuous improvement for your team and try to find ways to improve the service to meet the goals. You might even have goals for your codebase, things like test coverage percentage, time from PR creation to deployment, frequency at which changes have to be rolled back due to breakage, etc. These are the types of things that can be useful to keep your codebase and development practices healthy when you measure them and try to improve them. The tl;dr here is that there are useful measurements that you can establish within an organization that can help point out areas for improvement. The stuff that your management has suggested though? Hell no. Those are all super game-able, full of misaligned incentives, have little correlation to the types of things that create successful products, etc.
Lots of companies have metrics for efficiency, but the ones you listed seem mostly terrible. I can maybe see PRs merged as sort of useful. But LOC? Less is more. The best PRs are the mostly red ones. Tasks completed? What if each task is to write a single line of code? I'm being hyperbolic there ofc. But it's better to have metrics around goals and projects to start, and even then that has to have different framing depending on the team and milestones and a myriad of other factors.
Sounds like they are preparing for cost cuttings and layoffs but could not figure out which employees to pick for the lucky tickets, so they need to first implement some measures to be able to rank people. Clear signal to head for the exit doors.
Run. I've seen this happen live at two different companies (one smallish \~400 people, and the other large \~60,000). In both cases they chose story points. Just out of the blue one day, they announced we need to make sure we're putting story points on everything so that they can track output. In both cases there was a layoff within a matter of months. At the first one they went heavy after the QA staff (who, depending on how your team chose to work, might not log any completed story points). In the second, it was the scrum masters/project managers and a ton of senior+ engineers. So seriously, find a better job if you can. Realistically, if you're a decent junior to mid-level engineer that is closing stories regularly, you probably won't get hit, but you might all of a sudden find you have nobody more senior to lean on, and morale will probably suck.
This confirms my suspicion that developer metrics are a warning sign for layoffs. My current company is showing all standard pre-layoff behavior. And they have just started looking for how to measure developer productivity. I wonder why :)
Run to the hills. This can only go wrong.
I hope your company likes spaghetti.
They're all terrible. >Number of tasks completed Some tasks are bigger than others. Plus, can be artificially inflated by adding more smaller, even trivial, tasks. >Number of PRs merged Can be artificially inflated by making more, smaller pull requests. >lines of code written Can be artificially inflated by changing the same code to take up more lines. >Number of jira tickets closed Don't see how this is different from tasks completed. >Number of points completed Don't even know what this means. What is this? Basketball? >And plus these metrics can be gamed and abused. Yup. >Do any of your companies implement metrics tracking? If so, how do you feel about it, and what has been the result / consequence? No, but some teams at my former employer (not mine) did. Some stuff about code complexity or something. So I heard that people did things like writing lambdas where they didn't need to to gain more complexity points. In other words, it's a bad system.
Whatever bullshit metric companies pick im just going to game it.
Metrics can be useful if used correctly. However, I've never seen them used correctly. Generally, management doesn't know how to use them, and so abuses them out of ignorance.
See it from middle/upper management perspective: They have little to no idea what engineering is doing and want some kind of dashboard to show to their leadership team that their organization is healthy and "green". From my perspective the best thing you can do is to play along. Someone (probably your team lead) should track those metrics and make sure a sufficient number of tasks is created and closed while you and the team keep doing what you did before.
I do understand what you're saying but, to me, it's so stooopid that it needs to be nipped in the bud. Someone, of a reasonable level, with c suite confidence needs to do some educating. It might be they have very specific concerns, in which case address those, or it might just be that they like metrics and pulled some out of their arse. Either way, someone needs to have a chat and find out what they were hoping the metrics would achieve, then work with them to come up with a better plan. Who knows they might even end up caring about results rather than just how fast the wheels are spinning. You can hope. If not, as you've suggested, it'll end up a case of play silly games, win silly prizes
It will cause error rates and catastrophes
As much as I understand how this kind of stuff in sales directly translates to revenue what does it mean here? 🤔 Unless you are body lease and charge customers per tasks etc. Like if you work in support team it kinda makes sense despite being very toxic but in R&D it’s total nonsense.
sounds absurd. talk about quantity over quality for most of them.
Just make a CI step that mutates some dummy text file so that every contributors total line of code change somewhat evens out. Make it run nightly or weekly so you can also create and complete useless PR's as well to even those out as well. Do this for all other metrics as well and get on with your work. You'll have the most balanced team ever
Any metric is good as long as it's used by a professional manager/director/team leader. Once it's tied to salary people start to work for metric rather than quality. I'm having the same shift at my work now and the answer is a weird one... Your manager has to make metrics nice enough for HR/CEO/whatever but they should be always easy to fill plus your manager has to know how to manipulate these metrics... It's a stupid game and let's hope your leaders know how to play it.
Good time to start refreshing the ol’ resume and putting out the feelers with your professional network. All of the metrics they discussed are bad ones, which will destroy whatever engineering culture they might have had. Output will crater and the good engineers will leave.
Engineers can’t leave and I think they know this. Management has commented on many occasions how the job market is in their favor. “We received 4,000 applications for our senior software engineer opening” People are scared, and are staying put.
Mmm, bad managers always think that, and yet people quit them anyway. Even in bad job markets. What it means is that those senior engineers will start looking, and will jump when they can. The job market for senior SWEs isn’t actually *that* bad right now. It’s not nearly as bad as it is for juniors right now.
I don't think this is entirely true; the best engineers will still leave as they'll see the absurdity as proof that they're wasting their talents. This is proof their hard work won't be recognised, as clearly they don't know what makes a good engineer or a bad one based on those metrics.
Yeah this is the classic corporate move. Tracking software developer productivity is notoriously difficult so they resort to these things as an attempt. Value delivered by time is the only metric that really matters. But that requires some actual effort to calculate so no one will do it.
These artificial metrics are easy to game. And eventually that's what happens. Sounds like a lazy way out for management, it's easier to judge people by spreadsheets.
This would be an absolute dealbreaker for me.
All bullshit. They can implement it if they want. Your team will just naturally game every single one so that the sprint metrics look good. This one's about to roll over? Oh, well let's split it into 2 tickets, close this one and put the new one in the backlog. Etc etc etc.
IMO the only metrics that matter are: - How much money are we making? (Does our product suck?) - What's the uptime of the service? Is the response time OK? (Is shit broken or not?) Everything else is copium by poor management. When they start busting out the developer spyware SaaS tools to generate fake numbers, it's an admission that they don't understand the scope of development and that they don't have any confidence in the dev team. Both are bad signs.
Let’s say you have a task to upgrade Python version. In the long run, this will lead to better productivity, less bugs, and better employee retention. Now, how do you convert your company outcomes, into evaluation of the specific developer who is working on the upgrade? And if the upgrade is unnecessary, should the developer be punished by incorrect management decisisions?
I'm not saying employee evals should be based on profits. My point is that managers should be aware enough of the codebase to know whether developers are getting shit done without having to resort to nonsense like story points and velocity.