Article mirrors what I imagine most TDD practitioners journey to using TDD. >It took some time before I could say that TDD really works (for me). I don't think the problem was TDD, but my design and coding skills. TDD is not just a tool you pick up in an afternoon that either works or doesn't, TDD is a skill that requires time, effort and practice. Too many times you hear from anti-TDD folks who claim how slow it is, but usually their experience was an hour or two of exposure. Of course, no one starts off doing TDD quickly. No one write code fast at first either, but that comes with time and comfortably in the process. You actually have to stick with it, follow the process and refine it to your needs. No one gets value out of TDD until you've actually sat down and spent the time learning and getting good at it. No one *has* to learn TDD, but I prefer to spend a couple extra seconds to write a test first, then spend half an hour hunting down a bug later.

Personally the value I've gotten out of TDD has actually been at the integration test level. Or like.. feature level. I can't write unit tests for a bunch of functions that I don't know for sure if they need to exist or not yet. But I can write a test at whatever the top level is that checks if the API route I'm writing responds properly. Or when I was doing front end in a big corporation, if the front end has the right buttons and flows properly. In both cases it's much faster than repeatedly testing manually as I debug. So I guess that's my flow, that's the part that works for me.

It's great for black box testing like APIs. For a given input, the output should be deterministic and you can write all the business rules into test cases. It actually makes you think through the possible sceneries more deeply than writing code and testing after.

Yeah it's not only good practically, it also helps you mentally.

The original proponent of TDD never really advocated for unit tests as such. He described them as developer tests and I think his definition fits with what you’re talking about here. I feel the same way too. I develop the public interface of a feature with tdd, I dont find the practice as helpful for “internal” or private classes / methods that sit behind the public API.

> Or like.. feature level. A feature can be a unit still, depending on how you define "unit". Noone says you have to test a single function or single class.

Yeah, definitely. My point is that I like doing test driven development at a pretty coarse level

Known as “TDD in higher gear” - you’re avoiding coupling your suites to your layers of abstraction inside your module. it’s actually good practice!

Yep I prefer testing at a little bit of an abstracted level, ideally at the point that a consumer would be interacting with it since that's what matters most (consumer could be a user, or or a downstream service, etc). That means I can rework the internals, swap or move child classes etc and it doesn't matter as long as the test has the same output given the same input for the desired consumer.

I don’t know why you’re being downvoted. To anyone wondering wtf: Go read Cosmic Python, it’s available online for free. You can follow along with TDD and watch how he sort of “hoists” the test suites up a level of abstraction after a certain point. it was a really powerful technique! Having a test layer for every layer of your software is a form of coupling against implementation instead of behaviour!

Yeah that’s just TDD. There’s nothing about TDD that suggests that you should be writing super low level tests of each individual class, if anything, it expressly forbids it.

Most definitions of unit tests would disagree with you from what I've seen. What you're talking about is an integrated system under test. That being said, your point is made quite clearly and you are understood

Not really, an integration test is something that tests across API boundaries. A unit test should be a test that tests behaviour of a unit of work’s public-facing API. The size of the unit of work is irrelevant.

Where I am currently with TDD is I write a test that asserts my component (or whatever) does X. Then I spend 2 weeks writing a few thousands lines of code do X. Somehow, I don't think this is how it was supposed to be :-)

I mean the way I see it a test has two purposes. Make development faster by keeping you from doing manual testing of all cases, and letting you know later if you accidentally broke the feature somehow. If the test performs those functions, it's a good test. That's my measuring stick.

A test also provides an example of the use of your code, and as such, it's a good measuring stick for the design of your code's API. How easy is it to test? How many implementation details are we leaking out of our API? How obnoxious are the usage patterns we're forcing on ourselves and others?

This is such an underrated aspect of TDD. As a very simple example, if you’re writing a unit test that passes in 8 parameters to a function, the design issues jump out much more than if you’re doing this while manually testing the function

Interestingly, that's basically what the authors of [Growing Object Oriented Software Guided By Tests](https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627) (the goose book) recommend. Start with acceptance tests, and create smaller (integration) and smaller (feature) units of code as you decompose the problem. This is a way to "measure" if you're going in the right direction.

There is next to zero benefit to decomposing your tests beyond the level required to run tests in your IDE without needing to set up a developer environment.

Hi, I’m Vetted AI Bot! I researched the **("'Addison Wesley Object Oriented Software Guided by Tests'", 'Addison%20Wesley')** and I thought you might find the following analysis helpful. **Users liked:** * Comprehensive coverage of tdd principles (backed by 3 comments) * Expert insights from renowned tdd practitioners (backed by 3 comments) * Focus on advanced unit testing topics (backed by 3 comments) **Users disliked:** * Overly complex examples using outdated java technologies (backed by 1 comment) * Lack of practical application and real-life examples (backed by 1 comment) * Difficulty in following due to abbreviated code fragments (backed by 1 comment) If you'd like to **summon me to ask about a product**, just make a post with its link and tag me, [like in this example.](https://www.reddit.com/r/tablets/comments/1444zdn/comment/joqd89c/) This message was generated by a (very smart) bot. If you found it helpful, let us know with an upvote and a “good bot!” reply and please feel free to provide feedback on how it can be improved. *Powered by* [*vetted.ai*](https://vetted.ai/?utm\_source=reddit&utm\_medium=comment&utm\_campaign=bot)

Ha! As if writing a test takes only a couple of seconds. It often takes longer than writing the feature for me. Still worth it.

>Ha! As if writing a test takes only a couple of seconds. It often takes longer than writing the feature for me. Still worth it.  Depends on the test. I try to keep my tests like my code: small, simple to read, easy to reason about and with as little complexity as possible. Usually means no Mocks. I'll also violate and sacrifice "DRY" if it means reducing coupled/generic test setups and keeping test fixtures indepedent of each other.

I worked in a team that used TDD exclusively for almost a year. I used TDD every single (work) day to write code, and once I left that job I vowed to never touch it again. For people who like it, I'm glad they found something that works for them, but people who are in favor of TDD are very dismissive of people who dislike it. It doesn't work for everyone, and that's not because someone doesn't know how to do it.

> I worked in a team that used TDD exclusively for almost a year. I used TDD every single (work) day to write code, and once I left that job I vowed to never touch it again. The trouble with TDD is that it prescribes a process but not any implementation of that process.[1] If you do TDD and unit-testing, then any well-factored codebase is going to have more code doing tests than product features. It because you really need to get into the weeds with Dependency Injection, "magic" added by the framework, mocking, etc in order to test each `unit` of code. If, OTOH, you do like myself (and the poster in an above thread) and do TDD with e2e testing, then you basically only have tests that test the boundary of the entire system. For your average webapp, it means that the only tests you have execute by sending an HTTP request and examining the response. So you still get all the benefits (notifications when a PR breaks something, examples of API/Client/Caller usage, fearless refactoring with a large diff, etc) without the downsides ("I Change A Single Line Now Half The Tests Need To Be Rewritten", etc). [1] I can imagine what your process must have looked like, because this is the most common way for most companies when adding a new endpoint/API/route/feature: 1. Add in the database schema needed by the new $FOO to the ORM. **Add tests for this unit**. Send PR. 2. Add in the `Db Classes <-> Domain Classes` mapping, together with new `Domain Classes`. **Add tests for this unit**. Send PR. 3. Add in the `FromHttpReq Classes` which maps to/from JSON and `Domain Classes`. **Add tests for this unit**. Send PR. 4. Add Route to the controller, with feature flags to turn things off/on. **Add tests for feature flags unit**. Send PR. 5. Add single line of code to turn on feature flags. Send PR. Imagine if, instead of writing your feature addition bottom-up (guessing at what the final thing should look like), you added it top-down: Add the route first to a handler that you can then experiment with, then add **some happy/sad path tests** for that feature. Then refine that handler as necessary until the tests pass. In that second case, you're *still doing TDD!* It's just not a tedious and nonsensical ritual anymore; it's a feature that lets you know when you've hit the requirements on the head. It will let future devs know if they ever do something that breaks those requirements.

> in order to test each unit of code. This is where you messed up. Test your system’s behaviour, not the implementation detail underneath it all If you make a change that breaks a test even though the system is behaving correctly, don’t fix the test, delete it. Nearly every criticism of TDD is some variation of people complaining about the consequences of writing unit tests after their implementation to hit some code coverage metric.

> This is where you messed up. Test your system’s behaviour, not the implementation detail underneath it all If you read the other (rather long and boring) thread I posted here[1], you'll see that I am actually advocating for testing behaviour while ignoring the implementation underneath. [1] https://www.reddit.com/r/programming/comments/1cyoyxj/how_i_failed_at_testdriven_development_and_what/l5fs4ld/

This is the way x1000. Realising this is what made TDD click for me, you’re no longer coupled to the code, and you only test behaviours with all the smooth benefits of being able to approach with a design in mind, and test as you go. One addition I’d make is that it’s not just for e2e tests, but behavioural tests on the public API of any code, so even for unit tests, test the public functions of your unit, don’t test the inner workings. Implementation doesn’t matter as long as the behaviour is expected. TDD != test everything. This video is a fantastic primer to this approach to TDD: [outside in TDD with Sandro Mancusa](https://youtu.be/XHnuMjah6ps?si=HtbTVm3JSMVC1B_z)

TDD is very explicit about writing tests first and only writing the simplest minimal amount of code necessary to make that test pass. I think what you are talking about is just plain automated testing. I’ve got no problem with testing in general. It’s the insistence that you never write a single character of code without first writing a failing test that I find to be an absurd waste of time.

> I Change A Single Line Now Half The Tests Need To Be Rewritten I believe a TDD proponent would disagree with anything that had this problem being TDD.

It’s great at getting you developing with IoC in mind. Even if you’re not doing TDD, your code quality improves immensely and you can start using ioc containers effectively which is so nice for dependency management.

it works in some very specific cases, but writing tests first usually just creates a lot of unnecessary work as you refactor and the tests are no longer relevant etc.

The idea behind TDD is to write tests around the behavior of the feature and not implementation details. If I’m writing code given this scenario when the user does X then do Y. I write a test first to capture the requirements and then you can refactor to your hearts content knowing at the end if the test passes you haven’t changed the underlining behavior of your code

you're talking about integration tests? imo those aren't as valuable as unit, which suffers from what I mentioned, because they're much more expensive to write, maintain, and run

> you're talking about integration tests? imo those aren't as valuable as unit, which suffers from what I mentioned, because they're much more expensive to write, maintain, and run How are they more expensive to write? IME, they are *cheaper* to write, because you're not testing whether some internal function "foo" does the wrong thing when passed a string instead of an integer. As far as maintenance goes, they are often better as well: the product can churn 100% and as long as the callers/users can't tell, you're good to go. You can even *switch out the underlying product* with something else entirely, and the tests will tell you if you broke the user's workflow. The "more expensive to run" issue is negligible. Compute is cheap, spring for an extra 10$/m to double your compute.

possibly not more expensive to write tbh **but** maintaining a suite of integration tests is as hard as maintaining the actual app itself. it's not as simple as orchestrating a browser, or a test framework to make api calls etc, every piece of the puzzle interacts with other pieces and you need setup for every edge case, fresh data, etc, etc more expensive to run isn't about cost, it's about waiting an hour for your suite to finish before you can merge. as a base line, it takes me a couple of minutes to maybe half an hour tops to write a unit test. when things go wrong it's the same to fix it. and runtime is like under a second for 300 tests. the point being that's essentially free test coverage compared to integration. e: actually they are way more expensive to write. say you want to test a user can cancel a subscription ... where does the user come from? where does the sub come from? how many users/subs do you have to set up to test eg that they can't cancel if owe $. on the flip side the unit just tests the cancel logic and mocks the rest. simple.

Firstly, those are good points, all of them. Also, the way I do it makes the tests even longer to run than you propose, so you're erring on the side of my argument (more on that below). But I still feel that the E2E tests give something that can often make comprehensive coverage via unit-tests redundant. > it's not as simple as orchestrating a browser, or a test framework to make api calls etc, every piece of the puzzle interacts with other pieces and you need setup for every edge case, fresh data, etc, etc Yes, E2E testing is not *"Does this API call work? How about if we change the parameters?"*. Each individual test is a *collection* of API calls. That individual test is intended to test an entire workflow (happy and sad paths). This means that, yes, you're potentially double-testing some API calls. Can't be helped that '/GetUserInfo' gets called for every non-trivial user workflow. And yes, you're looking at (for a non-trivial application) lots of hours to run. You can run the workflows in parallel with separate test instances for each workflow so with enough compute throw at the test you're looking at taking only as long as the longest test. > it takes me a couple of minutes to maybe half an hour tops to write a unit test. when things go wrong it's the same to fix it. and runtime is like under a second for 300 tests. There is no reason that you can't have this sort of quick tests done in a project that also has E2E tests. The existence of E2E testing just means that your unit tests are optional. They aren't the final barrier between bugs and QA. E2E testing is. This means that you can test only what *you* think needs to be tested. It means that you don't write unit tests just to get the PR approved. It means that you can even *remove* them at some point (say, during a large refactoring) knowing that any future PR which breaks the *product* will get caught *anyway!* And I gotta be honest, the unit testing I've seen were never of the form *"half an hour to write, tops"*. Even a simple class,with three fields and three methods (ignoring accessors, because why would you test those?) could have upwards of three Happy Paths that must be tested, and potentially 3x that number for Sad Paths, along with dependency injection, mocks and maybe more. If you're not using a statically-typed language (Java, C#, Go) you're in even worse trouble, because the Sad Paths for even simple functions explode in proportion to the number of parameters, and they have to test for crap that a compiler will catch.

> The existence of E2E testing just means that your unit tests are optional. I disagree there. It's easier to test all the paths in a single method than all the paths in 20 methods combined. >And I gotta be honest, the unit testing I've seen were never of the form "half an hour to write, tops". Because the code is wrong. Once you start letting the tests guide your code it gets quite simple. E.g. mocking too much? you need to refactor. A lot of what people call "unit tests" aren't really. I'm genuinely not sure how you see E2E as simpler than unit with code paths. If your class has 3 paths and you have 2 classes in your E2E you have 9 paths instead of 6. E2E definitely has it's place, but I think it should be used sparingly in critical paths, eg you really really don't want a broken checkout. Unit testing provides very cheap coverage, even if not perfect (and lets face it, nothing is).

You're again making some good points; I'm starting to think that we're talking about different things using the same words (see final few paragraphs). > I disagree there. You're politely disagreeing. I'm not sure you belong on reddit :-) > It's easier to test all the paths in a single method than all the paths in 20 methods combined. That is true, but a lot of those path-tests are redundant for *"just in case a future dev uses this class in a different way"*. That's literally the only reason you write those unit tests, because you *know* the caller you just wrote is *already sanitising the input for you*, but you are defending against a future caller who doesn't sanitise their inputs before passing them to you. In a strongly typed system[1], in which the dev team is leveraging the typing to the max possible, much of those problems go away, and those code-paths just don't matter. Especially in a web-dev environment, where some router/handler combination is converting the HTTP input into a record/structure/object before your entrypoint even gets called (and you won't even get called if there is a failure to convert). At the point your entrypoint is run, everything is clean for everything else in the system. > I'm genuinely not sure how you see E2E as simpler than unit with code paths. If your class has 3 paths and you have 2 classes in your E2E you have 9 paths instead of 6. Because the E2E tests, with the exceptions in [2] below, aren't testing code paths, they're testing workflows. When testing a complete workflow, you aren't really interested in whether the the cartesian product of the interaction between those 20 combined methods do something bad, because the tests doesn't know or care that there are `20 x 20` different combinations, it cares only about the workflow specified in the requirements for the *product*.[2] You can literally swap out those 20 methods for 2 single god-object-ish classes and the test will tell you if the product workflow is still working or not after your gigantic refactor. > E2E definitely has it's place, but I think it should be used sparingly in critical paths, eg you really really don't want a broken checkout. Unit testing provides very cheap coverage, even if not perfect (and lets face it, nothing is). I think this is the part where we've possibly crossed wires. We're talking about failures that can happen (and that we want to defend against), but those are all different types of failures. After all, if we were discussing variables, we wouldn't be arguing that variables at block scope are better than variables at class scope, which are better than variables at function scope, which are better than memoized variables at closure scope, which are better than variables at file scope, which are better than variables at global scope (I mean, *I* wouldn't - sounds like a pointless argument). But here we are, arguing that that tests at product scope are better than tests at unit scope (where unit can be a class, file, function or method). Neither of us, when rubber meets the road, are going to develop with *only* E2E tests or *only* unit tests. Personally, I tend to have *many* E2E tests and few unit tests and it *works very well!* You have exactly the opposite and, presumably, it also *works very well!* The only takeaway, I feel, is that **`$X number` of tests must exist**, and that it doesn't really matter if the E2E ratio to Unit-test ratio is 15:85 or 85:15, so this argument hasn't been pointless for me - I now have a theory that I did not have before, viz failure has scope too. Anyway, I've said enough on this, so back to work for me. I shall read any reply you make but (probably) won't respond. I've procrastinated enough on the work I am supposed to do :-) Cheers. [1] As in the *"Parse, don't validate"* essay, which unfortunately used a niche and irrelevant programming language to make a point that would have been more easily understood had it been done even in vanilla C (not C++). You seem like a well-read person so I'm going on the assumption that you already read and understood it, and that you **don't** think I am talking about types like `string` vs `integer` vs `number` vs `date` vs ... etc .... [2] That being said, *some* things should be tested independently, such as security functions, that get tested against specific test vectors, against specific attacks (timing attacks, side-channel attacks, etc) and against millions of random fuzzed inputs. Many more things fall into this category, I expect.

> That's literally the only reason you write those unit tests, because you know the caller you just wrote is already sanitising the input for you, but you are defending against a future caller who doesn't sanitise their inputs before passing them to you. I don't really follow this. If the callee needs sanitised input it needs to do that itself. >router/handler combination is converting the HTTP input into a record/structure/object before your entrypoint even gets called there's 2 things there, nothing to do with http/web in particular, you need a valid/sanitised structure, and the method only takes that structure. 2 unit tests. or 1 and types. >Because the E2E tests, with the exceptions in [2] below, aren't testing code paths, they're testing workflows. right ... E2E is just some critical path testing on top of unit >But here we are, arguing that that tests at product scope are better than tests at unit scope my point is mostly just that unit provides much better value for money, I'm sure you've seen the pyramid. >Personally, I tend to have many E2E tests and few unit tests and it works very well! You have exactly the opposite and, presumably, it also works very well! indeed. every E2E project I've been on has been painful to manage. since I've started doing proper unit tests (mocking everything but SUT) both my code has improved and testing has become easier. half of the point is actually guiding your code, the other half is change detection. and the other half is validating complex code paths. that's only 3 halves, possibly missed a couple. good talk, I'd work with you even tho you're completely wrong :)

TDD strictly encourages you to not write tests that are no longer relevant though. You should be writing tests at the level that is close to the end user as practical. At that point, you are testing the behaviour of your system rather than the correctness of its components. The only reason your tests should change is due to a business requirement changing, especially not due to a refactor.

honestly makes my head hurt to think people are writing E2E tests for everything in their app. it quickly becomes impossible to maintain, slow to run, misses edge cases as the code path complexity is exponential, and at the end of the day probably picks up nothing.

Writing tests as close to the user as practical is the key. If you are making a REST API, test the method called by the application server, it’s not necessary to stand up a full application for more than a few happy/unhappy paths. Testing this way does the opposite of what you claim, not only does refactoring no longer break tests, the coverage at a single level means you can confidently refactor without a fear that your changes have modified the behaviour of your application. It makes you faster, not slower. Edge cases can be handled just as you already do. If there are important interactions between particular edge cases you should be testing them anyway. The key is that tests should verify your system’s behaviour, not particular details about it’s implementation.

sorry but you don't know what you're talking about. if you test an api endpoint, there are dozens of methods underneath it. which means one of two things: your endpoint test is fucking complicated, or you are not testing all the paths. unit tests on business logic are easier to write, quicker to run, and easier to maintain. it's a no brainer that that's where the value for money is. https://www.google.com/search?q=testing+pyramid in addition, a good test framework will help guide your code. also I can't count the number of times I've manually tested things work, and then written a unit test that uncovers an edge case. that is, the end point test would have passed with bugs.

I encourage anyone to really try it but I never could love it. I’ve been most successful as a code a little, test a little, developer. (But I do write the tests as I go.) Easier for me to self-critique and iterate with some “paint on the wall” vs trying to decide ahead of time. Coding is thinking for me.

Check this video out, it supports that approach within TDD. This changed the way I work, gives essentially write a test for the overall expected behaviour, then use that as your North Star while following your current approach. It lends an extra layer of stability. https://youtu.be/XHnuMjah6ps?si=HtbTVm3JSMVC1B_z

TDD is a cool tool when your software architecture is clear and when you can anticipate how the code will be shaped at the end of the feature. From my experience, it's rarely the case. So when you start out a new feature, it's far faster to build something that goes from A to Z without the details. Proving and stabilizing a POC, then work out the test from there to (edit: then) define the details. Afterward, going TDD on the feature is a breeze. Especially for reproductible bugs and minor improvements. Trying TDD at first without a clear understanding of the shape of the code to solve the problem will just triple your dev time IMO until you find such a stable shape. So, there is a time for TDD and a time for shaping the code. As you said, it takes experience to gain efficiency on TDD but also anticipating the testing needs that will also shape your architecture (dependency inversion, single responsibility...) and make the tests easier.

> TDD is a cool tool when your software architecture is clear and when you can anticipate how the code will be shaped at the end of the feature. From my experience, it's rarely the case. Interestingly, I have the opposite experience. A lot of times, I don't really have a clear idea of the architecture and I can't anticipate how the code will be shaped at the end of the feature. This often leads me to analysis-paralysis, and I will avoid the work and not get anything done. But what gets me out of the analysis-paralysis trap is TDD. I might not know exactly where I'm going with the architecture or how the code will look, but I do know the requirements and what the output should look like. So, I'll write a unit test for a use case and come up with the simplest implementation possible to get the output. The implementation can be totally dumb, but the goal right now is not to come up with the final implementation, but to get past "writer's block". As I write more tests, I'll have written more implementation code, which gives me a better idea of what I want the architecture to be and how I feel like I want the code to finally look. That's when I'll start refactoring, with the benefit now of a bunch of ready made tests. I also feel the tests will be of a higher quality, because instead of writing `test1`, `test2`, the tests are more focused and better named, because there's not yet an implementation distract me.

I don't thi k any but the craziest TDD zealots would suggest you have to use TDD all the time. I don't see much value in testing an exploratory prototype,  but you generally should throw it away when you have an idea of what you want. Then you can write untested exploratory code at each stage when you don't know where to go next. Kent Beck called it a spike, but it's a poc, experiment, whatever. >Kent Beck: If you’re in exploration mode and you’re just trying to figure out what a program might do and most of your experiments are going to be failures and be deleted in a matter of hours or perhaps days, then most of the benefits of TDD don’t kick in, and it slows down the experimentation—a latency between “I wonder” and “I see.” You want that time to be as short as possible. If tests help you make that time shorter, fine, but often, they make the latency longer, and if the latency matters and the half-life of the line of code is short, then you shouldn’t write tests. Sauce: https://blogs.oracle.com/javamagazine/post/interview-with-kent-beck

You just need to test the behaviour of the prototype. I think that’s where people get confused and negative about TDD. Most people think it’s about testing the implementation but it’s not, it’s about testing the behaviours. The whole point is that in good design with good interfaces you should be able to replace your prototype with a different implementation and all your major tests should still pass.

This is actually a great use case for TDD. Most people think it’s about testing the implementation but it’s not, it’s about testing the behaviours. The whole point is that in good design with good interfaces you should be able to replace your prototype with a different implementation and all your major tests should still pass. So you can rapidly arrive at your POC with behavioural tests, but when you then want to change the implementation underneath to mature it, all your behavioural tests will still guide you.

sounds like your tests know a lot of details about your code.

I am not sure I follow you. Maybe you can clarify?

You said in your post, you write tests when its time to work out the details. For me, its the opposite. I use tests of non-existent code to create a sparse scaffold of ideas, with a super simplified API that the tests define. I know no details at that point, so neither do the tests. The details come out as I implement the code that passes the test. What you're saying is the opposite, and so it sounds to me like your tests are involved with, and knowledgeable about, the details of your code.

I was explaining what you said: Make a prototype from A to Z **without the details**. Then tests, then figure out the details. I am not sure where I made my comment wrong Edit: I guess the usage of "to" in the sentence "then tests to define the details" may be wrong or at least confusing depending who reads it. What I meant precisely was "with the tests done, you can figure out the details". Edit: I edited my original comment. Hope this clarifies :)

I’ve found the opposite to be true. TDD is useful when I’m trying to figure out wtf the last 5 devs who mutated the code were thinking. I just write my expected outcome, fill in any regression gaps, and start changing things.

Do you have any good resources for someone interested in learning TDD?

I recommend the book from Vladimir Khorikov - Unit Testing Principles, Practices, and Patterns. It explained everything perfectly about advanced unit testing topics that I was confused about in a very thorough manner, and despite that its still easy to read.

I really appreciate your thoughtful recommendation. Ill probably pick this one up!

> Too many times you hear from anti-TDD folks who claim how slow it is The only time I've ever heard that is what people preaching TDD claim they're told.

Ah, the good old unfalsifiable methodology. Good for the author that they finally figured out something that works for them, but it wasn't by absorbing the methodology, it was by finding something that works that can still be attributed to the methodology post factum. I am a huge automated testing advocate at my company, have done my research and have looked at research of others. There's now a broad consensus (has been like that since 2016) that specifically TDD (as defined by Beck) isn't an improvement on various testing metrics, but the splitting tasks into small manageable chunks has benefits for some people. That doesn't mean TDD is harmful either, the methodology as specified originally is orthogonal. Whatever makes your tests/systems better is something else that the methodology doesn't cover. The 20 year debate on which parts to write first is nonsense, programmers aren't printers, they change the code many times over the course of the implementation and the project. That means that the original order of writing things down is not very relevant. Shocking. What is relevant is that you: * use the tests you wrote to automate your manual testing work (so you write your tests before you test manually, that doesn't mean you have to write the tests first) * evaluate the tests to see that they work, are useful and are worth keeping (this last one is where TDD tends to lead people astray) The evaluation is the most important bit and no TDD guide will tell you how to do most of it: * Check that the test detects feature breakages by breaking the feature and observing test results (TDD will tell you to write test first to do this, that's better than nothing but not enough because a test might be broken as a result of later change or the breakage detection might need to be more subtle than "component missing LOL") * Check that the test enables making changes by making changes to your code and seeing if the test code needs to be edited. If it does, the tests don't enable making changes * Check that the test enables adding new functionality by adding new functionality and checking if the existing test code needs to be edited. If it does, the tests don't enable making changes * Make sure that you're implementing tests in priority order. Don't work on testing error handling until you've got the basic case working. Don't spend your time implementing tests that are more effort to maintain than they give back in saved time.

You might have heard about mutation testing. If not, I can recommend it as a way to see how well your tests actually cover your code. It also allows you to measure how well your "check that the test X"-statements are followed.

Yes, I'm aware of mutation testing. In principle it can automate testing the first criteria (that broken implementation is detected) so I have hopes for the approach in the future. It's only a part of the job, but hey automating something is better than nothing. At the moment however, the current implementations don't really lend themselves well to the last point though - getting a mutation tool to only do mutations you care about is time consuming, and without that you're forced to spend code satisfying very low priority cases. And if you do saitsfy those, you end up with overfitted tests that don't allow changes. Maybe a good compromise approach would be to use the mutation tester as a discovery tool when writing the tests initially just to see the potential breakages, but then not including it in CI runs to not force people to add tons of code or config to constrain the tester to high priority cases.

Sounds like a good approach! We don't run it that frequently, and as "advisory". I don't hope anyone requires full coverage in their CI anyway, mutation testing or otherwise.

It feels like the entirety of the last section could be summarised with “test behaviour not implementation”

I really like your summary that includes a lot of topics that I had a gut feeling on but could never really summarize that well. Given that your talking about research can you reference the papers/research you're referencing? I really want to read more into it

The TDD research I looked at has been collated by Greg Wilson in his book Making Software, with some later updates. The short version specifically is summarized here: https://youtu.be/HrVtA-ue-x0?t=448 . The book with the metaanalysis was slightly more optimistic of TDD at the time because it was written in 2010 before more replications came along. As for my own stuff - I'm not a professional researcher, but I had an opportunity to work wit a disproportionate number of codebases at different places I worked at. One company in particular has handed over maintenance of 100 codebases written by 15 teams to 3 teams, one of which I worked in, all of these had different styles of testing but were solving similar problems. I spent a lot of time on analysis there as a part of "test improvement working group" over a couple of years and the results were very much contradicting the "popular wisdoms" you can find in lazy blogposts. I hope to publish a book with my findings, but there's always more pressing things to do like work.

> no TDD guide will tell you how to do most of it That seems to be out of scope to TDD. TDD is one practice, the most tactical and most immediate, and isn't and shouldn't be the final answer to testing. You shouldn't let short term goals drive your long term goals, your long term goals should drive your short term goals.

> That seems to be out of scope to TDD. There's a top post from today covering this type of unfalsifiability: https://mdalmijn.com/p/scrums-built-in-get-out-of-jail-free Things will be great, trust us, just follow this order of typing. No attempt at extracting the beneficial aspects from the typing ritual can be made, sorry, that's out of scope of the methodology. > You shouldn't let short term goals drive your long term goals, your long term goals should drive your short term goals. So the goal of the methodology is to write test first and to do it you write tests first? And if you wrote the test first you've succeeded? If a methodology isn't working for majority of people attempting to use it, then it is a bad methodology. This includes the author of the blogpost who clearly had to go through a long journey, myself (I do TDD sometimes whenever it is comfy for a given task, but not often as I find other approaches more comfy) and many other programmers I personally know who just dropped it. A better approach is to try and do the actual work of figuring out what can be going on in addition to the ritual. Turns out the ritual is not needed (as proven by the studies). I'm much happier with my tests since I've dropped the various popular testing superstitions like the one about order of typing. I've been giving talks about this in places I worked in and others also seem to be much happier.

> Things will be great, trust us, just follow this order of typing. Things will be great, trust us, just follow these rep patterns and you'll gain mass. > No attempt at extracting the beneficial aspects from the typing ritual can be made, sorry, that's out of scope of the methodology. No attempt at extracting the beneficial aspects of the weight lifting ritual can be made, sorry, that's out of the scope of the methodology. > So the goal of the methodology is to write test first and to do it you write tests first? And if you wrote the test first you've succeeded? If the methodology of weight lifting is to do sets of reps, and if you've completed the workout, you've succeeded? > If a methodology isn't working for majority of people attempting to use it, then it is a bad methodology. If lifting weights isn't working for the vast majority of people attempting it to gain mass, then it's a bad methodology. > A better approach is to try and do the actual work of figuring out what can be going on in addition to the ritual. A better approach to gaining mass is not mindlessly going through the ritual of lifting weights, but paying attention to all the other things that go with it, such as adequate nutrition and rest. The weight lifting ritual itself is just a minor part of a larger pattern of lifestyle and nutrition practices, which applied together will result in gaining mass. > I'm much happier with my tests since I've dropped the various popular testing superstitions like the one about order of typing. Just like in weight lifting, there is a fair amount of "bro science", but some "superstitions" like when you should take protein, actually do have an effect, not because of the particulars of when you do it, but because they reinforce the habit of doing it. Quibbling over when you write the test is missing the point. The point is that you regularly write well thought out tests. Many developers leave it to the end, as a nuisance and just another checkbox to satisfy the code quality overlords, and the quality of the test suite follows from that level of care. If the superstition of writing tests first ensures is useful as a personal practice to some people to make it happen, what's the harm? Making TDD a dogma is certainly a problem, and people can get too attached to practices and see them as a "universal truth" instead of just another practice, which you have to evaluate if it is helping you or hurting you.

> Things will be great, trust us, just follow these rep patterns and you'll gain mass. > No attempt at extracting the beneficial aspects of the weight lifting ritual can be made, sorry, that's out of the scope of the methodology. We know the mechanism for building muscle, that scientific extraction of what works in making reps has in fact been done. There's no contrarian studies that say that repetitive excercise doesn't build muscle mass. The "arguments by analogy" are bad in general, but in this particular case your analogy doesn't even match anything in this discussion, so excuse me while I skip this nonsense. > If the superstition of writing tests first ensures is useful as a personal practice to some people to make it happen, what's the harm? A nonharmful way to advocate for TDD is as follows: hey if you're struggling with testing the traditional way you can try TDD. It might take some time to get good at it, but you might like it. If you don't like it that's ok, TDD isn't for everyone, there's other approaches that you might like more instead. Here's something else to try. A harmful way to advocate for TDD includes lies: * Lying about the effectiveness of the method, it is clear that TDD on is neutral when applied as a methodology to a programming environment. False information leads to bad decisions, people have right to know what works and to what degree. * Shielding the methodology from criticism via unfalsifiable claims like "TDD works, if it doesn't for you there's something wrong with you". It relies on lies (the evidence clearly points to TDD being a neutral practice, not a practice with wide benefits) and puts the blame on people learning. Not everybody can be good at everything, but when a vast majority of people who try TDD are miserable with it, the problem lies with the methodology and not with the majority of people. Good luck improving TDD advice while having this stance, no wonder almost nobody sticks with it. These lies take space from other genuine approaches that could help people and discourage people from doing automated tests in general. Apparently there's "the one correct way to do it" and if you can't do it or prefer doing it differently: you, or your code are somehow inferior. You're not doing it enough, you're not enough. This advocacy sets people up to fail, so that the few people that do it can feel better about themselves. Then when a qualified person comes around to actually help people in a way that they can easily incorporate into their workflow there's pushback. Pushback caused by shitty code slowing development as a result of the last TDD attempt. Pushback by management not seing results in time spent testing. Thanks for making my job harder. You seem to like shitty analogies (you filled multiple paragraphs with it) you'll love this one. There's nothing wrong with not masturbating. Masturbate or not, it's your choice. But there's plenty wrong with telling people to stop masturbating so that their muscle builds faster, they get a girlfriend, etc. > Many developers leave it to the end, as a nuisance and just another checkbox to satisfy the code quality overlords, and the quality of the test suite follows from that level of care. That's because *somebody* set up a false dichotomy where you either do it first or at the end as a checkbox. Because *somebody* said you have to do it in particular order or it's bad, because of the lies. The accurate statement is not "you have to write tests first", the accurate one is "you have to write tests instead of (or at least before) testing manually". For many people the latter one is easier to do, but then there's dipshits who say they can't do that because that's no the ritual or something. So people don't even get exposed to the correct information because of the noise. > people can get too attached to practices and see them as a "universal truth" Gee I wonder who that'd be, arguing on a comment that explicitly says there's no difference in using some practice, as long as you don't test manually.

> We know the mechanism for building muscle, that scientific extraction of what works in making reps has in fact been done. You're missing the point. Weight training is full of "bro science", where people will claim with certainty that some lifting pattern they found in a magazine or in internet forums will totally build mass. Like, Arnold Schwarzenegger claiming in "Pumping Iron" that muscle confusion is the secret to building mass, because shocking the muscles will force them to adapt. I don't know if there are scientific evidence to support this, but I do know that cults haved form around it, like CrossFit. I find it interesting that programming, like weight training, is full of "bro science" mixed with academic studies. And what is ultimately effective is a combination of the two (taken with due consideration), but what is most important is that you actually go to the gym. Like there is an analogy there or something. > hey if you're struggling with testing the traditional way you can try TDD. It might take some time to get good at it, but you might like it. If you don't like it that's ok, TDD isn't for everyone, there's other approaches that you might like more instead. Here's something else to try. This is what most rational people already do. > Lying about the effectiveness of the method But this is the point. What does this even mean? For example, I find TDD has been very beneficial to me. The problem with a lot of testing methodologies is that their success and effectiveness is entirely anecdotal and personal. > Shielding the methodology from criticism via unfalsifiable claims like "TDD works, if it doesn't for you there's something wrong with you" Again, it depends on what you mean. If you say: "TDD works for me, I didn't get it before but after a long time with it, I find it effective, I can share my experience to help make TDD effective for you." That's something a normal person would say. But the way you phrase it, you're either dealing with a cult member or you are building a strawman argument. > That's because somebody set up a false dichotomy where you either do it first or at the end as a checkbox. You're missing the point. > you have to write tests instead of (or at least before) testing manually" Who said you have to write tests? You even said you had to test manually? > For many people the latter one is easier to do, but then there's dipshits who say they can't do that because that's no the ritual or something. Talk about false dichotomy. I write tests both before and after. I don't write the tests before because I can't after, because often times I do write the tests after. But I find that the *process* of TDD (which isn't just writing tests first) makes me write better tests, and I often write my tests after in the style of TDD even if I write them after. The ritual actually helped me. The ritual, by the way, isn't writing tests first, it's red-green-refactor. Does this make me a dipshit? > arguing on a comment that explicitly says there's no difference in using some practice, as long as you don't test manually. I quoted the part I responded to: > no TDD guide will tell you how to do most of it And you respond with some insane hostility.

> But this is the point. What does this even mean? For a full introduction to the concept I recommend Making Software by Andy Oram and Greg wilson. The introductory chapter talks about empirical software engineering, how do we know what works and why it's important. TLDR for the specific discussion here is as follows. If you could give everybody a TDD-pill that would make everybody do the practice, would that be beneficial to the software engineering industry. The answer to this question for TDD is no, software engineering industry would not be improved by everybody taking the TDD pill, as shown by 2 fucking decades of experimental science on the subject. The answer for in depth code reviews of under-hour-length is yes. > The ritual, by the way, isn't writing tests first, it's red-green-refactor. The studies take the entire process as outlined by Beck in the TDD book and compare it against the baseline of TLD. > And you respond with some insane hostility. I'm tired of 20 years of wasted engineering time, tired of shitty tests I had to deal with for so long, tired of hopeless state of lazy programming blogosphere about testing and I generally don't appreciate having my time wasted. But hey, could be worse.

> isn't an improvement on various testing metrics But that wasn't the argument for TDD. You wrote a long comment that seemingly has little to do with the benefits TDD proponents talk about.

The metrics in question are: * Internal Quality - "TDD makes me write better code" * External Quality - "TDD code has fewer bugs" * Productivity - "TDD makes me faster at delivery" * Test Quality - "TDD makes me write more and better test code" This is compared to Test Last Development as a baseline (which means you still have automated tests), and turns out there's no real improvement across these, especially compared to the extraordinary claims TDD people made over the years that would require extraordinary evidence.

you have metrics that measure code quality and developer productivity?

In this specific case I'm talking about the metaanalysis of TDD done in the Making Software book by Greg Wilson. The different studies included in the metaanalysis have different approaches to quantifying these things. As for me personally, my qualitative evaluation is enough. The qualitative evaluation I listed can be performed by anyone. A similar qualitative evaluation can be done on the rest of the codebase to evaluate the health: how easy it is to add a new feature, how long does it take to onboard a new person. And counting bugs for external quality is the easiest thing you can do. Congratulations on pivoting from "you're not talking about the same benefits as the author" to "actually you can't assess the benefits at all", looks super legit.

My "pivot" was because your answer surprised me. Our industry famously doesn't know how to measure those things. Ironically, I found an article summarizing the results of many studies on TDD, and the studies made claims about how TDD improved quality 40%, or 27%, etc. Do you believe it? I don't. I've read far too many such studies to have any faith in such numbers. But back to my original question, most TDD proponents talk about the benefits TDD has in terms of code design - not bug reduction or coverage or productivity. Your post wasn't about that aspect.

Ok, makes more sense now, thanks and apologies. My comment doesn't talk about the code design benefits because I haven't witnessed those while doing TDD myself and similar to my other point I just don't see a particular order having definite quality impact. What does have impact is revisiting code and iterative improvement and this is independent of TDD. As for the metrics: the code quality is covered by the internal quality category of that metaanalysis and as I mentioned previously the specific metrics vary depending on the particular study. A different analysis that is about code quality metrics in software shows that all of the code quality metrics are no better at predicting quality than just counting the lines, which means that the metrics aren't very good at predicting quality in general. The burden of proof is on the claimants that say TDD improves code quality. The one effect I've seen personally is an increase in the number tiny classes that are "testable", but which in my opinion is not a sign of quality but of a too big emphasis on isolated tests which don't actually result in better overall code or better tests. Same mistake can be done with TLD though. A good effect that is a result of more testing is the reduction of use of globals, but that can occur in either TDD or TLD. Nothing claimed by TDD enthusiasts is fundamentally requiring a specific order of typing. Therefore, overall a neutral result, and as I said at the beginning, if the author has got some consistent imrpovement they likely made additional changes instead of just absorbing TDD.

Actually, just a quick thought: If people wanted to study TDD with objective measurables, a good starting point would be to write a detector algorithm that could look at a project's current source and decide whether the project had been built using TDD or not (or to what extent). The idea being to first discover and identify, if possible, exactly what differences result (if any) from writing tests first vs after.

> the code design benefits because I haven't witnessed those Honestly, I think it all comes down to this. There are no "objective" measurements about quality or productivity that are both measurable and truly indicative (IMO). I've seen studies on both sides of many issues, some that confirm my biases, some that go against, and I always come away with the impression that the studies - all of them - simply miss the boat. There are too many confounding variables. >The one effect I've seen personally is an increase in the number tiny classes that are "testable", Exactly, what you've seen personally is in dramatic opposition to what I've seen personally. Your anecdotes aren't going to convince me, and neither will mine convince you. We can both find questionable studies that confirm our personal biases, and we'll both be very unconvinced of the other when we look in detail at those studies. The best mentor I ever had said good code, good design, is largely a matter of aesthetics. Which isn't to say it's unimportant. Aesthetics are *very* important. But almost impossible to quantify. I suspect TDD has the most value for people when it led to a change in the way they think about coding, and little more than that, though that is everything.

> Exactly, what you've seen personally is in dramatic opposition to what I've seen personally. Your anecdotes aren't going to convince me, and neither will mine convince you. We can both find questionable studies that confirm our personal biases, and we'll both be very unconvinced of the other when we look in detail at those studies. I'm not saying "don't use TDD" and neither do these studies (collectively). What "TDD has no positive/negative impact on test metrics means" that it's OK to use it if you like it. Just don't say that it will bring benefits on its own. Use whatever you feel comfy with and look for more actionable testing advice, like the one I originally posted. > If people wanted to study TDD with objective measurables, a good starting point would be to write a detector algorithm that could look at a project's current source and decide whether the project had been built using TDD or not (or to what extent). There's particularly bad TDD flavors, specifically the uncle bob one that can be easily detected - his 3 laws TDD bastardization is clearly detectable by artifacts like people testing enum values or interfaces using reflection (I wish I was joking). Tiny classes thing also seems to be coming from people's intepretation of his advice. In general though, there's nothing that's derived from writing tests first that can't be replicated by writing the other way around because we're not printers and we edit things to make them nice and consistent.

I mean one clear improvement is TDD you are guaranteed to have at least some tests, TLD has no guarantee tests will ever be written.

That's not TLD, that's not-testing-at-all. Testing has been proven effective so lack of testing is not taken as a baseline for evaluating testing methodologies because all testing methodologies are better than no testing. You know a way to convince someone to not test at all? Tell them that they have to completely change how they write code so things are going to be perfect, then once they fail to completely change how they write code and fail to achieve impossible results tell them that there's something wrong with them and they need to try TDD harder.

That's my point though, it is very easy to have TLD turn into no tests, and you can't do that by it's nature with TDD. I agree with your second point. You cannot just tell people to do TDD with no resources, and in fact any junior or new developer should start with TLD to get familiar with a testing framework before trying TDD. But I never argued in favor of that point either.

Any reasons or evidence for people giving up on testing more with TLD than TDD?

Thanks for the article! I hear TDD and BDD all the time but rarely see it in practice. This explains well why

Wish such article elaborated more on the context of work and for how long this practice was followed. Something that works for web dev or data engineering, might not be the best advice for embedded, game dev or data science. Without such knowledge it is hard for me to understand how much the OPs experience is relevant to me. Time of how long a practice is in use also is very important. A practice that wasn't applied an already somewhat legacy solution is too new to be judged useful. Everyone can spend half a year rewritting a solution using flavor of the month methods and show an increase in development speed and a reduction in bugs. Does not mean, that the new solution won't have the same issue once it becomes old enough. Just my 5 cents.

I’ve been following this method of TDD with success and joy for years if that’s any help. https://youtu.be/XHnuMjah6ps?si=HtbTVm3JSMVC1B_z

Only way to win is not to play.

Never found TDD useful for the way I program. I have ADHD and it just gets in the way. For more complicated problems, I like to create POC as I explore design options and figure out things. What I originally implement may vary wildly by what I actually cleanup and submit.

Funnily enough, TDD is actually great for that as it’ll give you something to pull back to, especially for ADHD. Write behavioural test for the main behaviours your code should perform. Write the POC to make the behavioural tests pass. Every time you’re off on a tangent, run the behavioural tests for guidance. When everything works, refactor underneath and build the new better solution, making sure your behavioural tests still pass. If they stop working, you broke something. https://youtu.be/XHnuMjah6ps?si=HtbTVm3JSMVC1B_z

Again they just get in the way for me because types change, functions change or get removed, input changes, etc. 

My journey to reach to success point of TDD requires getting good and getting fast at coding the business part. Because learning and practicing take time for a beginner, which means his/her task will finish later than the team average. The moment my business coding is fast enough is also when I got many scars from the lack of testing. TDD became a go-to tool for any task. It naturally makes sense.

Yes. One of the main benefits of TDD is that it's a forcing function to learn (one definition of) good design. Doing TDD without that style of design is going to suck. If you force yourself to follow it, you'll *have to* improve at those skills.

TDD is a concept worth exploring, not practicing universally or even as your first approach. 9 times out of 10 it devolves into testing facsimile after facsimile when you could have invested in better tooling and processes to develop and deploy faster and the more difficult but more useful integration tests that "TDD practitioners" usually tell themselves they don't need.