Lean TDD is an attempt to reconcile some conflicting aspects of Test Driven Development and Lean Software Development.
I’ve mentioned Lean TDD on the podcast a few times and even tried to do a quick outline at the end of episode 162.
This episode is a more complete outline, or at least a first draft.
If you feel you’ve got a good understanding of TDD, and it’s working awesome for you, that’s great. Keep doing what you’re doing. There are no problems.
For me, the normal way TDD is taught just doesn’t work. So I’m trying to come up with a spin on some old ideas to make it work for me. I’m hoping it works for you as well.
I’m calling the new thing Lean TDD. It’s inspired by decades of experience writing software and influence from dozens of sources, including Pragmatic Programmer, Lean Software Development, Test-Driven Development by Example, and many blog posts and wiki articles.
The main highlights, however, come from the collision of ideas between Lean and TDD and how I’ve tried to resolve the seemingly opposing processes.
Transcript for episode 180 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:00 I’ve mentioned Lean TDD on the podcast a few times and even tried to do a quick overview at the end of Episode 162. This episode Episode 180 is a more complete outline or at least a first draft offer for test and code comes from SAS Labs, the end to end solution provider that helps development teams build digital applications so they work exactly as they should on every browser, OS and device. Every single time. They offer full stack and full software development lifecycle testing, including low code, mobile app, mobile beta, API, error bordering and monitoring, cross browser UI and visual and automation. Learn more at saucelabs.com SauceLabs test continuously test smarter. Develop with confidence.
00:01:01 Welcome to Test and Code Hey Lean TDD, what’s wrong with the old TDD?
00:01:15 If you feel like you’ve got a good understanding of test driven development and it’s working for you, that’s great. Keep doing what you’re doing. There are no problems for me. The normal way TDD is taught just doesn’t work. So I’m trying to come up with a spin on some old ideas to make it work for me. I’m hoping it works for you as well. I’m calling the new thing Lean TDD. It’s inspired by decades of experience writing software and influence from dozens of sources, including the Pragmatic programmer. Lean Software Development Test Driven development by example in many blog posts and Wiki articles and discussions with other people. The main highlights, however, come from the collision of ideas between Lean and TDD and how I’ve tried to resolve the seemingly opposing processes.
00:02:07 Test Driven Development What is Test Driven Development? Let’s run through TDD rather quickly. At least one flavor of TDD TDD can be remembered as red. Green, refactor, red, green, refactor. Red. Write a failing test. Green.
00:02:25 Write enough code to make the test pass and refactor. Change the code into something you’re proud of and then just keep doing that. The colors, if they’re not obvious, refer to the colors test runners often use to show passing and failing tests. Green for passing, red for failing. Some test runners show a progress bar that starts green, and as soon as one test fails, the whole bar turns red. We like green bars.
00:02:53 Green is good. Red is bad. Just for sanity’s sake, I thought I’d grab a definition of TDD from somewhere that at least sounds authoritative.
00:03:02 Here’s a definition straight from the Agile Alliance page. Test driven development refers to a style of programming in which three activities are tightly interwoven coding, testing in the form of writing unit tests and design in the form of refactoring. It can be succinctly described by the following set of rules.
00:03:23 Write a single unit test describing an aspect of the program.
00:03:29 Then run the test, which should fail because the program lacks that feature.
00:03:34 Then write just enough code, the simplest possible to make the test pass.
00:03:40 Next, refactor the code until it conforms to the simplicity criteria. I don’t know what that is, but that’s. Okay.
00:03:49 And then repeat this accumulating unit tests over time.
00:03:53 Okay, I’m just going to keep reading. Now as the next section on the page is expected benefits.
00:04:00 Expected benefits.
00:04:02 Many teams report significant reductions in defect rates at the cost of moderate increase in initial development effort. The same teams tend to report that these overheads are more than offset by the reduction in effort in projects final phases.
00:04:21 Although empirical research has so far failed to confirm this, veteran practitioners report that TDD leads to improved design qualities in the code and more generally, a higher degree of internal or technical quality for instance, improving the metrics of cohesion and coupling.
00:04:39 Well, that’s a mouthful, but I buy most of this.
00:04:43 Tdd has roots in test first programming. Tdd was once called a test first programming. It’s interesting to think about this name for a bit because it highlights what TDD was a reaction to.
00:04:56 One way software can be written is some form of waterfall process.
00:05:02 You find out what the customer wants and encode that in a set of requirements. Then you write up a design for the system. Then you code the system, you document it, you test it to make sure the system meets those requirements. And then when it doesn’t, you debug all the test failures and change the code as necessary and go back to testing it. And if the documentation doesn’t match the code, you refer back to the requirements and see who wins or change the requirements. Maybe change the docs the debugging part that debug all the test failures and change the code as necessary and then go back to testing. Sometimes that part can be up to half of the project time, especially if you end up having to rethink the architecture to hit a requirement you totally forgot about. The test part can be either manual testing or automated.
00:05:53 It was often manual and set in scripts. As in the scripts were documents that told a test engineer which buttons to push and what to look for. They weren’t automated scripts.
00:06:06 However, let’s be generous and even assume automated tests. If the tests were written and run only after the software is mostly done, then still that’s the total opposite of test driven development.
00:06:20 A clever development developer might think Wait a second, all of this would be easier if we had tests in the first place and use those during development. Yes, it’s easier to pass a test if you have the test before you start coding. Then you can just code until all the tests pass and you’re done. Except writing all of your system acceptance test is really difficult without a system. Also, requirements can change as the system is being developed, and then many of the tests need to be changed and it’s hard to know which ones when most of them fail early in the software development cycle.
00:06:58 So the TDD idea is to write the tests one at a time to then implement just enough to get it to pass and then continue and keep writing the test and then write the code.
00:07:09 Also refactor regularly while the tests are passing. Problem solved, right?
00:07:14 Except that’s not how TDD is normally taught. The tests in waterfall in the waterfall model were system level acceptance tests that match the customer requirements or the system constraints.
00:07:28 Tdd tests are unit tests. Usually there isn’t a lot of discussion with TDD and in TDD training materials that I’ve seen that talks about utilizing system level tests as part of TDD, or about tying tests to system requirements. They’re just this fuzzy term unit.
00:07:49 What is a unit?
00:07:50 In the early writing from KentBack and others in the extreme programming world, a unit test was a test written by a developer. This is, I assume, as opposed to the tests written by a QA team. There is some fine print in a lot of TDD literature that says the unit tests are not a substitute for independent verification.
00:08:12 This is one thing that always bothered me. This hand wavy someone else is also going to test the stuff. What if I don’t have someone else?
00:08:22 Isn’t this handing off to QA thing what we’re trying to avoid in the first place?
00:08:28 Okay, back to developer written tests. This doesn’t necessarily mean that a unit test isn’t a system test. However, lots of consultantlike folks reinvigorated kind of recently by the popularity of Scrum in some cases, and many books and blog posts specifically don’t include system tests as unit tests, even if the developer writes them. I personally thought that the term unit test was just a byproduct of all the unit test frameworks like Java’s, JUnit, Python’s, Unit, Test Library, etc. They have unit in their name, so a test that they run is a unit test.
00:09:07 But these tools can run system tests just fine.
00:09:11 But what’s a system test?
00:09:13 Even if I’ve given up on the term unit test as meaning anything other than a tiny test of one function or class, I don’t want to give up on system test.
00:09:23 There’s a lot of TD writing and test pyramid writing that treats system test as this nasty thing because you’re trying to use a third party tool to click buttons or enter text or stuff, and it’s through the GUI or web interface of your application.
00:09:40 Frankly, if that’s your only choice, then your architecture, I think is broken.
00:09:46 I would prefer to separate my GUI or CLI or web interface from the lion’s share of the code through an API that’s at a high enough level that I can write my system tests in Python and just call the API.
00:10:00 There will still be GUI or web interface tests, but those aren’t system tests. Those are tests of the GUI or CLI or web interface.
00:10:08 I still might lose this battle, though, of what system Test term means. So many people have seen the test pyramid, believe it without thinking, and believe system test should be limited due to high maintenance costs, but the maintenance costs they’re talking about are these GUI clicking things?
00:10:27 How about instead of system test? I call my thinking feature tests. I’m using the term feature test borrowed from a Twitter engineering blog post.
00:10:38 A feature test is a test run against the public API, verifying a service or library as the customer would use it. It uses as much of the system as possible, only mocking or using some kind of double where absolutely necessary to avoid an expensive service or something.
00:10:57 But I’m getting off topic now. Feature tests are not normally taught as part of TDD, even though I believe they were intended to be there as part of the classical interpretation of TDD. I think this is a good enough time to start talking about Lean software development.
00:11:14 Lean Software Development I was reading about Lean software development about the same time I was reading about XP, TDD and the Pragmatic programmer in the early two thousands and lead made a lot of sense to me. Here’s a Lean overview. I read about it in a book called Lean Software Development by Mary and Tom Popenyke. They start with discussing Lean in general. It comes from lean manufacturing, from the Toyota way and stuff like that. Lien has been applied to a lot of industries, and the Papa Dykes attempted to apply it to software.
00:11:47 I think they nailed the Lean principles pretty well. However, the Lean practices, which take up most of the book, seem a bit wacky to me. I think you can get a decent overview of the Lean principles by reading about the principles on the Wikipedia article on Lean software development, but here’s a quick rundown of them.
00:12:08 One, eliminate waste, two, amplify learning. Three, decide as late as possible for deliver as fast as possible. Five, empower the team, six, build integrity in, and seven, optimize the whole these all sound great.
00:12:26 I’d like to Zoom in and eliminate waste before continuing because a lot of people get confused about that. In the lean way of thinking, everything that is not adding value to a customer is waste. Let me repeat that waste is everything that is not adding value to the customer.
00:12:44 There are activities that add value to your development team or to your company. However, even if we value it, if it doesn’t add value to the customer, it is waste in the Lean practices sense.
00:12:58 So don’t get your feathers ruffled too much if your primary job falls into waste. Keep an open mind and think about it a bit and you’ll be fine. What is Waste in a Lean sense?
00:13:10 Lean Software Development lists eight categories of waste. Here they are one partially done work, two extra features, three relearning, four task switching, five waiting, six hand offs, seven defects, and eight management activities.
00:13:29 These all seem logical, especially if you read the Wikipedia article or the first chapter of the Pop and Dick’s book. These are not completely avoidable, but I think we can agree that a lot of stuff on this list doesn’t add value to customers, but does add cost or time to the completion of our code.
00:13:48 How does TDD fare against Lien?
00:13:52 Let’s look at the principles. I would say that of the eight principles, TDD does okay with empower the team? I think so. That’s nice.
00:14:03 I think TDD is probably neutral with eliminate waste. We’ll cover that separately, deliver as fast as possible, I guess. So we can build up a product if we prioritize the testing and features to implement early. To have a working system quickly, it kind of requires some oversight on prioritization and a mindset for getting complete systems working and milestones, et cetera. That’s not really part of TDD, but it’s often how it’s practiced. Also, there’s a lot of rework involved in rigid TDD, which I think totally gets in the way of as fast as possible. Also, tons of unit tests can get in the way of changing course. So there’s some good and bad of TDD associated with delivery as fast as possible.
00:14:53 How about decide as late as possible?
00:14:55 Well, we don’t make designed decisions until we need them for the test of the feature we’re working on. We don’t do a lot of upfront API design, maybe, but a lot of unit test build up can get in the way of medium to large refactorings if not kept to API endpoints. So decisions as late as possible, I guess. Maybe good.
00:15:21 How about amplify learning? That seems like a benefit. Why am I saying neutral? Well, if we tackle the hard stuff early, we can learn more about the problem space early. That’s good. We also kind of intentionally ignore experience for each part of the system until the tests tell us something isn’t going to work. So trying to forget about your learning so that you just pass the tests.
00:15:46 I’d say that maybe might be a negative. Not sure.
00:15:49 Okay, how about build integrity in? Well, we have a lot of tests, right? But unit tests, if we only have unit tests, it doesn’t really tell us if the product works. So I don’t think that that counts as system integrity. Maybe unit integrity, but not system integrity. And also, TDD is kind of terrible at the optimize. The whole I think we’ve covered that.
00:16:13 But TDD teaching often has the mindset of as long as the part works, the whole should work. But I just don’t buy it. Okay, let’s revisit TDD and waste.
00:16:26 To be fair, the kind of waste Lean talks about our mixed bag management activities. Well, neutral, maybe TDD doesn’t really affect that hand offs. It’s better if devs are also doing feature tests. Neutral. If they’re still handing off code to a QA team for final verification, how about extra features? Ddd is good with this on large scale, I guess. Maybe, but on the small scale, not so good devs can be pretty creative in thinking of imaginary worst case scenarios that we could write into tests to make sure our helper functions handles really large data sets, or imaginary numbers as such.
00:17:08 But what if the system only ever calls this helper function with a dozen or so elements and they’re all small integers?
00:17:15 You may have just over designed the hell out of that function because you want to write a bulletproof code bit that seems like it’s inching towards waste. Or maybe sprinting towards waste. How about task switching? This one kills it. Edd is almost founded on task switching. You write one test, then one code, then refactor and repeat. You’re always switching these behaviors of writing tests, writing code refactoring, but to me they’re different mindsets. I do agree that you shouldn’t go too far ahead of yourself with either tests or code, but sometimes you’re in the zone coding up a bunch of test cases. Why not? Why force yourself to switch to application code?
00:18:00 Well, because you might be going down the wrong direction. That’s fair enough. So don’t go too far.
00:18:06 Another reason often given is to keep the test suite green.
00:18:10 You don’t want your five new test cases failing, and then only get the first one green before the end of the day. Well, why not throw a skip decorator on them with a not implemented yet reason, commit it to your branch, and call it a day? Who’s harmed by that? Okay, I’m going to skip ahead to something not on the list so far, but probably should be as waste. And that’s rework. Tdd is all about rework.
00:18:38 That’s not necessarily a bad thing. I usually don’t know the best way to code something until I’ve already done it once.
00:18:45 But do I really have to write it six times or eight times, depending on how many tests I have?
00:18:51 Unless I’m reading it wrong, the refactoring part might mean I have to change a lot of code a lot of times, possibly once, for every completed test.
00:19:00 This at least has the potential for waste. What about all the design benefits of TDD?
00:19:06 Well, supposedly, since you’re writing small testable bits, the modularity that gives you leads you to better design. Maybe. But even the Agile Alliance admits that empirical research has failed to prove this one. I say go ahead and look at more than one test, more than one requirement to determine how to design some part of the system, but be willing to change the implementation if it starts to not work out. So I think know that a lot of refactoring and rewriting may happen, have tests in place to support those refactoring efforts, but don’t restrict developer experience, creative experience, and creativeness in the name of the simplest possible code that gets the test to pass.
00:19:52 By the way, small unit tests are inherently tied to implementation details, not behavior. This actually hinders refactoring, not enabling it. So let’s flip that around and focus on testing behavior and not implementation.
00:20:06 Which brings us to Lean TDD.
00:20:08 Lean TDD is like TDD, but with, I think, more pragmatism.
00:20:13 Here’s a Lean TDD overview. Let’s throw down some bullet point features so it’s not too hand wavy. One, have fun. Two, start with Tracer bullet tests and build out three, test behavior implementation. Four, develop automated test and code code together.
00:20:30 Five, focus on your most valuable tests. Write feature tests. Test your USPS.
00:20:36 Six, write tests when you need them, and seven, refactor when you need to. Do you need more than that? If not, run with it. If a refresher of all of these is necessary, well, let’s get into it. Here’s Lean DDD in a bit more detail. First, have fun. We’ve got into coding because it was fun. Make sure your testing is fun also, or you’re not going to want to do it.
00:21:00 Next, start with Tracer bullet tests and build out for new features. Start with a tracer bullet feature test that go through as much of the system as possible.
00:21:10 Then build out those test cases either at the feature you test level or other levels as necessary.
00:21:16 Next, test behavior, not implementation.
00:21:19 You want to be able to refactor, even if it’s big chunks of code or entire modules or subsystems or move functionality from one part to another. If it makes sense, you can’t do that with reams of tiny unit tests that focus on every detail of your implementation. Keep your tests about behavior. If you have tests at lower levels, try to keep them at the highest level of API. That makes sense.
00:21:47 I get trying to keep test fast.
00:21:50 There are ways to do that with larger tests, like in memory databases, local mocks of services, and reams of other tricks. It isn’t just small unit tests that make your sweet fast.
00:22:02 Develop automated tests and source code together. Tests need to support application code development.
00:22:09 Maybe it makes sense to write the test first. Maybe to try out what an API or class interface is going to look like to use as a client code before you actually implement it. Great, do that. But maybe it feels more natural to actually just try out stuff with the UI while you’re developing something and you want to capture that manual behavior those manual behavior expectations in a test after you write the code. Well, that’s cool. Why not? If you don’t trust yourself to actually write the code afterwards, then lean on coverage metrics to gamify it for you. If the code you’re writing is not accessible from the UI, maybe you’re writing a library or a module or function or whatever. Why not try driving it from test code, even if you don’t have asserts in there, test methods are great code snippets to drive other code or API’s. And when you’re ready, put asserts in for expected behavior. Pay attention to what outcomes you’re expecting to see happen.
00:23:11 How are you testing it? How do you think that it works? What changed in the system is telling you that the code is working and put that in the test.
00:23:20 Next, focus on your most valuable tests. Feature tests are like unit tests for units of functionality. Utilize them if they still seem too wide of scope to debug features, then Zoom in and implement a similar test closer to the code that you think is at risk. Do remember, though, that the more small scale you Zoom in, the more your tests are testing, implementation and not behavior.
00:23:44 Test your unique selling propositions. Your USPS. Make sure you have tests around your USPS of your application.
00:23:51 The reason people use your software over other software.
00:23:55 Write tests when you need them. Your tests are there for you. Use them. Let them help you develop more solid code faster. Let them help you learn. One of my favorite uses for test code is to test my understanding of the behavior of a data structure, or an API, or a driver or service.
00:24:14 These knowledge building tests may not test your system, but they’re still very useful, and it helps you to get in the habit of writing more tests.
00:24:22 Refactor when you need to. One of the great things about all flavors of TDD is the emphasis and permission to refactor. Don’t forget to do it, but it doesn’t have to be all the time.
00:24:33 Lots of churn in the code base can get confusing. Just don’t forget to do it. The best person to write a bit of code is someone who has just written it once.
00:24:42 Think of all your initial code as first drafts. Maybe it’s good enough, but it’s usually not. Are you proud of it?
00:24:51 If you’re proud of it, you don’t need to refactor. But if you’re not proud of it, maybe try to refactor. I’m hoping that Lean TDD makes sense for a lot of people that feel like other forms of TDD just don’t work for them. This description of Lean TDD is also written down as a blog post on Python test.com. I’ll leave a link in the show notes. Feel free to send me feedback, but be nice. If you hate the idea, then it’s not for you. I don’t really need to know about that. But if you’d like to give me constructive feedback, there’s a feedback form on my podcast site at testandcode.com/contact. I’m also @brianokken on Twitter.
00:25:36 Thank you SauceLabs for sponsoring SauceLabs test. Continuously test smarter. Develop with confidence. Learn more at Sauce Labs.com. Thank you, Patreon supporters join them at testandcode.com. Support all of those links are in the show notes at testandcode.com 180 that’s all for now. Now go out and test something.