Software development processes create value and have waste in a lean sense of the word. Lean manufacturing and lean software development change the way we look at value and waste. This episode looked at lean definitions of waste so we can see it clearly when we encounter it. I’m going to use the term waste and value in future episodes, and I’m using waste and a lean cents so we can look at software processes critically, see the value chain, and try to reduce the waste. Lean, as in lean manufacturing and lean software development, cause people to talk about and examine waste and value.

Even in fields where we didn’t really think about waste that much to begin with, software is just ones and zeros is their waste. When I delete a file, nothing goes into the landfill. The mistake I’m making here is confusing the common English definition of waste. When what we’re talking about is the lean definition, let’s try to clear up that confusion.

Transcript for episode 161 of the Test & Code Podcast

This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.

00:01:09 Welcome to Test and Code.

00:01:18 You may have heard that Agile methodologies like TDD Instagram, were a reaction to previous ways of making software, including big design up front and waterfall. Before the manifesto for agile software development. These were called lightweight methods. I started reading about all of this in the early two thousands.

00:01:38 At the time I was writing embedded code for an RF coms test instrument, specifically the code that sits between the protocol stack and the FPGAs and ASICs. Another hardware. Long lists of register definitions were needed. Lots of room for error testing was needed and needed during development, and I was reading in some order that I don’t remember the Pragmatic programmer, the Lean Software Development book test, first programming, then TDD, mostly from W two S, which was born cuttingham’s Wiki and also from Kim Back book, Extreme programming from various posts and books.

00:02:18 And also at the time, my company had everyone take some Six Sigma training. So I was thinking about all of these things at the same time, and so they kind of all blend together and I think of them all together. The only part of Six Sigma that stuck was to make.

00:02:33 I think it’s stuck because I try to have a small process of improvement project for my team.

00:02:40 DMAIC is an acronym for Define Measure, Analyze, Improve Control.

00:02:46 Normally it’s used to save money on a large scale, but I wanted to save developer time and effort on the daily level.

00:02:54 The process improvement project was just our code change process.

00:02:58 The process was make sure you have current code.

00:03:01 Create a branch, make a change. Build load the compiled code onto an instrument, restart the instrument. Run a smoke test to make sure it all passes. Commit the change. Merge from Maine to your branch. If there are any conflicts or any changes on main, resolve conflicts. Build load smoke test again. Merge from your branch to main.

00:03:22 If there’s any conflicts, there aboard the whole process and go back to the merge.

00:03:27 Remain to your branch.

00:03:30 Continuous integration wasn’t widespread use yet, so these were the steps done by developers, and the process was a bit of a pain. So lots of people took shortcuts, like reusing the same branch name for everything or not, testing or only mode merging from their branch to mean et cetera. I thought the whole process was important, so I wanted to try to make it easier. I wanted to make it easier so people would do it. So I tried out to make I measured time and keystrokes for improve.

00:04:01 I wrote a handful of shell scripts with short but descriptive names, and I kept going until the whole thing seemed easy to me, at least with no wasted effort. And then I remeasured. I had reduced time and keystrokes dramatically, but also the steps made more sense. Now in the work full was more obvious, with less steps to remember. I presented the findings and the scripts in a team meeting. I was probably the least senior person on the team at the time, so I expected people to say, That’s nice, but we’re fine. Instead, I got awesome. Thanks, and people just started using it. That was cool. I didn’t realize at the time, but the project was also a waste reduction project. It reduced motion, unnecessary movement by people.

00:04:50 It reduced weighting, it reduced extra processing, doing more work than is necessary to complete the task, and it reduced defects before testing, more before merging code domain waste reduction and software processes not only save time and money, but it can also just make engineers happier.

00:05:09 Making complicated workflows easier reduces mental burden. That’s one of the reasons I love automating the painful processes, and I love tools that help with that, like pytest, of course, and CI systems and make and talks and pre commit black and any tool that makes my developer life easier, like PyCharm, for instance.

00:05:36 – ad spot –

00:06:43 Let’s find some definitions of waste. There’s the common English definitions, the ones that seem relevant here from Miriam Webster, are damaged, defective, or super famous material produced by a manufacturing process, such as material rejected during a textile manufacturing process, or scrap, which is fragments of stock removed in manufacturing. Manufactured articles or parts rejected or discarded. An unwanted byproduct of manufacturing process, chemical laboratory or nuclear reactor. Like toxic waste, hazardous waste, nuclear waste. And then, of course, there’s refuse from places of human or animal habitation, such as garbage, rubbish, and sewage.

00:07:27 Well, in common English, waste is definitely a bad thing. So if these definitions are all you know, obviously someone saying that your work is waste is hurtful. The lean definitions are a little different.

00:07:40 Lean manufacturing takes its definitions from the Toyota Way. In lean manufacturing, work processes are redesigned to either eliminate or at least reduce waste through the process of continuous improvement.

00:07:53 There are seven types of waste listed in lean, overproduction weighting, unnecessary transport or conveyance, over processing or incorrect processing, excess inventory, motion and defects.

00:08:08 Now we’re getting somewhere. These are not waste in the sense of scrap, garbage, or thrown a work. There’s more nuance. Let’s try to think about these with software so overproduction. Actually, I’m really not sure how this fits into software, but maybe somebody else has an idea. Waiting. There’s lots of waiting in software development, writing code and waiting for someone else to test it. Waiting for CI to test alone on all the platforms, waiting for compiles features, waiting on one subset system to be implemented, waiting for bugs to be fixed.

00:08:43 It’s a long list. We wait a lot.

00:08:46 How about unnecessary transport? There’s some nuance here in room for interpretation.

00:08:51 Actual transport might be copying files from one system to another, or from repos or or copying source into a Docker image so we can compile it on a different OS.

00:09:03 But there’s also transport between teams and people.

00:09:07 My part is done. It’s your term, so maybe that counts.

00:09:11 How about over processing or incorrect processing? No imagination needed here. There’s always a lot of that, like, let’s say, premature optimization, or maybe generating tons of system tests before the API is really stable. Excess inventory. Well, maybe that’s similar to the last one.

00:09:30 There might not. And it also just might not apply to software. But if you think about it, think about maybe different teams or stages. And it makes sense, like building one subsystem to completion before even starting the API or the UI or other parts. That seems like excess inventory. This is kind of really where tracer bullets and skeleton implementation help building just enough in all the layers and not charging ahead too far in one of the layers.

00:10:01 How about motion? This also could be like transport between teams, but also just motion and effort. An occasional long command is fine.

00:10:10 The common command should be short or automated. And of course, defects. That’s kind of obvious. But also, why are the defects happening? Is there an effective communication between teams causing misunderstandings? Is there not enough testing done at the local level or subsystem? Those are good things to look at.

00:10:29 So that’s kind of the definition of a lien from our definition of waste from Lean Manufacturing.

00:10:37 There’s also Lean six Sigma, which was an idea of combining the six Sigma ideas with Lean, and it added a new type of waste. At least I think this is where it came from, and this waste is non utilized talent.

00:10:51 This is interesting. So this is defined as a waste of human potential and skill, including when employees are not given the opportunity to provide feedback or recommendations to improve the process.

00:11:04 Lack of training, lack of incentives for employees, like placing employees in jobs or positions that do not utilize all of their knowledge or skill. So this is terrible. And I definitely think that we need to think about this type of waste and software. There’s also a nice extra definition for extra processing, which was extra processing is doing more work than is required or necessary to play a test.

00:11:30 Examples include double entering data, unnecessary steps in production, unnecessary product customization in using a higher position equipment than necessary.

00:11:41 I think this is applicable to software as well. And of course, we also need to talk about the book. So here’s where I kind of started thinking about this is a book called Lean Software Development, and specifically, they defined waste as I’ll just list them without defining them too much. But there’s partially done work, extra features, relearning task switching, weighting handoffs defects, and management activities.

00:12:11 I love that.

00:12:12 It’s interesting that management activities are in there. I think I’m kind of offended, but, you know, whatever management is necessary, but obviously over management can be a problem, so it’s nuanced.

00:12:26 So for example, one engineering, eight managers might have some trouble, but also eight engineers and one manager could be an effective team. But why stop there went out 80 engineers and one manager.

00:12:37 Well, that sounds bad too, so I don’t know where the line is, but, you know, kind of. Now I’m kind of annoyed the management is in the mix because that could be said of any sort of improper loaded mix, like any front end developers in one back end or one one front end and 80 back end. It’s all bad. You kind of need balance.

00:12:58 What about software tests, though? I love it when engineers and managers are writing tests.

00:13:04 But how many tests more is better, right?

00:13:07 No, it’s also balanced enough tests at each level and at each stage of development to have confidence in the behavior of the system and the components.

00:13:15 What does that mean? That means don’t write all of the subsystem test before you start implementation. We don’t start all the system test before you start the implementation and write the tests for the features you’re working on, not features way in the future. Right? Enough subsystem test so that someone working on that subsystem can reasonably be confident in a change in that system won’t break the rest of the things like that.

00:13:44 Also, let’s talk about value and waste before I get too far into the weeds.

00:13:49 Let’s do an example of a different kind of product. It can be anything. Let’s take a roll of paper towels.

00:13:55 I buy a roller paper towels. It has paper towels. That’s why I bought it. But also as a rapper and a car bar tube, I’m not buying it for the rapper of the tube, but both are necessary. The tube making it stronger or thicker or fancier material, or maybe adding sparkles to the tube.

00:14:15 I don’t really care about that, so improving the tube doesn’t make a lot of sense.

00:14:19 That’s not where I’m getting the value. The value comes from the paper towel.

00:14:23 Improve that if you want to improve something, but improve the tube. And if you just remove the tube, that’s not good either. I’ll just have a limp roll of paper towels and it’s going to get hard to deal with. So I need the tube.

00:14:37 Even if I don’t really think about it too much and thicker, thicker probably would be bad. I’d like to be able to squish it afterwards and toss it in the recycle bin. About the Rapper, A minimal wrapper that doesn’t tear but is easy to take off is good. Rapper seems like waste. I throw it away immediately, but it’s still necessary. But if you triple wrap it, it would obviously be accessible and wasteful, but some brands only wrap the outer pack and not the individual role.

00:15:07 Well, for me personally, I don’t like that because I store the individuals in the garage.

00:15:13 But for some people, the role rap is just too wasteful. So there’s balance, of course, improving the design of the rapper or the logo or the picture on it that might entice me to buy it. So there’s financial value in the rapper for the company that isn’t obvious.

00:15:32 It isn’t obvious value to me as a customer.

00:15:35 So even though the roller paper towels, it’s not always clear where the value and waste is. And there’s balance back to software. The source code is value if the application is valuable, obviously, I think.

00:15:53 But really, a customer may just be getting an executable and don’t really care about the source code. So the source code waste twice the amount of source code is not adding value if the behavior is the same. And I never heard somebody brag.

00:16:10 That’s a really cool application, but I can write the same application with ten times the amount of code. That would be weird. What about test code?

00:16:18 It’s also necessary, but not obviously valuable to the customer.

00:16:23 I mean, they expect it to be tested, but is ten times the same source test code.

00:16:29 Good. Probably not. If it isn’t a testing, anything more is either source code or test code waste.

00:16:36 Well, okay, maybe I’m going too far into a touchy direction. I don’t want to go in. They’re both part of the value chain that ends in an application.

00:16:45 The value chain will have processes in it that can be improved, and we want to do a least amount of work as possible to get the end result happening. Actually, why do I not want to talk about it? I’m a little uneasy because because it gets into that weird definition between, like, the common definition and the normal definition. The waste is garbage versus waste is just part of the process that we’d like to reduce. And that’s where I think things get touchy. And also we’re trying to eliminate waste. And if I say that any part of somebody’s job is waste, then we’re trying to eliminate it.

00:17:23 Nobody wants to hear that. Nobody wants to hear that the work they’re doing is waste.

00:17:28 But I do like automation and making our jobs easier. So I would say let’s stop talking about elimination and have it be more about moderation and balance.

00:17:40 It’s also about getting ideas into working applications and to customers with as little friction as possible, and also, hopefully the least amount of work, effort and time and cost.

00:17:52 So when I in future episodes, I’m talking about value and waste.

00:17:57 And or if you hear that from other people, please don’t jump to touchiness or defensiveness too soon and think of it as a nuanced thing that we want to produce value with the least amount of work possible. That’s a good thing.