Within software projects, there are lots of metrics we could measure. But which ones really matter. Instead of a list, Benjamin Harding shares with us a way of thinking about business outcomes that can help us with every day decision making.
Transcript for episode 134 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:00 Within software projects. There are a lot of metrics we could measure, but which ones really matter? Instead of a list, Benjamin Harding shares with us a way of thinking about business outcomes that can help us with everyday decisionmaking.
00:00:26 Welcome to Testing Code.
00:00:38 Testing Code. I’m proud to welcome Ben Harding. Ben, could you introduce yourself before we jump into things? Yes.
00:00:45 I’m Benjamin Harding. I work as a team leader at Ryan. I’ve been a front end developer for about eight years based in Wellington, New Zealand. Looking forward to having a chat with you, Brian.
00:00:56 Yeah. So this is kind of an interesting topic. We’re going to talk about business outcomes and why developers should care about those and stuff. And I’m not actually sure where to start this, so I’m going to let you leave.
00:01:08 This sounds great. What we wanted to talk about today is business outcomes and why developers should care about them and more in particular way, they should worry about test and code them when developing their work instead of what’s termed vanity metrics. Now I think what we can do to begin is kind of define. Now, what do I mean by a vanity metric or what do people mean when they term vanity metric?
00:01:33 Okay.
00:01:34 I’m not sure if you have a particular idea around what you testify some vanity metrics brand. But for me.
00:01:40 Well, let’s see. I’ve got over 500 LinkedIn connections. Is that a vanity metric?
00:01:46 Well, it certainly can be.
00:01:49 I’ve also been part of our other work environments where they’ve wanted to advertise how well the project is doing, how well somebody’s contributed by the number of code commits that they’ve made. They wanted to have this leaderboard via this person made these amount of code commits. And to me, that seems like a pure vanity metric, because number of code commits does not mean that they are necessarily good code commits or doesn’t affect the thinking behind them or the amount of effort, because each person can have a slightly different in what they commit, whether they commit everything in one or they break it down a bit more.
00:02:29 All right. Yeah. Are there other things that you’d be lumping into the vanity metric? Yeah.
00:02:34 So I would imagine when you’re working on a project, some of these vanity metrics would be classified more as outcomes, which don’t matter so much in a business sense. So these entity metrics can relate more to each individual project and how they’re defined. But if we maybe come back and look at business outcomes, when you’re working on a project and you’re developing a piece of software, the business that you’re working with, they were working for, there’s a particular reason why you’re developing this bit of code or working on this project. And so when you start looking at it from a business sense, you want to try and scope what you’re developing and what you’re working on for that given outcome. Now this could be customer retention. It could be acquisition for me, business outcomes, the way I typically see them, because most businesses are to make a profit slightly related to money or cost. And so whether that is gaining more customers on board or getting more people to view your website sort of tangible to this whatever business performance you want. And so vanity metrics, in the sense of what we’re talking of today, will relate to metrics that are easily gained, or metrics which are easily influenced, which really don’t matter to whatever project or bit of work that you’re currently doing. Hopefully that makes a bit of sense to you, Brian.
00:04:10 Okay, so a metric that can be gamed in some way but doesn’t affect my business outcome would be a vanity thing.
00:04:21 So I don’t know, what do I care about that? Let’s say it’s a web app, and if I’m just looking at my Google Dashboard or something, I can see things like number of pages viewed per person or on average or something, or the number of visitors or how long the bounce rate and things like that. Are those vanity or do we care about those?
00:04:47 I think those metrics are really valuable. Okay.
00:04:49 Some of them are more valuable than others, and it depends on context.
00:04:54 And they are what you need to consider when you’re thinking of each project. If you’re working on updating a home page of your website, the outcome you might have for this website redesign that you’re having is to increase customer engagement. And so business outcome there might be increasing the number of page views in order to increase the conversion rate or number of people who view the website and then go on to actually signing up. And so pay pews would be a good metric, but probably the business outcome you’re looking for there is increasing the conversion rate because for that project, you would like to see more people actually flow through from your website into your sign up form. Or maybe in the case of a podcast, you’d be looking for more people signing up and listening the whole way through or viewing your listing and then actually hitting that play button.
00:05:48 Okay, yeah, those are interesting. I don’t know if I even have ever looked at that.
00:05:53 Well, the podcast example, I’m not entirely sure about whether you can do having never been on the site or actually recording it. So you might know a bit more about that. But I know from being a developer and having integrated Google Analytics many times the page views are great, but when you can start hooking them up into a funnel and you can start seeing the percentages of people who then view your website actually signing up, sometimes it’s not the actual amount of page views you want to drive, it’s the amount of people who view your site to then signing up. And so when you start looking at your projects from the lens of what is the business outcome we want to achieve that’s when you can start to reevaluate whatever solution you’re given or whatever solution you’re actually trying to build and start to rethink, will this actually help drive that outcome that we would like, or do we need to rethink the solution? Not the best possible path, and it’s actually just a bit of busy work.
00:06:53 Okay. Actually sounds hard.
00:06:56 It can take a bit of time.
00:06:58 Definitely. As developers, it requires a little bit of mental context switching sometimes, and so having to look back into it. Hang on, why am I doing this?
00:07:09 Why does this fit in the cog matter?
00:07:13 Should I reevaluate or look into something else?
00:07:20 Highcharm is the Python IDE for professional developers high Charm’s huge collection of tools out of the box includes an integrated debugger and test runner, Python profiler, a builtin terminal, integration with version control and built in database tools, remote development capabilities with remote interpreters, an integrated SSH terminal, and integration with Docker and Vagrant. In addition to Python, PyCharm provides first class support for various Python web development frameworks, specific template languages, JavaScript, Coffee script TypeScript, HTML, CSS, Angular, JS, NodeJS, and more. High Term integrates with a Python notebook and has interactive Python console and supports Anaconda, as well as multiple scientific packages including Matt, Lot, Lib, and NumPy. Try out all of these timesaving features of PyCharm Pro for four months with the link testandcode.com PyCharm. Make sure your editor is working with you to save you time. Use PyCharm also, how would I as a developer, I mean, how would I even change what I’m doing to try to care more about business outcomes?
00:08:29 So probably a couple of things you could do as a developer to try and actually care more about them. When you’re picking up a bit of work, you could start to question the importance or maybe not question so much as consider the bit of work that you’re doing and what is the actual driver behind it. And so a couple of good questions that you can ask. There are what are the business drivers for this work and how would we measure success?
00:08:54 How do we know that whatever project we’re working on is going to be when we know it’s a complete all? When will we know that it’s failed? Yeah, because I’m sure in my past experience and you might be similar to this. Brian has made projects in the past where you’ve worked long and hard many hours, and then the moment you release it out in the wild and then a couple years later, you then just end up ripping them out or you see them rock down because they didn’t end up meeting a success. And it can be a little soulcrushing at times, and it can be a little hard to then see this project that you put hours of love into just suddenly fizzle out and not become a reality because those business outcomes went and so when you can start to think about bigger picture, that’s when you can start to allocate the time more efficiently.
00:09:47 Yeah. I’m actually thinking about a few examples. I’m not going to name them, but there are services that I’ve used, Web services usually that worked great, but they were slightly ugly and had some redesign happened after they got I think they started making enough money to where they can start bringing on a designer or something. And the changes made it the site look prettier, but it made it clunkier slower, didn’t load as fast, took away features.
00:10:21 You have to be careful when changing an existing system to make sure that you’re not hurting your business outcome or your customer value to hit some other metric or something or just make it look more professional, I guess I’m not really a user of this, but I think an example that some people talk about is Craigslist. It hasn’t really changed much for a really long time, and I don’t think anybody would put Craigslist as one of the top, most beautiful sites or anything like that. But it’s very functional and it hasn’t lost that. So I guess be careful with that.
00:10:57 Yeah. And if they did try and change that sort of design that they have around Craigslist, I could imagine there would be some people who would possibly be up in arms about it because they’re so familiar with the existing interface that they’d see that change is sort of a pushback.
00:11:13 Yeah. One of the interesting things around that. I know we weren’t really going to talk about design, but even though Craigslist doesn’t look like super pretty compared to a lot of other sites, the thing that is replacing the old classified ads, those were never pretty either. They were just like these little blurbs of text. And so at the very least you’ve got blurbs of text with pictures now, so more functional that way. But we’re kind of talking about development and software and business outcomes. I’m thinking about nebulous things because it does apply. I thought we were just talking about websites, but now that I’m thinking about it, it applies to all software efforts, even like embedded systems and other things you can think about. It’s hard, though. The business outcome is important, but I’m trying to tie in customer value as well and making sure that we’re thinking about our product in terms of how the customers using it, hoping that satisfying customer needs is going to increase business outcomes somehow, yes.
00:12:17 I think the two are very related because the business outcome is tied to the customers need and the value that they see. So if you do increase the amount of value that the customer sees or perceives or enjoyment in your product or its performance or possibly how it looks or how it feels, then inherently they are more likely to either stay on your site or keep using your product in whatever way, shape or form that is. And so I think it is quite valuable to remember that these business outcomes are tied to customer engagement and the customer outcomes as well.
00:12:55 So do you recommend things like when people are using a B testing, for instance, to make changes that the value that they’re measuring as a success or failure are closely tied to business outcomes and not the metrics that don’t really matter, then?
00:13:10 I think it’s one of those things where it depends on the case, like a B testing frameworks. And testing is super valuable because it can help reduce the number of incidents. And the quality that we produce shouldn’t in most cases be influenced by the business outcomes. Now, there are some cases where you might be writing a one off tool or you’re writing a particular project piece of code that has a lifespan of only maybe a couple of minutes or a few hours, in which case you might see that reduced need to put in that maybe sort of quality. But it should always be there inherently. And the value in considering the business outcomes is when it comes to doing writing those AB tests and writing the actual testing frameworks is when you have to communicate that value to shareholders or to people more senior than you who would start questioning those frameworks. You can start to frame it in language and terminology that they understand.
00:14:10 If you don’t write a test, then your errors can come in and regressions could occur. And so therefore, people, the website or your application can have a negative impact on the customer because you are removing the safety nets. And so by being able to frame it in terms of a business outcome, such as incidents could increase and start coming through and customers could start seeing oddities, which would be easily brought up. That’s where the value comes in from being able to communicate in a business sense and reframe the conversation instead of saying, oh, right, test because tests are good, you can say we need to write tests because they help reduce incidents. And every minute of downtime or the moment this doesn’t work will impact the business in this regard.
00:15:00 Yeah.
00:15:01 Okay. I’m also thinking about like, cost now. So trying to reduce cost on efforts that might not matter so much or not matter so much in the business outcome side. For instance, testing, even making sure that you’ve got like going from 97% coverage to 100% coverage might be several months worth of work, but if it doesn’t affect the customer at all, it’s just work and with very little benefit sometimes.
00:15:30 Yeah. And it’s all about the value that you get out of a bit of work versus the cost it takes to build. Going up from that 97% to 98% code coverage, it takes hundreds of hours build and you won’t get much of an increase. Then it might not make sense to do that versus adding test and code other area or improving the feature in some other way, shape or form.
00:15:51 Yeah. Interesting.
00:15:55 I want to tell you about a really amazing sponsor I’m partnering with. Monday.com is an easy to use, flexible and visual teamwork platform that powers teams to run processes, projects and build custom workflows in one digital workspace beautifully designed to manage any team, organization or process online. Monday.com powers over 100,000 teams daily work, and they just launched a contest to build apps that will be included in their Marketplace launch. You can build an app that can improve the way teams work together on Monday.com. Whether it’s an app to help marketing, construction, sales, software developers, or anything in between. They are looking for creative impactful, amazing apps to feature in their upcoming apps marketplace. They’re giving away $180,000 in prizes, including three Tesla, Ten MacBooks, and more. Have you ever dreamt of building an app that impacts the daily lives of hundreds of thousands of people? Well, now’s your chance.
00:16:51 Check it out at Monday Comtestcode and start building now. That’s Monday Comtestcode.
00:17:01 So you said you’re a team lead or a lead engineer or something. You manage a bunch of people or some people.
00:17:07 Yeah. It team leader, Regan. We build a software solutions for other developers.
00:17:13 Yeah. Reagan has one of the best logos of any company, in my opinion.
00:17:18 Thank you.
00:17:19 I’m a fan of Retro Sci-fi, so if anybody is not sure, just go ahead and look it up. It’s like this cool looking Regan thing.
00:17:28 I like it.
00:17:29 Yeah. Funny enough, my friend, I think designed it.
00:17:34 Oh, really?
00:17:35 Yeah, I’d be quite happy to hear. But everyone here loves the Reagan logo and the sort of Sci-Fi retrofuel that it has.
00:17:43 Yeah. So Reagan is go ahead and mention it. What does it do? It’s the product.
00:17:48 Yeah. So we build a suite of products evolved around developers and for improving the software quality that they produce.
00:17:57 We have a crash reporting product for monitoring your errors and exceptions that come through. There’s a real user monitoring product for printing performance and measuring what pages your users come through and how long they take to load and how long they view it. And then we also have APM, which is your service side performance monitoring, checking to see how long actual queries took, and trace only all websites. They’re all focused around software solutions for other developers so that when you integrate our products, you can then start to improve the quality of software that you produce.
00:18:31 And so what your end users experience now are some of the metrics that you’re collecting with monitoring tools? Are those things that we would consider like value added or business value metrics?
00:18:44 Yeah, definitely. Like a lot of crash reporting tools, it’s an easy one to one relationship with regards to business outcome because every time your user or your customer experiences an error, they’re having a degraded experience. And so there is that reduced value to them. And so the business outcome will inherently be reduced if they have to then repeat the action or you then have to get in touch. The business isn’t really saying that grade of an experience from it.
00:19:14 Okay.
00:19:14 Likewise, the longer a website takes to load for any reason, the less likely your user will wait for it and will be there.
00:19:24 How long have you been with Reagan?
00:19:25 I’ve been Reagan for four and a half years now.
00:19:28 Okay. And did you background wise, are you a software person or did you get into team lead from some other route?
00:19:37 I’ve been a developer before Reagan, but I became a team lead at Regan, so it’s been quite a nice journey, being able to work on products and then being able to help lead the team here to help improve the experience of the software suite of tools that come to build.
00:19:56 Okay. Are you enjoying it?
00:19:58 Yeah, it’s great. It’s always a great time. Everyone has engaged super focused on the customer and wanting to deliver the best experience that they can have in the team lead role.
00:20:10 Do you find yourself you care about this idea of developers caring about aligning their activities with business outcomes? This sort of a topic that is important to you because it’s been important to you before you were a lead or as you became a lead, did you see the need to stress this alignment with the people that you work with more?
00:20:32 I think it’s actually a part of I don’t want to say like culture here, but it’s something that every developer lesson in each of the teams that I’ve been a part of has been cautious of as considered. So what is the actual business value that we’re getting from this bit of work that we’re doing, and could we get more value out of another piece that we’re developing every day, or at least almost every day? My team and I have the conversations with our product manager around other bits of work that we could work on or hang on. We’ve had a roadblock on this bit of work. Does it still make sense to continue doing it, or do we have a better impact looking at some other feature or working on some other bit of work or adding tests around something else?
00:21:20 Okay.
00:21:21 It’s really driving it from what bit of work are we doing here? Does it provide the most value or could we provide better value working on something else? It’s something that we do on a day to day basis with all of our work that we deserve, and we want to get that the most value. And it can lead to some discussions which sometimes are a bit tough. When you’re having a look at you have one opinion and people have another, but that’s when you can start to actually come back with everyone trying to do the best job that they can. Everyone’s working on, working on what is best for the business, and then you can holding each other account for what you’re working on.
00:21:58 Okay. And that’s great. And I think I like the idea of talking about that in terms of a company culture, because it’s definitely going to be important because if you don’t have a culture that allows developers to question requests like the roadblock part, for example, I’ll bring that up. So let’s say somebody decides who knows where it was decided we need to change this thing. And then people go, yeah, okay, that makes sense. They go off and start working on it, and then you get into the point where you’re realizing that in order to make this change to our software, there’s like dozens of files that need to be refactored to make this one change. Now we need to maybe step back and go, is that the only change we need to do, or is this restructuring indicative of that? Maybe we need to definitely need to do this, and it might cost us a lot right now, but future changes might be easier. So let’s bring it up and talk about it and say, hey, the cost might is it worth it? Is it going to weigh outweigh the value of this one change? But maybe it’s going to help us with changes in the future. And if that’s the case, do we really want to do it right now or do we want to schedule it for like two months from now or something like that? And I think that’s cool to have that culture, to be able to question that and bring that up, because we definitely don’t want a culture where engineers will just try to beat themselves up and work extra hours when the management change just thought it was an easy change and they didn’t understand the cost involved.
00:23:32 Right, exactly. And you want to have those conversations, you want to say, hang on, this is going to require if it’s going to require a lot more work than what they originally intended, then they probably also would agree on the management side of things of hang on, this probably doesn’t make sense because we’re not going to get that return on investment or we could get a better output elsewhere.
00:23:54 And as a developer, you don’t really want to work on something for hours on end to then know it might be discarded or that it wasn’t as valuable as it might have been because that’s just kind of a Socratic thing to work on something for it to then not have the sort of impact that you were hoping for that you’re wanting.
00:24:13 Yeah. I think that’s also important to try to get one of the reasons for developers to care about what the business is doing. Anyway. I remember early in my career having these, like, I don’t know, all hands meetings where they would talk about different sales, markets and success for this product and that product and stuff. And it wasn’t stuff I was working on. So I was like, why should I care about this? But that’s part of the reason why. And now that further into my career, I can understand that. Trying to understand where the business that you’re working, the company you’re working for, where it gets its money, where the value is, what the culture is, and not trying to pay attention to that. And even on projects that you’re not working on, trying to pay attention to those enough to just kind of soak it in to just sort of know it’s there. It allows you to start making those day to day individual small decisions that align with where the company is headed.
00:25:12 Like, for example, that roadblock thing. Let’s say there was like 100 hours needed for small change because of a refactoring. But if I’m paying attention, I would know that there’s like maybe another group that’s leading a team to replace this product in the first place. So a major refactor doesn’t make sense if we’re going to scrap the product in six months anyway, things like that. So there are times where Frankenstein and Patching are the right decision when you’re planning on sunsetting a product anyway.
00:25:44 Yeah. And it reminds me of the fact that all code that we end up writing does have an expiry date behind it at some point.
00:25:51 Yeah. And also around the test, I find it surprising that some people don’t think about the cost of tests as well and their software as well, because, I mean, I definitely am super proud if I can refactor an algorithm that’s like two pages of code and make it into like, maybe split it up into a couple of different functions and make them small so that I’ve deleted, removed a whole bunch, replaced 200 lines of code with like 30. It’s really cool. And it also has business value because it’s less code to maintain in the future. However, on the test side, people just always are bragging about how many hundreds of tests they have, and all of them carry maintenance cost just like the rest of our software. So putting time into making sure the tests are maintainable in the future is important too.
00:26:44 Maintainable and readable goes a long way, especially something that I always more considerate of now. Being more of a team lead and watching junior developers come through is the amount of documentation and having assessed there as a safety net for them. There are some times when they’re working on an area and if you’re either not paying attention or not looking out, then they can easily rabbit hole down a different path and spend time wearing dual documentation around it or test to show this is what is the intended behavior can really go a long way to ensure that they are set up for success.
00:27:24 Yeah, definitely.
00:27:27 Having the tests around things are good.
00:27:32 The reverse part, so it’s easy to get across to people that the test will help us verify that the software testing is working. The hard part is to say, okay, we also want it to if the test fails, to be able to easily find out what’s wrong. And it’s not necessarily that I’m an advocate for these pinpointed micro tests that test little tiny pieces of the system, but I have seen tests that are like asserting way too much stuff. And so there’s like, lots of things that could go wrong and have caused this test to fail. Those are costs as well. When we have like a little tiny failure and it fails like 50 tests, it’s not as helpful if it failed one that would be more helpful because we could pinpoint what’s wrong.
00:28:17 Yeah. Rather than having to go through the 50 and try and decide to cipher which one is it?
00:28:23 Yeah, definitely. So I’m also curious about this team lead role within I’m officially a team lead at Ron Schwartz, but team lead means different things to different companies. So are you still writing software or are you more of a manager role at this point?
00:28:41 So I’m still writing software. It does vary, sometimes depending on the day.
00:28:46 I definitely think the team sometimes refers to me as half of a developer.
00:28:51 There are some days where I can add a solid amount of time. I can go away and code and help contribute to the sprint. And then there are other days where I’m just setting up other meetings or I’m talking to people or I’m having catch ups. So I’m lucky to still be able to write code and work and contribute towards the sprint. But definitely for that progression to being a team leader seen a reduced amount. And I think one of the most valuable things I can do is actually supporting the rest of the team and enabling them to produce more. Removing roadblocks, helping them learn and understand doing more of the mentoring so that they can actually keep pushing on.
00:29:30 I couldn’t put it better. That’s great. To be able to enable the rest of the team to work better while still keeping enough technical knowledge so that you don’t lose touch with the rest of the team.
00:29:41 Yeah. I think one of the best metaphors I’ve heard is that you want to act as a multiplier rather than a single contributor. So you can act as a single contributor to writing code yourself. But there’s a multiplier, you can actually have more output or more impact by helping everyone else contribute more, removing roadblocks where when they hit roadblocks and helping them learn before they actually require that piece of information.
00:30:08 Okay, cool. Like, I’d probably enjoy working on your team. You seem like somebody that wouldn’t be so that the engineer that says, I don’t think what you just asked me adds business value. Would you take that?
00:30:20 Well, I probably wouldn’t take that too well. I’m not sure if anyone particularly would, but we definitely hope you’d phrase it in a slightly different scene and during before even picking up the ticket, you’d consider it.
00:30:36 Okay.
00:30:36 Hang on. What are you hoping to achieve here? What are we hoping to accomplish? Probably wouldn’t frame it in the way of you’re not adding value, but more what are we hoping to achieve?
00:30:47 Yeah, okay. There is like being good at communication.
00:30:51 Yeah, it’s one of the other things we learned always reminded every day is communication is key.
00:30:56 Yeah. Well, thanks so much for sharing these ideas with us today, Benjamin. I wish you luck in everything you do and we’ll talk to you sometime later.
00:31:06 Thank you, Brian. It’s been great talking to you as well.
00:31:12 Thank you, Benjamin, for that great discussion. Thank you PyCharm for sponsoring the show. Try PyCharm yourself at testandcode.com. Pycharm thank you Monday.com for sponsoring join their contest at monday.com testendcode and thank you to all the listeners that support the show through Patreon join them by going to test and Code.com support all those links are in the show notes at testing Code.com. One, three, four.
00:31:37 That’s all for now.
00:31:38 Now go out and test something.