Who should do QA? How does that change with different projects and teams? What does “doing QA” mean, anyway? Answering these questions are the goals of this episode.
Transcript for episode 175 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:01 Who should do QA? How does that change with different projects and teams? And what is doing QA mean, really anyway? Answering these questions are the goals of this episode.
00:00:12 Thank you, Pie Charm for sponsoring this episode. Each arm saves me time. Error highlighting lets me know as soon as I type a mistake. Code completion and parameter hint pop ups save me from having to look stuff up. The Get integration makes all of my Git workflows super easy, and then you get a whole bunch of bonus features like the database front end that is super cool and a super fast debugger. I wonder how PyCharm will save you time? Find out yourself at testandcode.com. Pycharm this episode is sponsored by Rollbar. Rollbar is a leading platform that enables developers to proactively, discover and resolve issues in their code, allowing them to work on continuous code improvement throughout the software development lifecycle. Robar has plans for all situations, from free to large enterprise. With Rollbar, developers deploy better software faster and can quickly recover from critical errors as they happen. Learn more at rollbar.com and if you use the link in the show notes, there’s a raffle you can enter. That’s. Cool. Thanks Rollbar.
00:01:28 Welcome to Test and Code.
00:01:39 So our questions are who should do QA?
00:01:42 How does that change with different projects and teams? And what does doing QA mean anyway? Let’s just go ahead and do short answers to all these questions and then dip into more details of each and probably jump around a little. Who should do QA? There are two reasonable answers. Only one, almost everyone involved in a software project, and two, it depends. But usually almost everyone involved in a software project.
00:02:08 How does that change with different teams and different projects and different teams? Well, if you’re the only person on the project, well, you have to do everything you have to do planning, development, QA, marketing, triaging issues, really everything. If it’s a huge project with lots of people and sub teams, then still everyone is still responsible for quality. But it also might make sense to have some people specialize in making sure the whole system works like it should. And if lives are at stake and or the software can’t be changed after it’s deployed, like, for instance, space systems or car navigation, et cetera, it’s still everyone’s job. But then you also definitely want independent verification. For the most part, I’m not talking about these kinds of systems in this episode. Last question. What does doing QA mean, really anyway?
00:02:58 Well, it’s a trick question. Intentionally vague. Qa is quality assurance, but that doesn’t really help much. It’s ensuring we’re building a quality system that needs to have everyone involved. There are specialized activities that might be divided into different specialists.
00:03:16 Let’s talk about things that have to happen in lots of kinds of software. Well, there’s unit tests, of course, at least for complex algorithmic functions. There’s going to be unit tests written with the code at the same time or near the same time, and it makes sense for developers to do that. But Unit tests are not focused on what the system needs to do, so we do need system level tests. Does that mean GUI tests or API tests or functional tests? That’s really a per project and a per tech stack decision, and hopefully tests are all automated.
00:03:51 Should there be manual test procedures written and followed? I’d like to have everything automated, but there are times where a few manual procedures are way faster to utilize than automation. When you have that, though, you just can’t rely on CI to test your software and make sure it’s in a shippable state because you’ve got those manual tests around. Migrating those to automation as soon as possible is desirable. But still, let’s be pragmatist about it. Stuff happens. There’s also Exploratory testing, either by specialists or by the whole team. Exploratory testing is super powerful, and that’s why dog Fooding is so common. If you use your own software, you’ll see obvious junk that breaks. So that’s a good thing. Who should do all of this? All of this writing of system tests, writing manual test procedures, running manual test procedures, exploratory testing. But then there’s also defect triage and checking documentation against implementation and making double sure new features and release notes are really there and are rock solid. How about making sure your unique selling points stay rock solid and there’s security testing and performance and load testing. There’s so much testing, so it might be nice to have somebody specializing in this. The common division that people think about sometimes is whether you have a specialized QA or to have developers do the QA.
00:05:15 As a developer, I want it all. I want access to all of the tests so that if there’s a failure, I can run it and reproduce it and verify that I fixed it later after I have fixed it, hopefully, and I want to be able to read the test easily, and hopefully it’s in the same framework that I’m using for my Unit test, or at least similar.
00:05:38 Yes, I am a demanding engineer, but that’s what I want, but maybe I really don’t want to do the writing of all of these tests. It would be really great if I could have somebody else do some of that stuff. I think it’d be awesome to write my happy Path tracer bullet test, though, to get the first implementation done ASAP so that people depending on my code can run with it and get their stuff done. But what about the corner cases and the performance testing and all that stuff?
00:06:10 I’d like somebody else to do that if I can. But what about the corner cases that I do care about? I’d rather run those from a System level and not a Unit level, because I don’t want to overdo the system.
00:06:23 But what if I’m working on a subsystem and the System API isn’t ready so I can’t do system level tests yet, or if it is ready and I write this system level test from the API. But I’m not writing the API interface. Aren’t I testing the whole thing?
00:06:44 And shouldn’t the API person be doing the testing or do they expect me to do the testing? What does my boss think?
00:06:53 Someone has to write the system test and code at all the edge cases, but if I’m doing a subsystem, is it me or somebody else?
00:07:01 Anyway, if you have a large project with multiple teams where there’s questions like these of really who’s responsible, it does totally makes sense to have some specialists doing the integration tests of the system. If you can do that, that’s great. Then there’s not disagreement to who’s doing the system level tests. It also makes sense that the tests they write are runnable by everyone on the team, though I don’t really want to have to ask the test team to run the test again. I’d like the development team to be able to run it, but I’d also like the system level test people to run the unit test if they want. That’d be cool. Why not? Let’s say we have this system test engineer person. This system test person is going to be a software engineer because they’re writing automated tests since their code needs to be elegant and readable by other developers and managers and maybe Doc writers, et cetera. There’s lots of skills that would be handy. These test engineers hopefully are experts of the product, the domain that the product is going into, and the customer expectations. They should be aware of the project schedule, the risk tolerance we have of different areas of the software, the team skill levels, et cetera. They should be talented, communicators across teams, skilled and pragmatic test case designers, good at both writing and speaking. Excellent software engineers, of course. So basically they need to be a superset of awesome software engineering engineering skills. If you have test engineer specialists on your project, I hope you respect them for the role that they provide.
00:08:35 But wait, do we need this? I thought that developers were doing all of their tests with TDD and stuff, with their Iterative thing and their agile, and if they’re bragging about 100% coverage. So what’s left to test? Haven’t they tested at all? Okay, if you’re asking this, get out. Go sit in the corner and think about what you just said. Seriously though, developer tests and TDD tests are about small bits of code, and they’re about expectations of what that piece of code is doing. They’re usually not expectations of what the system is doing. We still need system tests. However, if there’s no one doing that specialized test engineering role, then it totally rests on the rest of the team. The devs are probably going to do the writing of the tests, obviously, but it can’t just be the unit test. We need system test too. But the management team, the project leads, document writers, everyone on board with making sure of everything like manuals matching, application and everything. We need to be making sure that that works. Doing the dog Fooding, doing exploratory testing, evaluating issues, thinking about risk and schedule, coming up with test cases, and making sure that those are all implemented and tested. It shouldn’t lie just on the developers. If you don’t have specialized test engineers but you have a larger team, it should be spread out. It’s everybody’s responsibility. But you know what? If you have specialized engineers, having everyone else conscious of quality and involved is the right way to do it anyway, right? So I’ll go back to that. Who should do QA? The answer is it depends, but usually almost everyone involved in the software project. I want to talk about one thing that I hope isn’t an issue anymore, but I think it is somewhere, and that’s the embedded versus independent QA an independent test team totally makes sense for things like Rockets and pacemakers, et cetera. But in those projects, the developers still are testing their code. The independent teams are there to independently test, not to do the only testing. Testing engineer specialists for most projects should be embedded in the team. My opinion, they should not be independent QA teams, they should be embedded. Anything else seems nuts to me. Thought Works had an article on who should be involved with test automation, and they had a list called The Four Principles of Test Automation. And these are great. One, only work on test automation if you have the skills to work on the production code if required. Two, only work on the production code if you have the skills to work on the test automation if required. Three in case of one and two not being true, pair with somebody who meets the requirement of the other one until it is. And number four, the team must own and contribute to test code, not certain individuals or disciplines within it. This makes sense to me, and I want to expand on number four. Really, the last one about ownership and contribution is important.
00:11:38 I take this to mean that software developers are writing tests, even if the test engineers are writing the bulk of the system level tests, and that the end result suite. Test suite is owned by the entire team. If something is missed, the team missed. It, not the test engineer. If a test is broken, it’s the team’s responsibility to fix it, just like the software. We don’t finger point. We fix problems, right? I know there are tons of kinds of software and tons of kinds of team structure, and I’ve only been exposed to a handful. If you have some perspectives on this topic that differ from mine, or even if they’re the same, please let me know. Maybe we could do another episode on some other ways to split the problem.
00:12:27 Thank you, PyCharm for sponsoring visit test and code.com PyCharm for a four month free trial of PyCharm pro save time, use PyCharm thank you Robar for sponsoring Robar enables developers to proactively, discover and resolve issues in their code so they can work on continuous code improvement throughout the software lifecycle. Learn more at rollbar.com thank you Patreon supporters join them at testandcode.com support all of those links are in the show notes@testandcode.com. That’s all for now. I’ll go ahead and test something.