Alan Page explains how the Modern Testing Principles reflect the how testing actually happens in a healthy way in modern software development teams.
Transcript for episode 75 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
This is Tessa Code Episode 75. If you do software testing right, it’s really done all of the time throughout the whole life of a software project. This is different than the verification and validation of the classic model of QA teams. It’s more of a collaborative model that actually tries to help get great software out the door faster. And iterate quicker. One of the people at the forefront of this kind of push of software testing is Allen Page.
Allen and his podcast cohost, Brent Jensen tried to boil down what modern testing looks like in the modern testing principles. I’ve got Alan here today to talk about the principles and also to talk about this transition from classical QA to testing specialists being embedded in software teams and then on to software teams doing their own testing. But that only barely scratches the surface of what we cover. I think you’ll learn a lot from this discussion.
Welcome to Testing Code, the podcast about software development, software testing, and Python.
Welcome to Testing Code. I’ve got one of the kind of the heroes of testing Ellen Page on the show, also the Angry Weasel. I’ll let him introduce himself. So, Allen, welcome to the show.
Thank you for having me. I consider myself more of an antihero, but I’ll take it perhaps it’s just because I’ve been around so long in testing. But yeah, I’m Alan.
I have a podcast at Brent Jensen called Moderate, called Shoved a Gun there called the AB Testing Podcast, which is a little bit of a play on words. A B testing, of course, is a method for deploying ideas or experiments to production. But in this case, the A and B stands for Allen and Brent.
And we just finished our 100th episode about a week ago, so that was pretty exciting.
And currently working at Unity Technologies, I have a blog at angrywefle.com. Some selected writings are listed there. You can track them down, and that’s probably anything else you could want to find out about me. You could probably find from that website.
But the real quick story about where Angry Weasel came from. I’m not an angry person.
I was in a band once, and we’re trying to come up with a band name. This is an important decision that a band always needs to make. And we were talking to some people that had the story thick German accent, some story that I barely remember now about beware of the tooth of the Weasel. And from that, we came up with Angry Weavel. That could be our band name. So I bought Angryweaasel.com, and our guitar player said, no, I don’t like that name, so we never got to use it. I kept the domain and eventually thought, you know what? I’m going to use that for my blog. Yeah, that’s where Angry Weasel came from in a short story. And that’s why I’m Alan the Angry Weasel.
What was the band name that you came up with.
That’S a sad answer.
We were mostly Microsoft employees at the time.
And the name for a while we were severity one.
Okay.
That’s pretty bad. And eventually we became Groove Mob.
Groove Mob.
I like angry weasel better.
Yeah, definitely.
You were at Microsoft before Unity, right?
Yes. For 22 years.
Okay.
A short stint.
Yeah. Well, while you were there, I assumed that you wrote the how we test at Microsoft.
You are correct.
Maybe right about the halfway point, I wrote a book called How We Test Software at Microsoft, which is now a nice snapshot in time.
Okay.
How we Tested Software at Microsoft at that point in time.
Okay. So that wasn’t the case when you left Microsoft then?
No, absolutely not.
Okay.
When I left Microsoft, there were no or very few people in dedicated testing positions.
Okay.
They had removed most of them at that point.
Well, that’s actually kind of a theme that I’ve seen in a lot of companies, for better or worse. Some people for good reasons and some people for bad reasons.
Yeah.
My theory is done right.
Not a dedicated testing specialist. I’ll say it backwards.
A dedicated testing specialist used incorrectly as a bottleneck and slows down the team.
But done right. You can be the way I’ve done in teams that I’ve been. Teams can be more efficient without creating that bottleneck of the dedicated testing specialist.
Yeah, definitely agree.
It depends a whole bunch of context.
One of the main reasons why I wanted to ask you on the show is because I think it was both you and Brent did came up with this modern testing principles. Is that right?
Yes.
Is it okay if we just read through these like seven principles?
Yeah. And maybe first let’s do that. But let me give you a little background why it’s called modern testing first, because you’ll notice before we go through the principles aren’t that much about testing, and they’re not really that modern.
The way the term came about was we were talking about traditional testing, air quotes. Traditional where the waterfall? Test last, do all your testing at the end and try and test quality in pretty much the way I did testing for most of the first ten years of my career.
And modern testing is more of the antidote to that.
How do we test now? And Brent and I are both working pretty heavily on services. I applied a lot of these working on Xbox as well.
But the idea is that we want to be testing all the time. It doesn’t matter whether we’re delivering all the time. We want to test all the time. We want to work on optimizing the team from our rules.
In my role as a test specialist, I use that rule to help the team learn how to test better, how to get better at making quality software and sort of leading them towards a more accelerated view. And then in talking about it, in that way for a while. We use the term modern testing for a while. That way until maybe a year and a half ago, maybe a little longer.
We decided to try and come up with principles to describe these things we’ve been talking about for a while. Modern testing. And that’s where the modern testing principles came from.
Okay, so we don’t need to discuss all of them. I’ve got questions about it, but just so people and I’ll actually list these in the show notes as well, and give a link to your site that lists them.
Yeah, you can just go to moderntesting.org and that will redirect to the principles.
Okay, so let’s see. There’s looks like seven principles.
One, our priority is improving the business.
Two, we accelerate the team the use model. Wait, you probably read these better than I will. But we accelerate the team the use models like Lean Thinking and Theory of Constraints to help identify prioritize and mitigate bottlenecks from the system.
Three, we are a force for continuous improvement, helping the team adapt and optimize in order to succeed rather than providing a safety net to catch failures.
Four, we care deeply about the quality culture of our team, and we coach, lead and nurture the team towards a more mature quality culture. Five, we believe that the customer is the only one capable to judge and evaluate the quality of our product. Six, we use data extensively to deeply understand customer usage and then close the gap between the gaps between product hypothesis and business impact. And lastly, seven, we expand testing abilities and know how across the team, understanding that this may reduce or eliminate the need for dedicated testing specialist.
That’s actually quite a bit of information there.
But they overlap and support each other in a lot of ways, too, once you get into them.
Right.
You came up with these or solidified them about a year ago, you say maybe about a year and a half.
I first presented these at a conference about a year and a half ago. Yeah.
Okay. And in your brief description of, like saying Microsoft has less dedicated testers, this push to have testing be basically learning about the customer, learning about the product, learning about the software, and then also trying to speed things up and make things smoother, not the other way around. That is a different role than traditional test stuff.
So you work at a company, right? You’re not a consultant with other companies?
I work at a company that makes software.
Yeah.
So I don’t know. I guess in your speak, you do go around and speak, though. Does this resonate with people?
Surprisingly so. The first time I talked about modern testing, about a year and a half ago at Brighton, at test bash, I was a little worried. Group of testers, what are they going to say? And not only did I not scare everyone, but I had people come up to me afterwards and say thank you for giving a name to what I’ve already been doing.
And like, yeah, this is where we’re going now. This gives me a framework I can work towards. So I was a little surprised how much of it was already being done. In many ways more than the pushback of all this won’t work. And there is controversy around at least two of these have started at least a few Twitter Wars, but overall the feedback has been pretty positive.
Okay, well, that’s the positive ones. That’s boring.
Which are the ones that are controversial.
Two of them. But let’s talk about the most obviously controversial first, which is number seven, which says we’re going to make the team better at testing. In fact, we may make them so good at testing that they may not need a tester anymore.
Based on the conversation we had earlier, that freaks people out. Like, what do you mean you’re just going to get rid of testing because you’re going to blindly follow these principles and that’s not at all what we mean.
But we do mean it the way I’ve said it, if you actually the team may get so good at testing because we believe and Lisa Crispin has really been supportive. Lisa Crispin, who with Janet Gregory wrote the Agile testing books, very supportive of modern testing and gets exactly how it builds on agile testing and loves the idea of test as a coach or test as a consultant and helping the team get better at testing because that helps accelerate the team. Number two, we spend a lot of time in modern testing. We want testers to help the team get better at testing. That’s doing pair testing. And I had a test team at Unity for a while and it took me two years, but I dissolved that team because we reached number seven and they were no longer needed. I didn’t lay anybody off. They merged into other roles on the team, most of them still in largely test specialist roles, but very well embedded in the team and leading the team towards the quality culture. So it’s controversial on the surface it looks like we want to get rid of testers, but what we found in practice is the way this principle works is you don’t have the goal of eliminating the dedicated testing specialist. What happens is eventually you get to a point where you go, oh, we actually don’t need a dedicated testing specialist.
So as long as you look at it that way, that one’s pretty safe. But if you look at it, it scares people, right? We have job preservation. They look at those last seven words and they freak out. But in reality it’s a logical and the step there, if you choose to make that step, will seem normal and there will be plenty of orgs that don’t eliminate a dedicated testing specialist. And that’s fine too. We’re just saying that it is a strong possibility in a lot of cases.
So people don’t lose their jobs then?
No, that’s what I’ve seen.
I see this as more eliminating the wall between QA and development.
Yes.
But there’s also some camaraderie. So if you’ve got a group of QA people or testers and now you’ve got them all in the teams instead, do they feel like solo people just in the fighting on their own? You know what I mean?
Yeah, I think what I’ve done, there’s multiple ways to solve that here. I keep the community going so those testers still hang out. I still send them a newsletter, and we have channels and slack where we hang out. So there is that camaraderie. Just everybody is a specialist in their team. I look at testing specialty the same way I look at performance specialty. You may be really good at doing performance work or doing front end work.
That’s great.
But we have people who are really good at quality work and testing work, and they’re not going to do all of it. They’re probably going to take on the harder stuff. They’re going to pair. They’re going to help people elsewhere on the team get better at those things.
But I think community is really important. So finding a way to keep those connections going is good.
But you may be the only tester on a very small company and then your community comes elsewhere.
Yeah.
This episode is brought to you by Patreon supporters. I remember early in this podcast I reached out to supporters to ask them what I could do for them to show my appreciation, something extra I could do just for them. Most of the responses I got back were essentially, we don’t need anything extra. We’re doing this to keep more shows coming, take whatever effort you are going to put into bonuses and make more episodes. Thank you to everyone. If you want to join these amazing people, go to testandcode.com and look for the donate menu item. Again, thank you, everyone. You rock.
Okay, well, what’s the other controversial bit now?
I was surprised, but we didn’t bend. I didn’t think any other one would be controversial. But number five came out as controversial. And I think about it, it goes back to the tester identity a little bit. And to be clear, number five, since probably most of listeners did not memorize them when you said them is we believe that the customer is the only one capable to judge and evaluate the quality of our product.
And for a long time, I’ve been in test for 25 plus years, for a long time as testers, we thought we are the customer. We’re the customer advocate. We are the voice of the customer. And we wanted to position ourselves as this last line of defense for the customer. And we were their savior.
Perhaps a little bit overspoken, but a lot of what we talk about in modern testing is borrowed and stolen liberally from books like Reinerton’s Flow and for this one, pretty much lifted right out of the Lean Startup by Eric Ries.
Yeah.
And there’s a quote in there that says you don’t get any validation from your engineering effort until your products in the hands of customers.
And we really believe that. We believe that we can do a lot to mitigate risk, for sure. For sure. We can do a lot to take care of other quality issues and to think about user workflows. But we really don’t know everything up to that point is a guess. We don’t know until the customers actually use the product and we get feedback and validation from them. And largely we get that from data, which is principle number six. We use data a lot, but we have to get feedback from the customer to know if we’re building the right thing for that validation step. But we actually got a lot of pushback from people who didn’t believe that.
A lot of the pushback was things. Some of it was actually quite rude, but some of the worries were this notion that we’re using customers to do all of our testing for us, which is not what testing in production means, which people have made that jump and think, well, this means you want customers to do all your testing. And no, it doesn’t mean that it means we want them to validate that we’re building the right product and we’re going to collect data and we’re going to do things in production to help figure that out, because that’s where customers don’t play in our staging environments. Customers are in production. So we need to do a lot of testing and validation in production to do that. So I think people get caught up in the fact that, one, that they want to be the customer even though they’re not. But two, they feel like this one says we’re going to do all of our testing in production and customers will do all of our testing for us. That’s why we don’t need testers, and they get a little lost in that. So that one’s come up. Those two have been pretty much the only ones people have complained about, and they’ve complained and pushed back pretty loudly on those two.
Well, that one reminded me of the extreme programming thought of having a customer embedded in the team, which also seems ludicrous in most circumstances.
Right. And that’s great if I forget which of the I remember reading about that which book. But yeah, if you’re building a software at Chrysler or wherever it was and you’re a customer of the accountant and they work upstairs. Yeah, bring them down.
But that doesn’t quite work for a lot of other software projects. So again, Lean Startup is all about this. Do you use data that we can get from those customers to help answer those questions? The data becomes our customer.
But this gathering the data also then changes this has got a feedback into project requirements to collect the data in the first place.
Yeah. And the difference is what I tell my team is instead of, well, in addition to thinking about, we’re looking at new features we’re working on, in addition to thinking about what testing do we need to do? We ask questions like, how will we know this is working in production?
How will we know if this is not working, what behaviors or monetary or changes we expect to see, and then making sure we have collection in place to help answer those questions.
Yeah. There’s definitely still some software and connected Internet connected software has a little bit easier time for that.
But there’s still a lot of software realms where that’s actually really hard. Still, it is.
But we did a reasonable chunk of this, even on Xbox, where it’s not one OS, it’s three operating systems that live on this box.
And of course, we weren’t downloading 50 changes a day, but our beta users would take updates every day or every other day, and we would do experiment groups and we would figure out use that to try and validate some of the things we were doing.
Okay, nice.
So you can do it. And again, people get installation fatigue. Even on your phone. You have the ability to update as often as you want, as often as you can get Apple or to approve your submission. But people don’t want to download a new version of Angry Birds every day. But you do get a new one every week, and you’re kind of okay with that, even though some of those some of that bug fix, some of that features, some of that’s experimentation.
Yeah. But I mean, there’s still even like, cases like embedded systems and other devices where there’s no data coming back.
And in that case, you have to resort to either some sometimes there’s some data you can collect and then can be uploaded via USB key manually. But generally it’s customer interviews or you have to be your only proxy then. Yeah, but you’re also shipping in those cases, generally a much smaller audience.
Yes, definitely. And it’s easier to find a representative person to ask.
Hopefully. I’m glad that it’s going well or it’s being most people are understanding that that’s really how testing is happening now anyway, because a lot of the tutorials and how to do testing is still on that old.
The goal of a tester is to try to break things.
And I hear from your chuckle that you don’t believe that that’s really the primary goal then.
I never thought it was.
I think it’s ridiculous to think that. I’m sure you’ve seen it as well. People saying that that’s the primary goal.
Yeah. But I see fewer and fewer of those, and they’re working on projects that have less and less impact to people.
Okay.
I think my goal is whatever role I’m in, my goal is to ship high quality software that customers love.
Yeah.
And I’m going to do whatever works required and help the team deliver that.
Well, one of the things I definitely like the principles, the aspect that you’re coming at it from the tester side, representing testing group or testing people. Whereas my experience mostly has been on the developer side saying, hey, all of us developers, we should test more.
How do we do that?
And trying to find information to try to teach people that don’t have there’s a lot of groups out there that don’t actually have testing people around to help teach them. One of the questions I have for you is this push towards more testing by developers. Does that mean pushing from high level tests to unit tests?
No. And I think there’s a fallacy in there that you and I both know. In fact, probably most people listening to this podcast know that I can write a whole bunch of unit tests that give me 100% code coverage and I can still ship with bugs because things working.
I think developers are very capable of making sure their chunks work in isolation by themselves. They don’t need really any coaching for that. Tdd will help with that. Or just a book on unit test and code. Thinking through on how to reaching levels of coverage is a pretty easy problem to solve.
Where it gets hard is when these pieces start to fit together is making sure my component works with your component in this system of components, and where I think most developers need some coaching and where the gap is and where I see people practicing modern testing principles or testers practicing those principles help is teaching the team to recognize risk and how to think about how those pieces fit together and what sort of tests they need to write. I don’t think a lot of developers I’ve worked with will write all the tests they can think of and there’s still bugs because they haven’t been taught how to think about what else can go wrong, how to bring a risk based approach to their software development and release. And that’s where we try and help them. And that’s where a lot of teams really screw up in getting rid of their testers if they think, okay, I have developers, they know how to write unit tests. I don’t need testers anymore. And there’s a huge logical gap in there because there’s a whole lot of surface area between good unit testing and a great quality software for my customers.
Yeah, definitely.
And there’s a world to live there. And it may get to a point where again, going back to principle seven, it takes a while. It is an automatic getting rid of the test. Remember Yahoo?
In fact, the reference to a safety net came from this article I read about Yahoo maybe eight or nine years ago when they said Yahoo fires all testers and removes safety net for developers and part of the problem once you saw them as a safety net.
But if that doesn’t solve the problem, it doesn’t make you automatically figure out all those quality pieces. There’s a gradual change that needs some coaching. One of the things I’ve done for my last several teams is help coach them away from unit testing enough to thinking about quality a lot. And I work with a lot of developers now who are great.
They’re really good testers. They think about the system and how it works together and someone goes to change something, go, wait a minute, you can’t do that because it’ll break this thing over here. And sure enough, they’re right. Some of those things that we’ve done as testers for a long time.
So anyway, it’s all coachable. It’s my short answer.
And people can do it. But you can’t jump from developers running unit tests to not needing testers.
Yeah, right. But the developers running unit tests or knowing how to write unit tests to a developer that can think about a higher level test, maybe even a system test or a subcutaneous or where to mock things and all that stuff is not obvious. Like you said, one of the things I like to teach people, just those questions that you talked about, what can go wrong? How do I know that it works and how do I know it’s not working?
Yeah.
And we do this like when we’re just testing things with I can write a little sample program to test my library that I’m working on.
But for some reason, a lot of people don’t believe that that’s testing. And at least the developers, if the developers trying to move by, say, hey, can you come up with some test cases for this? They’ll come up with all these weird corner case stuff that’s never going to happen in production. And I’m like, testing isn’t like that anymore.
Yes, exactly.
Common theme.
Yeah. Okay.
It’s not even their fault when I tell somebody that they need to learn more about testing and you Google testing, you get all of this rubbish.
In my view, because this is something I work on a lot. It’s not that they need to know more about testing. They need to know more about thinking about risk.
Okay.
In this example, you’re working on your library. You think which of our user flows would use this library?
How do I make sure I haven’t affected them? How do I get a reasonable level of confidence? I haven’t messed anything up in those user workflows. And the answer to that may be, well, I’m going to run some end to end tests. Okay. I did those manually.
I could really automate this and they can automate it then.
But you have to think about which gets into my automation approach. But you have to think about risk, and that leads to testing versus thinking. I have to do a bunch of testing.
Right. Okay. I like that. I also like that you brought up the notion of workflow because you can do thinking about a workflow from a user’s perspective and how that breaks down into my library and even use it like a unit test, just making sure that those workflows work, even not at a system level, but even at a component level or something.
Yeah. Now pair testing. What is that? Is that just sitting with a developer and helping to write the tests?
Yeah. So when I was running the test team for services here at Unity, I refused to let them be like, hey, please test this form. Let me know if it’s okay to check in.
If someone ever did that, they would ask like, Great, what have you already tested? Nothing. Okay, let’s test it together.
Okay.
And then the next time that go on for a while, what you already test? I tested A and B. Great, let’s do C and D together.
And now it’s just for as needed, a complex thing. I need some help on this.
I’m worried. But most of the time developers go, hey, I tested A-B-C and D. That’s cool. Check it in.
Okay.
Because we know the testing has been done and there’s some trust there. We know they know how to do good testing. So I gave you the really long answer of it’s as needed. But we built up a very high level of what that is over time.
I like it. As a developer, I’m always thinking about minimizing my manual tests and trying to push everything to automated tests if I can.
Is that different from your standpoint? Is there still a place for manual tests?
Yes. Sometimes you need eyeballs, and sometimes just because you can automate it doesn’t mean you should.
Okay.
And where it really comes to light in end to end, or often UI automation, which can be, in most cases, in my experience, not worth the effort.
I think you want to be really careful at the UI level because, well, I’m in the UI level. You want to avoid tests that can be flaky. I want to be able to trust my tests.
So my approach is my canonical line is that you should automate 100% of the tests that should be automated, which is an obvious topology. But what it means is it takes a while. This is the challenge of test design is figure out where are the places where a computer is going to help me do better testing rather than where are the places that I can automate. There will be an interesting intellectual challenge, which is the test pyramid, which also is surprisingly controversial. Write a whole bunch of unit tests that run fast and 100% trust. Write a reasonable level of integration tests that can help you make sure stuff is working together. And then a lot of people just look at the pyramid, never read the text around them, and some people argue it’s a triangle, not a pyramid. But whatever the top has UI tests or end to end test, depending on which one you look at and Mike Cohn’s text around. That said, write as few of these as needed.
And what that means is you only want to write those end to end or those high level tests because they’re longer, they’re harder to write reliably. They can be flaky is write test at that level only for the bugs that can only be found at that level.
Okay.
If you can find issues at the integration level, write the test there instead, because it’s going to be faster to run, more reliable.
And that goes back to do. You need always need manual testing if you don’t have any UI.
Maybe not.
Right?
I can even give an example from Unity.
Right now I’m working on our Ads platform, which is almost entirely services. There are some little pieces that will show up in your game on a phone, but we still do. And the developers do this still, it’s technically manual. They will run the whole scenario from Ad, request, the show on device to tracking all the way through in a staging environment, and lots of automated tests there. But they will almost always put some eyeballs on that process, not the UI looking at, monitoring how the system behaves, understanding what’s going on.
You could call that manually, but they’re not just looking at the results, they’re looking at the whole system. Because summarizing the way the whole system looks down to a result is difficult. So I actually don’t know whether you call that a manual test or not, but I will for the case of the story.
Yeah. Well, I mean, I’ve got cases where definitely we have not completely eliminated manual tests because we have user interfaces and there’s lots of things where just having people run it, look at the screen and make sure nothing looks funky.
Now, our manual test procedures are more specific than that, but essentially that’s what it is, because if there’s a specific thing that I want somebody to look for, I can automate that. But if it’s just making sure that it just doesn’t look funky and that’s most of the places where we see like a graph will be like discontinuous when it shouldn’t be discontinuous or a button is like half off the screen or something like that, I wouldn’t write an automated test for that. We just need to look at it once in a while.
Right.
It reminds me a quick story I’ve told probably a few other times, but I worked on Windows 98, I tested our graphics APIs, and I remember having wait, in hindsight, way too long of a conversation on whether I could use Get Pixel, a function that gets the color of a Pixel on the screen with Setpixel, the function that sets the Pixel on the screen.
So if I call Set Pixel red if I call get Pixel that same location, I should get red back. But was that a fair test? Or they need to look into the graphics, memory buffer and blah, blah, blah. And the answer is, if either of those were ever broken, you would know automatically.
Yes. One of the things there’s a separation between the developer community and the testing community. And if modern testing succeeds, there shouldn’t be. Right?
Right.
Does that separation of communities bother you at all?
No. And the reason it doesn’t is because there’s a lot of the industry. If you look at the distribution of innovation curve, which Clayton Christian made popular, which shows basically a Bell curve on the end, you have these innovations. And I forget the names of the labels. Doesn’t matter. You have legards in the back. And early adopters, modern testing is still just reaching the early adopter stage. So as much as we get a little bit of the people that go to conferences that come up to me and say, oh, this is great, we’re already doing this. The people that go to conferences are already ahead of the curve. People that pay attention to the Internet, people listen to podcasts are already ahead of the curve of a lot of people in development testing because they want to be on the edge.
There’s a whole bunch of industry that’s not there. And those people need a community to learn from and commiserate discover things they don’t know. They don’t know.
Yeah.
So the world isn’t ready yet for I want them to get there, but the world overall isn’t ready to see testing and development as a single thing versus two separate things.
Okay.
So I’m not bothered because I think it’ll just take time. And if it gets to a point where the majority of the world is following something closer to modern testing, great. I’m super happy, and I think it’s going to be good for the users of software, but it’s going to take a while to get there.
Okay. I really enjoy your podcast, too.
Good.
So what made you decide to do a podcast?
That’s a good question. So Brent and I both used to work at Microsoft together. That’s probably figured out I also mentioned earlier a big fan of community for getting people connecting together. And there was a group of us in this role of test architect at Microsoft, which really just meant we were senior level testers. And they thought it would be good if we early on when there are only a few of us, it thought it would be good if we would get together once in a while and chat. And I was chairing this group and making sure that all the test architects at Microsoft had a venue to get together.
And Brent was a frequent attendee of these meetings, and we played off each other pretty well.
We like each other, but we give each other a really bad time and through that we could vet a lot of really cool ideas and then one of our brighter moments of slightly insulting comedy, someone in the room said, oh my God you guys have to record this and put it on a podcast and I said yeah and we heard it a few more times and eventually we started a podcast and as I just mentioned it got to 100 episodes. I would say I wouldn’t start at the beginning but the last 50 or so have been pretty good.
It gets better as people get more experienced, right?
Yeah.
Hey, I want to thank you for your time. If people want to know more about you, it’s angryweezil.com. Is that right?
That’s correct.
Alright. And yeah, just thanks for your time.
This is a lot of fun. Thank you.
Thanks again to Alan for talking with me and for all the great work he does. Thank you Patreon supporters for your continued support and that’s all for now. Now go out and test something or maybe take a few minutes and pair test with someone on your team and let me know how it goes.