TDD is easier with dev tools, editor, and installs working together well. This episode discusses editable installs with flit, a good TDD editor layout, test case grouping, batching tests to reduce context switch time, and even utilizing git tags.
Transcript for episode 81 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
In the last episode we talked about going from script to supported package. I worked on a project called Submarck and did the packaging with it with a tool called Flit. Today’s episode is a continuation of that episode.
I’m going to cover a couple of other different things, though. I made a couple of changes to Submarck in a Tddish fashion, and I want to talk about that a little bit. In order to get the project into a place where I can do TDD with it, I needed to make the package editable and testable at the same time. There’s a way for Flip to allow this, and we’ll talk about that. And we’ll also talk about a decision I made about grouping up some of our test cases and some modifications to traditional test driven development that helped me to develop faster when things are going pretty good. Thank you to PyCharm for sponsoring this episode. We’ll talk a little bit more about Pycharm a little later in the episode. This is Test and Code, episode 81.
Welcome to Test and Code, the podcast about software development, software testing, and Python.
Well, I have been having a lot of fun getting back into markdown converter stuff and working with regular expressions, and last week’s time lapse episode was a lot of fun. So how do we continue with this so far? Submarine doesn’t support that much as far as we’ve got tested so far. We’ve got headers, H one through five and italics and bold or emphasis and strong. Those are supported.
There are other one liners that I think might not be too bad to convert with regular expressions, links, images, inline code, horizontal rules, and light and breaks at the end of text. Those all should be pretty easy to implement. I hope stuff that I’m intentionally putting off for now, but hopefully not for long, are block quotes, code blocks, ordered list, unordered list, multi level lists, and paragraphs. And I’m putting up paragraphs partly because they’re block element. I’m putting all block stuff off, but I’m also I don’t really I got some issues with the way traditional markdown uses paragraphs. I think it’s a little weird, but I’m going to talk about that in a future episode. Possibly, maybe if it’s relevant to more people.
But anyway, so I’m intentionally putting that off for now. But also once I get done with all that stuff, I would like to also add ID and class features. It’s part of often called Markdown extra, but I’d like to have ID and class available for headers, links, and images because for styling and CSS it’s important. But that’s not really what I want to talk about today. Let’s get started with versions.
In the last episode I mentioned checking in and tagging different stages of development for the Submarine project, and if you want to follow along, you do something like this, you’d get cloned the project, see into the Submarine directory, create a virtual environment activate the virtual environment, and then you’ve got the latest code. But how do you look and see those other changes? The previous tags?
I had to look this up, and I’m glad I did.
I use it a lot, but for some reason, until I just looked it up for this time and wrote it down. It’s hard for me to remember, but it shouldn’t be. It’s just so if you’re looking at different branches with get, you do a get checkout and then the branch name, but instead of the branch name, you could just put the version and that works. So if I were to say get checkout v zero one, I would go look at version zero one because I tagged it that way.
But I’d also get a warning that says you are in a detached head state. That sounds ominous, but that warning is just basically to tell you there, to tell you to don’t make changes.
Think of it like a view only mode.
I’m not a good expert. Maybe you can continue to develop from there, but I’m not sure. So there’s a 0.1 that’s available. 0.20, .30 .40 .5. Those are all from last episode.
And for this episode, I did bump it to version zero point on Pip. There’s version zero six and 0.7, but in the Tags there’s only 0.7. I forgot to tag the 0.6. Sorry about that, but a little bit more about that in a second. After recording the last episode, I published the package. After I pushed it to GitHub, I did a flip publish.
Really. It’s just that’s the command flit space publish and that pushes it to Pip. So I went to check it out to see if it was there. And yes, it’s there, but there’s no text there. It was a blank description. So that was a bummer. How do I fix that? So that’s the first thing I did.
I did a little bit of research, just a very little. And I realized that there was a split supports a thing for metadata. And so there was one line I had to add to the pipe project Tomo file, and it’s under the tool. Flit metadata section. I had to add the description file and set it to the readmead markdown.
And then I tried to flip publish again, and it didn’t work because it already had a 0.5 there. So I bumped the version of 0.6, checked in, tried it again, and it worked great.
And then I moved on to future zero seven stuff, and I forgot to tag zero six. So sorry about that. But the only thing that changed is that one line of the reading for 0.7. I want to add more features. I’m going to start with links and some of the other one liner stuff.
But first I need to set up my development environment, and I like to develop in a TDD mode like this, where I’ve got my test in one tab or one panel of my editor, my code in another panel, and I want to be able to run so they’ve got right and left there side by side, and I want to be able to run my test and code the output and all that stuff.
So that’s one part just being able to get my head all set up. But I also need to install my code such that when I’m running my tests, I see I can make changes to the code and run the tests. But in last episode we talked about setting up the testing so that it looked at the test, look at the installed version, not the local version. So how do you do that?
Well, with setup tools based projects, you set the project to editable mode using Pip installe e for dot for the current directory, and that sets it up in development mode or in editable mode. But that doesn’t work for Piperoject Tomoebased projects.
For Flip, I’m using Flit, but Flit supports it, so Pip doesn’t yet support it. It probably will at some point. Just doesn’t yet. But you can use flitinstall.
Pth file. These are going to be in the Show Notes or Flint install s. So using a path file or a symbolic link, the s doesn’t work on Windows, so use the path file option for Windows.
For Mac and Linux you can use either. I often use the S just because it’s a little less typing as all the other thing I want to be able to do, like if I wipe out my directory and start from scratch, I’m cloning. I want to install my development tools. In a lot of projects, there’s a requirement underscore Dev. Text file. I don’t want to add that primarily because Flit already has a process that I can use. And the thing that Flit has is it allows you to add a requires extra section to the pipe project. Tomo file, and so I added one of those also. This will be in the Show Notes as well.
Or you can look on the project on the submarine project, but there’s just an extra requires extra line that I’m listing so far. Pitesticov and talks as a test argument to that.
It’s kind of hard to describe on the air, but it’s a pretty simple little extra change. I can do my project at Homel. So what that does in the test section is if those things are not already loaded when I do a flat install or a flat install s or a file. But any flat install will install the current project, but it also installs all of these, all the test tools. So I don’t need a requirements underscoredev text file. Yeah, actually I talked about this already, but it simplifies the development workflow for somebody new coming in. So the general idea then is clone the project, create the virtual environment, install Flit, run Flit, install with a Pathfinder or s, and that’s it. You’re ready to develop and test. You can run talks.
You can run pipe tests with coverage looking at the submarine directory.
You can run individual specify individual tests or run them from the editor. It all works great. So with my editor all set up, looking at the correct virtual environment and the pipe test set up as my test runner and my panels set up, I’ve got test code and right panel development code in the left. My test is running it’s great. I can start developing.
Thank you to Python for sponsoring this episode. Pycharm is designed by programmers for programmers to provide all the tools you need for productive Python development, including this TDD style development. It’s really easy to set it up with your test code in one pane, your source code in the other pane, your tests running at the bottom. There’s easy to run controls on the left side of the test tree. And there’s even an extra cool feature that I love a lot, which is run the past failed test. So when I’m doing a TDD model, I know I’ve got some failed tests because I haven’t implemented the feature yet. I can just rerun those tests easily with the run failed tests.
All of that is there with the free and Pro version. So why do I have the Pro version? I have the Pro version because when I want the extra power, I want it right away. Just recently, I was debugging a postcard database problem. I needed to connect to a database, look at the tables, and it was so easy. And PyCharm incredibly easy, super fast. There’s even a schema design diagram tool to help me visualize the data and the interaction. The Pro license for the time it saved me, just that one problem is totally worth it. So if you work with databases even once in a while, you should check this out. You can try out Pro for four months by going to testinggo.com PyCharm before August 2.
So with my code and editable mode and my tools installed, I’m ready to develop the next feature. Link Support I look up the markdown syntax and there are a couple of versions, the brackets containing link text and the link URL in parentheses, and another version that’s the same, but with a title in quotes after the URL. The title is what shows up near the cursor when you Hover over a link on a page. It’s a cool feature, but I forgot about those, but I plan on supporting both of them. So I opened up my editor with the submar project and I’ve got it all set up with the panels and everything. So what do I write? Well, up until now, even though there are a lot of test cases, there’s only two tests so far. I had a test convert which had a parameterization of nine test cases, and that tested all the conversions I had and the parameters that it took in, took an input string and an expected output. So take the input string, ran it through the top level convert method, and checked it against the expected output.
And that flow was easy, so the parameterization worked really easy. The other test was the command line pipe to make sure the command line interaction works. So I was just testing the pike version.
There’s another way to run Submarine, which is with a file.
I could have another test that checks to make sure that passing in a file works, and that’s not too hard to do with pipe test. I probably should do that.
Okay, so the reason why I didn’t do it so far is two reasons. One, I’ve manually tested and it works. I don’t like that reason though.
The other reason is all the heavy lifting for that interaction is done with the standard library module called file input, and the only way I can see that pipe working and file is not working, or as if that standard library component breaks. And that’s really unlikely.
Now that I’m talking through this though, there is a reason why I should add the test anyway. And it isn’t because I think that it’s ever going to catch a failure, because I don’t think it’s going to catch a failure.
The reason that I probably should add that test case is because that will highlight that mode of operation is important to this package, to the Submarine project, and that’s important for me in the future or any other future developers. And it’s possible that a future refactor of the code could implement that command line interaction differently and maybe not use file input, then the test developer or the developer in the future, even if it’s me would have no way of knowing that if the refactoring breaks that intended use model. So yeah, I’ll go ahead and put that in a future test to do list. But right now that’s not what I want to test. What I want to test is links, because I want to implement that. I could add this test case.
I guess there’s two test cases for the two kinds of tests, two kinds of links. I could ask that add that to test convert, but instead I’m going to just copy the structure of test convert into a new test called to test convert new stuff and rip out all the old parameters and put new parameters in for the new one.
The reason why I split it up.
It’s easy for me to run the new stuff. I can just right click on it in the editor and say run that.
It also separates the old and new functionality so that when I’m editing I’m changing stuff.
This stuff doesn’t work yet from the I broke something that used to work.
I don’t always do this, but I felt like doing it today. So I copy a couple of the examples as input and expected output of the link structure and markdown and what it should look like after converted as parameters into the test. I run the tests just to see that it fails. But it’s also not just to see it fails, but it also makes it the existing test run in my editor. And so it’s easier to rerun it.
I’ll be rerunning it a lot while I’m developing.
And then I wrote the test fail. I went and added the link function. It’s only a couple of lines of code. It was pretty easy to implement. That one went pretty fast, and so it went pretty fast. It went so fast that I decided to even speed things up faster. I had a lot of new things I wanted to add. I wanted to add images. I wanted to add inline code. I wanted to add horizontal rules. I wanted to add line breaks, the end of the text line. So four kind of new feature ish things, new features and markdown that I wanted to add. And I knew I needed to do lots of sorts of things. I needed to look up all the syntax to make sure I had the syntax correct. I need to write new test cases and then I needed to implement it. Not too bad, but I wanted to batch things. So I went ahead and batched all of those together. So I did all the looking up. I looked down up all the markdown syntax for all these features at once and wrote them down. And then instead of just writing them down anywhere, I wrote them down as new test cases to the test convert, new stuff test.
So I’ve got a whole bunch of new test cases. They all fail because they implemented yet.
So I batched the researching stuff, I batched the writing the test cases, and then I can focus on just keep running, rerunning the tests and implementing everything until it goes pretty good. And the images threw me for a loop right away. So the image syntax looked like it would be a little tricky as a regular expression. So in the implementation, I also copied the example from the test into the implementation just as a comment.
I did that so that the regular expression that I was implementing was right underneath the comment of the example. So it helped me develop that.
It still was confusing. But anyway, adding the comment to the function helped me for implementation. So I just went ahead and did that. For the rest of these batch things, I did the examples.
It worked cool. So I just kept it up and then I got done. So once I got all these implemented, I re ran all the tests, I re ran talks and everything was green. So I’m done right. I can bump it to 0.7, call it good. Not quite. The read me was wrong. So I do have documentation, but the documentation is just the read me. But anyway, the README has examples of what’s supported so far. So I’m supporting new stuff so I need to update the documentation. Makes sense. And glad I went back and did this because I went back and looked at the wrote examples and for one of the features, I think it was images. There was an extra parenthesis in a weird place and it didn’t make sense. I’m like why is this parenthesis here? It’s not matched up. Doesn’t make sense. And I went back and looked at the test and the code and saw that the error was in both places. And I think one of the things that happened is I was writing the test and the code at the same time. But I did take a break after I was done and then come back with fresh eyes and went up to look at do the documentation and I spotted it then and I’m glad I did. So extra parentheses didn’t need to be there. So I fixed the test and the code broke of course. So I fixed the code and now the code and the test and the documentation match what I really wanted to do. So Yay documentation. One more way to help find errors now that I’ve had all the tests done. So I’ve got the new features in the new test, the test convert, new feature, new stuff, whatever I called it, I could have those, I could have all moved all those into the other test. The existing test parameterization.
That would have worked just fine for follow along purposes. I left them in the test new stuff function so that you guys can look at it. It’s checked in under zero point 70.
Moving forward, I’m probably going to move those parameterizations into 0.8. And the reason why I’ve seen this done before, even in work environments, you’re writing new features, you’re writing a new test. I think that’s fine, but don’t leave them there if they really belong somewhere else. And the problem is the logic of the two tests are identical. And if there are two tests with program amortization with identical logic, why are they two tests?
If the only reason is time, that doesn’t make sense in the future. So somebody else maintaining the codes are going to look at it and go why are these different? And if they’re really not different and there’s no reason, then go ahead and move them together. I was hoping with the new stuff name to be obvious and boring as to why, but it’s not going to be new stuff for long, so eventually it should go back.
When I was documenting it, I noticed a few more features while documenting and writing notes for this episode that I missed.
Inline code blocks are done with backticks, but what if you have a backtick in your code block that you’re trying to Mark down?
That’s tricky. So one of the ways that markdown allows that is for double back ticks. So my implementation doesn’t allow double back ticks. It’d be cool if it does doesn’t yet. So I’m putting it on the list of things to do in the future. Also, I noticed that the Alton title section for images are not optional. They should be optional, so that goes on the to do list in the future.
And also I don’t have a to do list yet, so I’m doing all of this on GitHub, so I’ll try to remember soon to convert all of these notes into actual issues in GitHub. That’d be great. And then once they’re there, maybe you can help me fix them or implement that’d be great.
I’m having fun with this project.
The next episode probably won’t be here on this, but who knows? I’m having quite a lot of fun. But one of the things I want to recap some of the things that we’ve covered in this episode because I kind of jumped around a lot. So get Tags are great for being able to look at different versions of the Submarine code or any other project that uses Tags. Check out the prime project. Tom will changes in Submarine. The changes that to look at are for the README so that it shows up on Pip and the Extras section to do the development dependencies.
Flitinstall path file or flitinstall. S will allow you to install the project in editable mode so you can run the tests and talks locally as well as installed. And it’s all the same.
The TDD style of testing of test, then code, then test and code. That works fine, of course, but you can also batch work and write a bunch of test cases and then run a bunch of code that works fine also. And it might save time and task switching, so don’t dismiss that. And the last thing is that documentation is a great debugging tool. Also, in this example, I found a typo that’s hard a typo for this project, but I also found in the past I found bigger problems, and often they are in the form of it’s hard to document. It’s hard to tell a user how to use it. If it’s hard to document, that’s a red flag, and maybe the design needs changed.
Also, that’s similar to if it’s hard to test, that’s a red flag.
So hard to test, hard to document. Those are good problems to find out early so you can change them before your users are dependent on that. And last but not least, Pyramid is awesome. Don’t forget to try out the TDD model and also some of the pro features, like inspecting your databases by going to testandcode.com PyCharm and gathering your four month free trial before August 2. Thanks to PyCharm for sponsoring this episode. Thank you for listening and for recommending the show to friends and colleagues and on social media. Thank you to patrons, reporters, you totally rock. And the link for this. The show notes for this episode are testandcode.com 81. That’s everything so go out and test something thanks.