A story about packaging, and flit, tox, pytest, and coverage. And an alternate solution to “using the src”.


Transcript for episode 80 of the Test & Code Podcast

This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.


Welcome to Test and Code. This episode is a story about packaging and flit and talks and by Test and coverage and an alternate alternative solution to using the source. Python makes it easy to build simple tools for all kinds of tasks, and it’s great to be able to share small projects with others on your team, in your company or with the world. When you want to take a script from just a script to maintainable package, there are a few steps, but none of that are also the structure of the code layout changes over time to help with the growth and support of the project. Instead of just talking about this from memory or theory, I thought it would be fun to create a new project and walk through the steps and report back in a kind of time lapse episode. It should be fun. This is Test Encode Episode 80.

Thank you to Pantheon for sponsoring this episode. Pantheon makes building, managing, and optimizing websites simpler. Check them out at pantheon. Io Test And Code and listen to their section later in the show.

Welcome to Test and Code, a podcast about software development, software testing, and Python.

Let’s say I have a little project, maybe even just a script or a module, and I want to Polish it into a project that I can share and maintain. What does that mean?

What I mean when I say that is that I want people to be able to Pip install it. This may be on Pip or an internal server or even just a share drive somewhere, but essentially it’s a wheel or a source distribution. I’m usually doing wheels lately. I also want full test coverage, and I want the tests to be testing the installed version, not the local version. Talk more about that later, and probably some documentation, even if it’s just to read me, but I’m getting it ahead of myself. Let’s get some code down first and I’ll talk about it. Oh yes. What is the project that I’m working on? It’s a project called Submarck. Submarc is a markdown converter for a subset of markdown using regular expressions. It’s loosely based on the original test tutorials I did for Python testing. Net many years ago, and I wanted to try it out again with all of the stuff I’ve learned over the past few years, and I also want to play with Flit. Flit is the packaging tool. That’s what I use it for. Okay, I’m going to write some code and get an initial version, and then I’ll come back shortly and I’ll talk about it.

Now I know what you may be thinking. There are three famous classic blunders in the order of one, never get involved with Land war in Asia, two, never go in against the Sicilian with death is on the line, and three, never write an HTML parser with regular expressions, since traditional markdown is allowed to have HTML in it. The warning to not write an HTML parser with a Reg X is probably applies to markdown as well. However, I’m trying to convince myself that it will be okay since first I’m only implementing a subset of markdown. And second, I think it’s a cool example to teach people some regular expressions, search and replace examples. Okay, so there’s my problem space a subset of markdown. I’ve got my first implementation done. I really thought I could just do it with a single file and it kind of is the implementation is a single file, aka a script, since I run it with Python three spacesubmark Pi. So it’s just running submar. Py with a Python. Well actually I also have to pass some markdown to it to convert it and I’ve set it up so that you can either pipe it to it using standard in or using a file so you can pass it a file name and it will convert that and all the output goes on standard out. Anyway, enough about the project.

The implementation is there in that file, but I also want to test it. So there’s also a test file. And if I want to run and they’re in one directory and if I want to run coverage, I’ve kind of got a mess on my hands because you give coverage a directory where the source is of where to measure the coverage. And this directory is got a whole bunch of stuff in it. It’s jumbled with test code and my source code, which I want, but it’s also got my virtual environment directory in there. So in my initial version I also wrote a coverage coveragerc file.

It’s very simple, it just tries to exclude.

I just excluding the virtual environment directory right now. Running coverage on tests isn’t terrible.

So the coverage works, the tests are running.

I also want people to know how to use it. So I wrote a read me in markdown, of course, and I’m going to push it to GitHub. So I’ve got it in git and so I also have a get ignore file. So even though I just have a little script, there’s a bunch of other stuff already. There’s a bunch of other files that are around just to get this to work, but they’re mostly straightforward. And actually at this point I’ve pushed it up to GitHub.

I tagged them with v. 0.1. So version. 0.1 tag on this code just to show you the little script. It’s really not that big, but it’s all up there. And at this point this sort of a project for a lot of people, this might be enough. This actually works okay. If I just want to have people be able to download the file and use it themselves.

I can’t Pip install it, but it’s there simple scripts like this are done a lot. I think they’re done sort of this process usually internally.

Well, maybe I put test and code and I’m hoping that people will put full test coverage and READMEs even for their scripts they’re sharing with their colleagues. That might be a little bit optimistic, but whatever. Now for the next version.

Thank you to Pantheon for sponsoring this episode. Pantheon Makes building, managing, and optimizing Website Simpler as the leading Web Ops platform, Pantheon features the fastest hosting on the planet, automated workflows with development, test and live environments, plus professional development tools and easy integrations with GitHub, CircleCI, Jira, and many more.

Stop putting out fires and start building sites and experiences that deliver results using Pantheon, rated as one of G two crowds top ten software products of 2019.

Get started for free at Pantheon. Io Test And Code that’s Pantheon P-A-N-T-H-E-O-N-I-O for version two, or more specifically, version 0.2, I want to be able to create a wheel to distribute or publish in some way. Thanks to Flip, this could not be easier. Really. The simple state I had the project in for version zero one was almost there. What do you do to get it to make a wheel? Pip install flit Flint init two commands. That’s it.

Flitinit asks a few questions, but it already has some filled out defaults that are usually right for me after I’ve used it a few times. So it asks for the module name. But I guess it’s the module name from the modules and packages in the directory it asks for my name, the author, author’s email, a home page, but it also guesses the GitHub repo based name based on the module name. But you can change that. It gives you a license choice either MIT, Apache, GPL or Skip. So you fill it in yourself that’s it takes a few seconds, run it. It generates a Pi, project, dot Tomo file and a license file if you don’t already have one. And those two check those two files into get and do a commit. And there you go. What do you do now is you do a flip build.

Yeah, that’s right, flip build. And that will give you a wheel and a source distribution sitting in a disk folder.

Really, it’s that simple. I did Flip and Flip build and I’ve got my wheel.

Now I can use that wheel and put it in Pip or in some other package or a directory or somewhere or an internal server for people to piping it from there. And there you go.

In fact, if you want to push to Pi or somewhere else, you can do flip publish.

I checked this into GitHub as submarked version 0.2 with an updated read me to talk about Flip a little bit, but really the only changes are there’s a new license file, a pipe project Tomo file, and I updated the version number in submark Pi.

That’s it.

It’s just a little tiny change.

The test still run, but I’m checking this in a 0.2 so that you can take a look at how little of the changes.

Now let’s look how I’m doing on my goals for polished maintainable package goals so people can install it because I’ve got wheels working there’s some documentation in the README and full test coverage ish I thought I had 100% coverage. Well, I’m creating distributions that can be Pip installed, but that whole process is not getting tested. The test still runs the local version that distributions can be built and installed. Let’s add one more tool to the mix. Let’s add Talks.

I use Talks for almost every project.

You think about Talks when you’re trying to test against multiple versions of Python, but right now I’m just using it to automate a lot of this stuff. We can add more interpreters and more checks like Flaky and other things later, but for now, just having it build the distribution and run the tests against the distribution are a great help. So to get started, just Pip install talks. And then I’m going to write a really simple tax any file.

And since I’ve used it, I kind of copy it from other places. But I’ve got a bunch of simple ones around and I’m just going to add a simple one and then run Talks.

Now, I actually already did that, and my tests get run, but the dot talks directory is included in the coverage report.

That’s just nasty. It’s just huge.

So I can change that by adding talks to the exclude list in coverage. Rc. So that’s nice. I run it again and everything’s great. Sort of. Well, not really. I mean, I get less coverage, but I do notice something. I noticed that when I’m looking at Talks, I’m actually within the talks any file talks is looking at my project directory for files and I’m just running pipe test like it was before. I was putting coverage at the local directory and they’re still at 100%. So I’m doing 100% coverage of the local stuff. That means I’m not testing the installed packages.

I got to change that. But I am using Talks now to start running tests, and that’s good. So I’m going to check this in so you can see that little bit of incremental change. And this is at version 0.3.

So now we have a couple of things to fix. First, I want to get coverage to measure the installed packages and not the local one. And I want pytest to test the installed packages and not the local one. So I’m going to move Coverage to look at someplace else, and I’m going to move Pipe Test to not see the local stuff. That should do the trick. So let’s take this one at a time. First, the coverage issue I’m passing in. I’m using pitestash Cove, which is a little bit of different flag, but it’s a flag that’s dash CoV equals, and then you give it a source, direct the directory to look for the source and you give it a directory.

So I’m telling coverage where to look at it. I’m telling it to look at the local directory, the current directory.

I can tell it to look at the site packages directory and there’s ways within talks where there’s shorthands to say hey, look in the talks generated virtual environment site packages and look there. And then I can add subdirectors, but there’s no subdirector ad. When I look at this site packages within talks, I see the submarine. Pi file just sitting there because it’s just this one module that’s getting installed.

So I’d rather have a directory so I can point coverage at it. So let’s make a submar folder and add the submarck top high to the directory and then add a Dunder init file. Okay, I’ll go do that.

Well actually when I was looking at the code, I noticed that I did a little bit more work than I thought I was going to do. I noticed that there was this really small main function that acts as a command line interface and what it does is it takes the standard input or file contents if you pass it a file and turns that into a list and passes that to a single high level function and in the rest of the code. So I decided to split things up. I decided to split up the code. I’ve already got it in a different directory. They’re in the submarine directory now. So now I’ve got three files in there. I’ve got a small CLI. Py file with the main function. I’ve got submarine. Pi, which has all the regular expression methods and the high level functions that don’t include external stuff. They’re just calling these low level regex functions and then the Dunder in it that imports the stuff I want to export as an API, which is namely just the main method from the CLI tool and a top level convert function from the other module.

I also updated the piproject. Tomo file to add this main as a script. So when I’m installing stuff, I want to be able to install submarine as a little script, a command line tool. And since the submarine. Pi is not going to be around it’s a package. Instead it’ll be just a submark without the dot pie.

And then I got the test we’re calling that. So there was one test that was calling that directly. I changed that to call the command line tool instead.

Now I’m feeling pretty good about these changes. I think they went pretty well.

I can now change the talks any file to point coverage at the installed version of submarck, since it does not exist as a directory and packages.

And now since it’s by itself I’m putting coverage there, I don’t need to exclude anything. I can get rid of the dot coverage. Rc file because the only thing I was using it for was to admit things so I don’t need it.

That’s where I’m at with version zero four but there are still problems.

So I’ve got coverage pointed, but I didn’t fix the other problem.

Coverage just still shows that all tests pass with zero coverage. We got to fix that.

Now we’re ready to fix the issue that the tests are not testing the installed package, they are testing the local package. The problem is that the tests are in the same direction as the project, and Python path search rules tell Python to look there first. It totally makes sense. That’s kind of how we want things to work normally.

The solution to this I used to promote for this problem is to create an SRC or the source directory SRC and put the submarine package in there inside of it. So you just got one level of indirection. Then you can there’s ways to change the setup pie if you’re using setup tools to look in the right place and blah blah, all is good. However, Flit doesn’t let you do this. Flit doesn’t like the source directory, the SRC, and at first I was frustrated with Flip, but I’ve come around to realize that it’s just a tooling problem. We should be able to get around this.

And it does feel like this is a long explanation to tell people why they have to have an SRC directory. So I felt there’s got to be a better way. And there is. And it’s been there all along. Well, it’s been there for a while. I just didn’t know it. So the first thing we have to do is create a test directory. T-E-S-T-S.

That’s the normal convention and put our test file in there. There’s only one file right now, but it’s eventually going to grow.

And I can split up the test cases into multiple files, and then I can just point pytest at the test directory. And the way it goes, it runs everything.

Actually, now that I have the test directory, if I run pytest at the top level of the project, pytest sees the test directory and does the right thing, and does not include the current directory in the Python path, which lets us not see the local package for the tests. This is perfect. However, if you happen to be running pytest not from the direct pytest command, but from Python. Mp test, or more commonly, running amp pytest to have coverage run pytest. That’s done in a lot of different projects.

In either the Python M or coverage run pytest, we’ll add the current directory to the Python path and we haven’t gained anything.

That’s actually kind of where the source directory advice came from. But okay, better way. The better way is to add a line to the talks any file to change the local directory to the test directory before running the test. Very simple. There’s a change to command and talks that you can just add in there. And now you can run either pytest or Python MP test or coverage runmp test whatever you want it all just works. So I’ve made these changes and they’re checked in and it’s very simple changes checked in as version 0.5 and all the tests run they run 100% and they do show that they’re running against the installed stuff so this project is still in its infancy. It would be fun to add to it. I want to add things like testing different versions of Python. I want to be able to test against 3.8 because I know 3.8 is coming up right now. I’m just using 3.7. Maybe I’ll backtrack and test 3.6 also.

Anyway I hope that stepping through this process of going from a simple script to a supportable package with test and code and everything and all the hurdles you have to jump through but I jumped through them for you and if you want to just follow my path this should work.

I hope this was helpful. Thank you for listening. Thanks to Patreon supporters, you totally rock and also thanks to pantheon for sponsoring this and the last few episodes. I really appreciate it. Pantheon makes building, managing and optimizing websites simpler. Check them out at pantheon. Io Test And Code that link is also on the show notes at testandcode.com 80 I’ll also include quite a few links of all the stuff I discussed in this episode and again all the code that I was talking about and all these versions is at GitHub coma submarck but again that’s also a link that’s going to be a testandcode.com 80 I hope you enjoyed this. I did and I’ll talk to you next time go out and test them.