In Delayed assert / multiple failures per test, I presented a first attempt at writing an ‘expect()’ function that will allow a test function to collect multiple failures and not stop execution until the end of the test. There’s one big thing about that method that I don’t like. I don’t like having to call ‘assert_expectations()’ within the test. It would be cool to push that part into a plugin. So, even though this isn’t the prettiest code, here’s a first attempt at making this a plugin. Test code that uses expect() Local conftest.py plugin for delayed assert Changes to delayed_assert.py Seeing it in action Possible issues and things I don’t like Alternative solutions Next Steps ...

A test stops execution once it hits a failing assert statement. That’s kinda the point of an assert statement, though, so that’s not surprising. However, sometimes it’s useful to continue with the test even with a failing assert. I’m going to present one method for getting around this restriction, to test multiple things, allow multiple failures per test, and continue execution after a failure. I’m not really going to describe the code in detail, but I will give the full source so that you can take it and run with it. Reasons for multiple assert statements and not stop execution Using a failure list to keep track of failures within a test Example test code that uses the delayedAssert module And an example for unittest The output for unittest The output for pytest The output for nose The delayedAssert.py module Feedback welcome ...

I had assumed that everyone has heard the story about the blind men and the elephant. However, in a very non-scientific poll of a hand full of fellow engineers at my day job, only about half had. So I was going to try to quote it here, but when I looked up a reference for it, I came across a joke that amused the pants off me. So here’s the joke: Six blind elephants were discussing what men were like. After arguing they decided to find one and determine what it was like by direct experience. The first blind elephant felt the man and declared, ‘Men are flat.’ After the other blind elephants felt the man, they agreed. Moral: “We have to remember that what we observe is not nature in itself, but nature exposed to our method of questioning.”- Werner Heisenberg wikipedia entry Well. I thought it was funny. Trust me that this ties in with software development and testing. ...

I don’t rememember how I ran across this article by James O Coplien. However, I was immediately impressed with the thought and experience that went into this paper. Regardless of your viewpoints towards unit tests vs. other types of automated tests, this article is important to read. If your first reaction to the title is anger, please take a deep breath, try to keep an open mind, and actually READ what Cope has to say. I am going to reserve my own reactions to this to a future post, as I don’t want to color your views before you read it. I am posting the entire article with no changes other than formatting. ...

Whatever your stance on the merits or pitfalls of Test Driven Development, I think it’s worthwhile and educational to pay attention to a discussion that’s going on lately. Testing is crucial. But is unit test focused TDD the right path? I care about the conversation about TDD because I see serious flaws in the conventional understanding of TDD. Much of the current view of TDD includes: Units are tested in isolation of the rest of the system. Unit tests are more important than any other form of testing. A “unit” is a class or a function. Nothing larger is a “unit”. If you test more than one class, that’s an integration test. If you test from the API with all resources, that’s a system test. Let QA deal with that later. Isn’t that exactly where waterfall failed? You can’t write any production code without a failing test. You have to write only one test at a time, and it must fail. Tests have to be fast. Therefore, they cannot touch hardware, the file system, other services, or database. Tests should be short. All of this rubs me the wrong way. I’ll get to my thoughts later, but my concern about this cemented view of TDD caused me to be very interested in the current talks. On to the discussion I came in after the 2nd video, while doing research on Agile and TDD. I’m not sure if the order matters, but here’s a list of what I know about the discussions. ...

I’ve been writing about the basics of using python test frameworks for over a year now. Still todo Big picture My history clouding my vision My code is done. Now how do I write unit tests for it? Short answer My part ...

In pytest fixtures nuts and bolts, I noted that you can specify session scope so that a fixture will only run once per test session and be available across multiple test functions, classes, and modules. In this post, I’m going to show a simple example so you can see it in action. ...

I’d like to wrap up this recent series of pytest fixture posts by presenting my version of some sort of reference. Since this post is running a bit long, here are some links to the content buried in here. Bare bones example Three ways to use a fixture Name it, usefixtures, and autouse. Usefixtures example Fixture features Return value Finalizer is teardown Request object Scope Params Toy example Real example Autouse Multiple fixtures Modularity: fixtures using other fixtures Experimental and still to cover yield_fixture ids ...

In pytest xUnit style fixtures, I presented a problem where: Two tests exist in a test file. One uses a resource. The other doesn’t. Module level fixtures don’t work if you just want to run the one function that doesn’t use the resource. I then presented class level fixtures as a way to solve the separation problem. In this post, I’ll use pytest fixtures to solve the same problem. I’m not going into details of all the goodies you get with pytest fixtures. I’ll just stick to solving this problem. ...

I’m going to cover the syntax for pytest support for xUnit style fixtures. Then I’ll give a more reasonable and typical example, using just one set of fixture functions. And then address the issue of having tests mixed in a file. Some that need the resource, and some that don’t. ...