pytest support for unittest style fixtures

As with nose, I ran pytest on the tests written to demonstrate unittest fixtures.
Some surprises.
There are some differences, which I’ll list out in this post.

Before we get started, I’d like to express that I’m a huge fan of pytest.
The cool stuff in pytest far outweighs any invonvenience that it doesn’t behave exactly like unittest.

However, it’s good to know what the differences are, so you can be informed about how your unittests will run under pytest.

This testing is using pytest 2.3.5.

Update: As of pytest 2.4, these issues have all been fixed. pytest now fully supports unittest fixtures

module: setUpModule()/tearDownModule() ~~NOT~~ supported ~~(use setup_module()/teardown_module() instead)~~
class: setUpClass()/tearDownClass() supported
around methods: setUp()/tearDown() supported
add cleanup functions: addCleanup() called from setUp() and from a test supported
skipping tests dynamically: testSkip() called from setUp() supported
error conditions: ~~handled differently than unittest~~

Missing support for setUpModule()/tearDownModule()

In my opinion, this is just an oversight.
The current head of the development branch of pytest DOES support these functions.
I know, because I just changed the code and submitted a pull request. And it’s been excepted.

However, 2.3.5 doesn’t support these functions.

Differences with error conditions

In unittest, the matching tearDown function is NOT run if the setUp function fails.

If setUpModule() throws an exception, tearDownModule() is NOT run.
If setUpClass() throws an exception, tearDownClass() is NOT run.
If setUp() throws an exception, tearDown() is NOT run.
If setUp() trhows an exception AFTER adding a cleanup function with addCleanup(), the cleanup function IS run.

What does pytest do?

Well, ~~setUpModule() isn’t supported yet.~~ But let’s look at setup_module() instead.

If setup_module() throws an exception, teardown_module() IS run.

If setUpClass() throws an exception, tearDownClass() IS run.

If setUp() throws an exception, tearDown() is NOT run.

If setUp() trhows an exception AFTER adding a cleanup function with addCleanup(), the cleanup function IS run.

So, for pytest, the only tearDown that doesn’t run in setUp failure cases.
All of the other tearDown like functions ARE run regardless of the state of the matching setUp function.
Whew… Still with me?

Update: As of pytest 2.4, these issues have all been fixed. pytest now fully supports unittest fixtures.
But… I gave up on strike-throughs, I just need to rewrite this post…
In the meantime, suffice it to say that you can use pytest to run unittest without fear of fixtures doing anything different.

They just work.

Good thing or bad thing?

All tests that pass with unittest will pass with pytest.
All tests that fail with unittest will fail with pytest.
All tests that error with unittest will error with pytest.

However, if you are relying on a tearDown like function NOT running, you have to watch out.
That seems like a small population though.

I’m not really even sure what the behaviour SHOULD be.

When could it be a good thing?

If you have several resources that you are opening, allocating, etc, in setUpClass(), then…
For unittest and nose, you need to make sure you clean up everything in setUpClass() if anything fails.
For pytest, you could only clean up in tearDownClass().

However, I wouldn’t rely on this behavior.

It’s extremely possible that a future release of pytest will match the unittest/nose behavior regarding this bit.

Code output TL;DR

You can look at my evidence if your bored.

>py.test -q -s test_fixtures.py::TestFixtures
..
in class TestFixtures - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in test_2 - test_2()
in test_2 - tearDown()
in class TestFixtures - tearDownClass()

>py.test -q -s test_fixtures.py::TestAddCleanup
..
in class TestAddCleanup - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_1 - cleanup_b()
in test_1 - cleanup_a()
in test_2 - setUp()
in test_2 - test_1()
in test_2 - tearDown()
in test_2 - cleanup_a()
in class TestAddCleanup - tearDownClass()

>py.test -q -s test_fixtures.py::TestSkip
.s
in class TestSkip - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in class TestSkip - tearDownClass()
>py.test -q -s test_fixtures.py::TestExceptionInSetUp
FF
=================================== FAILURES ===================================
_________________________ TestExceptionInSetUp.test_1 __________________________

self = <test_fixtures.TestExceptionInSetUp testMethod=test_1>

    def setUp(self):
        TestFixtures.setUp(self)
>       raise DemoException
E       DemoException

test_fixtures.py:119: DemoException
_________________________ TestExceptionInSetUp.test_2 __________________________

self = <test_fixtures.TestExceptionInSetUp testMethod=test_2>

    def setUp(self):
        TestFixtures.setUp(self)
>       raise DemoException
E       DemoException

test_fixtures.py:119: DemoException
in class TestExceptionInSetUp - setUpClass()
in test_1 - setUp()
in test_2 - setUp()
in class TestExceptionInSetUp - tearDownClass()
>py.test -q -s test_fixtures.py::TestExceptionInTearDown
FF
=================================== FAILURES ===================================
________________________ TestExceptionInTearDown.test_1 ________________________

self = <test_fixtures.TestExceptionInTearDown testMethod=test_1>;

    def tearDown(self):
        TestFixtures.tearDown(self)
>       raise DemoException
E       DemoException

test_fixtures.py:124: DemoException
________________________ TestExceptionInTearDown.test_2 ________________________

self = <test_fixtures.TestExceptionInTearDown testMethod=test_2>

    def tearDown(self):
        TestFixtures.tearDown(self)
>       raise DemoException
E       DemoException

test_fixtures.py:124: DemoException
in class TestExceptionInTearDown - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in test_2 - test_2()
in test_2 - tearDown()
in class TestExceptionInTearDown - tearDownClass()
>py.test -q -s test_fixtures.py::TestExceptionInSetUpClass
EE
==================================== ERRORS ====================================
______________ ERROR at setup of TestExceptionInSetUpClass.test_1 ______________

cls = <class 'test_fixtures.TestExceptionInSetUpClass'>

    @classmethod
    def setUpClass(cls):
        TestFixtures.setUpClass()
>       raise DemoException
E       DemoException

test_fixtures.py:108: DemoException
______________ ERROR at setup of TestExceptionInSetUpClass.test_2 ______________

cls = <class 'test_fixtures.TestExceptionInSetUpClass'>

    @classmethod
    def setUpClass(cls):
        TestFixtures.setUpClass()
>       raise DemoException
E       DemoException

test_fixtures.py:108: DemoException
in class TestFixtures - setUpClass()
in class TestExceptionInSetUpClass - tearDownClass()
>py.test -q -s test_fixtures.py::TestExceptionInTearDownClass
..E
==================================== ERRORS ====================================
___________ ERROR at teardown of TestExceptionInTearDownClass.test_2 ___________

cls = <class 'test_fixtures.TestExceptionInTearDownClass'>

    @classmethod
    def tearDownClass(cls):
        TestFixtures.tearDownClass()
>       raise DemoException
E       DemoException

test_fixtures.py:114: DemoException
in class TestExceptionInTearDownClass - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in test_2 - test_2()
in test_2 - tearDown()
in class TestFixtures - tearDownClass()
>py.test -q -s test_fixtures.py::TestExceptionInTest
F.
=================================== FAILURES ===================================
__________________________ TestExceptionInTest.test_1 __________________________

self = <test_fixtures.TestExceptionInTest testMethod=test_1>

    def test_1(self):
        TestFixtures.test_1(self)
>       raise DemoException
E       DemoException

test_fixtures.py:129: DemoException
in class TestExceptionInTest - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in test_2 - test_2()
in test_2 - tearDown()
in class TestExceptionInTest - tearDownClass()
>py.test -q -s test_fixtures.py::TestFailInTest
F.
=================================== FAILURES ===================================
____________________________ TestFailInTest.test_1 _____________________________

self = <test_fixtures.TestFailInTest testMethod=test_1>;

    def test_1(self):
        TestFixtures.test_1(self)
>       self.fail()
E       AssertionError: None

test_fixtures.py:134: AssertionError
in class TestFailInTest - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in test_2 - test_2()
in test_2 - tearDown()
in class TestFailInTest - tearDownClass()
>py.test -q -s test_fixtures.py::TestAssertInTest
F.
=================================== FAILURES ===================================
___________________________ TestAssertInTest.test_1 ____________________________

self = <test_fixtures.TestAssertInTest testMethod=test_1>

    def test_1(self):
        TestFixtures.test_1(self)
>       assert False
E       AssertionError: assert False

test_fixtures.py:139: AssertionError
in class TestAssertInTest - setUpClass()
in test_1 - setUp()
in test_1 - test_1()
in test_1 - tearDown()
in test_2 - setUp()
in test_2 - test_2()
in test_2 - tearDown()
in class TestAssertInTest - tearDownClass()

#

Missing support for setUpModule()/tearDownModule()#

Differences with error conditions#

Good thing or bad thing?#

When could it be a good thing?#

Code output TL;DR#

Missing support for setUpModule()/tearDownModule()

Differences with error conditions

Good thing or bad thing?

When could it be a good thing?

Code output TL;DR