Also, I should have tested it better. :)
In the audio I got the numbers wrong.  Doh!
This is episode 238, not 237. Oh well.

I'll still be around, of course, at:

• pythontest.com - where I write about developing software with tests
• pythonbytes.fm - Python news and headlines, delivered directly to your earbuds

Thanks for all the fun over the last 10 years.
I wish you the best.

Links:

• FastAPI Cloud
• FastAPI
• SQLModel
• Typer
• OpenAPI
• Pydantic
• Sequoia Open Source Fellowship

Key topics include 

• "cherry picking" for selective commits
• "git stash" for managing in-progress work
• "git diff", and specifically its `--name-only` flag, which provides a streamlined method for developers to identify which files have changed, which can be used to determine which tests need to be run
• "git bisect" for efficiently pinpointing bugs. 

This conversation offers valuable strategies for developers at any skill level to enhance their Git proficiency and optimize their coding workflows.

Links:

• Boost Your Git DX - Adam's book

Links:

• pytest-django - a plugin for pytest that provides a set of useful tools for testing Django applications and projects.

Links:

• pytest-metadata - The plugin we're talking about in this episode
• pytest-base-url - Adds the base URL to the metadata.
• pytest-html - Displays the metadata at the start of each report. 
  • See S2:E6: pytest-html - a plugin that generates HTML reports for test results
• pytest-reporter-html1 - Presents metadata as part of the report.
• pytest-selenium - Adds the driver, capabilities, and remote server to the metadata.

If you've got other plugins that work well with pytest-metadata, please let me know.

Normally, a test function will fail and stop running with the first failed assert. That's totally fine for tons of kinds of software tests. However, there are times where you'd like to check more than one thing, and you'd really like to know the results of each check, even if one of them fails.

pytest-check allows multiple failed "checks" per test function, so you can see the whole picture of what's going wrong.

Links:

• pytest-check
• Top pytest plugins

Can AI play a role in creating automated software tests?  
Well, yes. But it's a nuanced yes.  

Anthony Shaw comes on the show to discuss the topic and try to get AI to write some test for my very own cards project.

We discuss:

• The promise of AI writing your tests for you
• Downsides to not writing tests yourself
• Bad ways to generate tests
• Good ways to ask AI for help in writing tests
• Tricks to get better results while using copilot and other AI tools

Links:

• The cards project
• A video version of this discussion: Should AI write tests?

• is a pytest plugin that makes it easy to repeat a single test, or multiple tests, a specific number of times.  
• works fine on Python 3.14
• is tested on Python 3.9-3.14
• probably works fine still on 3.7 & 3.8

This episode also discusses the attempted April Fools episode.

Links:

• pytest-repeat
• The April Fools episode: Python 3.14 won't repeat with pytest-repeat
  
  

Note: This was an April Fools attempt, so the statement ...
"Unfortunately, it doesn't seem to work with Python 3.14, even though there is no rational reason why it shouldn't work." 
... is NOT true.

Listen to the NEXT episode to get an explanation

Links:

• pytest-repeat
• Next episode: pytest-repeat - works fine on Python 3.14

• pytest-html
• repo readme with screenshot
• enhancing reports
• pytest-metadata

Links:

• pytest-md-report
• pytest-md
• Top pytest Plugins

Michael Foord was a pivotal figure in the Python community and the creator of the mock library that's now unittest.mock. 
But he did so much more as well. 
His contributions continue to resonate within the developer community. 
This interview is just a small peek at his influence. 

In this episode

• Introduction to Michael Foord
• The Mock Library Origins
• Mocking and Testing Philosophy
• Career Path and Consulting
• Understanding Mocking
• The Power of Patch
• Testing Strategies and Design
• Mocking External Dependencies
• Teaching Testing and Mocking

python.org has put up an "In memoriam" page for Michael Foord, and many people have shared stories and memories.

Links:

• In memoriam: Michael Foord 1974-2025
• 30 best practices for software development and testing

In this episode:

• Why the pytest-mock plugin is awesome
• What is mocking, patching, and monkey patching
• What, if any, is the difference between mock, fake, spy, stub. 
• Why we might need these in testing
• Some history of mock in Python and how mock became unittest.mock
• From unittest.mock
  • patch.object
  • patch.object with autospec
  • using these as context managers
• pytest-mock:
  • The mocker fixture 
  • Cleanup in teardown
  • Using mocker.patch, mocker.spy, and mocker.stub
  • Why it's awesome and why you might want to use it over straight unittest.mock

Links:

• top pytest plugins list
• pytest-mock documentation
• unittest.mock
• Podcast episode discussing unittest.mock with Michael Foord
• monkeypatch

In this episode, we'll discuss:

• what Coverage.py is
• why you should measure code coverage on both your source and test code
• what pytest-cov is
• extra features pytest-cov gives you over and above coverage.py
• and generally why using both is awesome

Links:

• coverage.py
• pytest-cov
• how to set up context reports
• Top pytest Plugins

Errata:

• I mentioned that Coverage has the ability to show context (which line is covered by which test) for the past year or so.
  • However, that feature was released in Oct 2018. coverage 5.0 alpha 
  • That's over 6 years. Oops. Sorry Ned.

In this episode:

• Introduction to pytest plugins
• The pytest.org pytest plugin list
• Finding pytest related packages on PyPI
• The Top pytest plugins list on pythontest.com
• Exploring popular plugins
• Learning from plugin examples

Links:

• Top pytest plugins list
• pytest.org plugin list
• Top PyPI Packages
• And links to plugins mentioned in the show can be found at pythontest.com/top-pytest-plugins

This episode discusses

• the benefits of writing things down
• preparing for a meeting
• taking notes in meetings
• reviewing notes for action items, todo items, things to follow up on, etc.
• taking notes to allow for better focus
• writing well structured emails
• writing blog posts and books

We'll look at: `from . import module` and `from .module import something`

and also:  `import package` to access the external API from with the package.

Why would we use `import package` if `from . import api` would work fine?

In this episode, we discuss two options:

• Installing the code under test as a pip installable package with `pip install -e /path/to/local/package`.
• Using the pythonpath pytest setting.

Even if you never get a chance to go to PyCon, I hope this interview helps you get a feel for the welcoming aspect of the Python community.

I recorded this interview as an episode for one of my other podcasts, Python People. But I think it's got some great pre-conference advice, so I'm sharing it here on Python Test as well.

We talk about: 
- Juggling at PyCon
- How to get the most out of PyCon
    - Watching talks
    - Hallway track
    - Open spaces
    - Lightening talks
    - Expo hall / vendor space
    - Poster sessions
    - Job fair
    - A welcoming community
    - Tutorials 
    - Sprints
    - But mostly about the people of Python and PyCon.

"Python enables smart people to work faster" - Rob Ludwick

Some of the topics discussed:

• Building Django applications
• SaaS Pegasus
• placecard.me
• What boilerplate projects are
• Django cookiecutter
• Cookiecutter
• Which database to use, probably PostgreSQL
• Authentication choises, probably Allauth
• Docker, Docker for development, Docker for deployment
• Deployment targets / hosting services. Render, Heroku, Fly.io, for PaaS options.
• Front end frameworks. Bootstrap, Tailwind, DaisyUI, TailwindUI
• HTMX vs React vs straight Django templates
• Rockets
• Font Awesome
• and of course, SaaS Pegasus

We talk about:

• Balancing schedule vs testing
• How much testing is the right about of testing
• Should code coverage be measured and tracked
• Good refactoring can reduce code coverage
• Is it worth testing error conditions?
• Are rare error codes ok to just monitor?
• API drift and autospec
• Mitigating risk
• Deciding what to test and what not to test
• Focus testing on key money-making features 
• If there's a bug in this part of the code, how much business impact is there?
• Performance testing needs to approximately match real world workloads
• Cost of a service breaking vs the cost of creating, maintaining, and running tests
• Keeping test suites quick to minimize getting distracted

Links:

• Too much of a good thing: the trade-off we make with tests 
• Load testing is hard, and the tools are... not great. But why?
• Yet Another Rust Resource (YARR!)
• Goodhart's law - "When a measure becomes a target, it ceases to be a good measure"

Topics:

• Podcasting
• Building new SaaS (software as a service) products
• Balancing work, side hustle, and family
• Great places to snowboard in British Columbia

BTW.

Links from the show:

• Transistor.fm - excellent podcast hosting, Justin is a co-founder
• How to start a podcast in 2024
• Podcasts from Justin
  • Build your SaaS - current
  • Build & Launch - an older one, but great
  • MegaMaker - from 2021 / 2022

• Ruff can take the place of Flake8, isort, and Black, and so much more.
• uv can take the place of pip, pip-tools, and virtualenv
• Astral is Charlie's venture backed company, and what they have with `ruff` and `uv` is just the start.

Since uv is the newest tool, there's quite a bit of the discussion diving into uv.

Links:

• ruff
• Astral
• uv

Links from the episode:

• Blog post announcing the rewrite  
• Code repo for questions, comments, issues etc

• pytest-repeat
• pytest-rerunfailures
• pytest-flakefinder
• pytest-instafail

We talk about each of these in this episode.

Brian's commentary is saved for a followup episode.

Links:

• Canon TDD
• Test Driven Development by Example

In this episode, we'll talk about:

• Causes of dependence
• Testing for dependencies using plugins
• Debugging test dependencies

Plugins discussed:

• pytest-randomly
• pytest-reverse
• pytest-random-order

• Do we have to do the refactor part? 
• Does the refactor at the end include tests? 
• Or can I refactor the tests at any time?
• Why is refactor at the end? 

This episode is to talk about this with a an example.

This episode covers:

• Design for Test: Structuring your app or script so it's easier to test.
• pytest & capsys for testing stdout
• Adding debug and preview flags for debugging and testing
• And reverting to subprocess.run if you can't modify the code under test

Also, there's a full writeup and code samples available:

• Blog post: Testing argparse Applications 
• Code Repo

In this episode, we walk through a handful of useful examples of test code without asserts.
We also talk about how these types of tests are a great way to dip your toe into testing.

However, software tools and available CI systems have changed quite a bit since then.
Maybe it's time to re-examine the assumptions, practices, processes, and principles of TDD.
 
At least in the context of my software engineering career, modifications to TDD, at least the version of TDD as it's frequently taught, have been necessary.

This is the start of a series focused on examining TDD and related lightweight practices and processes.

Links from the show:

• From XP
  • Test First
  • Unit Tests
  • Acceptance Tests
• Test-Driven Development (wikipedia)

In this episode, we describe:

• what fixtures are
• what autouse fixtures are
• great reasons to use them

Johanna's book: Free Your Inner Nonfiction Writer

Joe Curley, vice president and general manager of software products and ecosystem, and Arun Gupta, vice president and general manager for open ecosystems, join the show to discuss open source, OneAPI, and open ecosystems at Intel.

At the top of the towncrier documentation, it says "towncrier is a utility to produce useful, summarized news files (also known as changelogs) for your project."

Towncrier is used by "Twisted, pytest, pip, BuildBot, and attrs, among others."

This is the last of 3 episodes focused on keeping a CHANGELOG. 

Episode 200 kicked off the series with keepachangelog.com and Olivier Lacan
In 201 we had Ned Batchelder discussing scriv.

Special Guest: Hynek Schlawack.

Links:

• Towncrier docs
• How to Keep a Changelog in Markdown - Towncrier docs
• Keep a Changelog
• structlog/CHANGELOG.md — Example of manually edited changelog.
• hatch-fancy-pypi-readme
• MyST Markdown 
• hatchling

Scriv "is a command-line tool for helping developers maintain useful changelogs. It manages a directory of changelog fragments. It aggregates them into entries in a CHANGELOG file."

Links:

• nedbat/scriv: Changelog management tool

Special Guest: Olivier Lacan.

Links:

• Shields.io
• Keep a Changelog
• "The Changelog" Podcast interview with Olivier Lacan
• An Open Source Rage Diamond

Pamela Fox, a Cloud Advocate for Python at Microsoft, joins the show to help us with that question.

Links:

• Python Bytes Episode #323
• pamelafox gitHub projects
• Deploy a Python (Django or Flask) web app to Azure
• Hosting Python Web Apps on Azure: A Price-Off
• Deploying a containerized FastAPI app to Azure Container Apps
• Getting started with hosting Python apps on Azure
• Pricing Calculator for Azure
• App Service Pricing 
• pamelafox/flask-db-quiz-example
• Tutorial: Deploy a Python Django or Flask web app with PostgreSQL
• Pamela Fox on Mastodon

 Classifiers are weird.

 They were around in setuptools days, and are still here with pyproject.toml. 

• What are they? 
• Why do we need them? 
• Do we need them?
• Which classifiers should I include?
• Why are they called "trove classifiers" in the Python docs

Brett Cannon joins the show to discuss these wacky bits of metadata.

Here's an example, from pytest-crayons:

Links:

• Classifiers · PyPI
• PEP 621 – Storing project metadata in pyproject.toml | peps.python.org
• Packaging Python Projects — Python Packaging User Guide — Configuring metadata
• PEP 639 – Improving License Clarity with Better Package Metadata | peps.python.org
• SPDX

 There are problems with supply chain analogy when applied to software.

 Thomas Depierre discusses some of those issues in this episode. 

Links:

• I am not a supplier - article

Anthony's list:

1. The import system
2. Multi-process support out of the box
3. Async support
4. Changes to the fixture system
5. Extend the assert rewriting to make it modular
6. Add matchers to assert mechanism
7. Ban test class inheritance

Brian's list: 

1. Extend assert rewriting for custom rewriting, like check
2. pytester matchers available for all tests
3. Throw out nose and unittest compatibility plugins
4. Throw out setup_module, teardown_module and other xunit style functions
5. Remove a bunch of the hook functions
6. Documentation improvement of remaining hook functions which include examples of how to use it
7. Start running tests before collection is done
8. Split collection and running into two processes
9. Have the fixtures be able to know the result of the test during teardown

Links:

• anthonywritescode - YouTube
• anthonywritescode - Twitch
• pytest-asyncio · PyPI
• async test patterns for pytest
• future-fstrings · PyPI
• re-assert · PyPI
• numpy.testing
• Sourcegraph

But how good is that software? 

How much of common best practices in software development are making it to those writing software in the sciences?

Patrick Mineault has written "The Good Research Code Handbook". 
 It's a website. It's concise. 
 And it will put you on the right path to writing better software.
 Even if you don't write science based software, and even if you already have a CS degree, there's some good information worth reading.

Special Guest: Patrick Mineault.

Links:

• The Good Research Code Handbook
• game-wrath-jam: A game jam game, theme: Wrath
• Robotron 2084 - Arcade - YouTube
• The Book of Why: The New Science of Cause and Effect

Learning to code a game is a way that a lot of people get started and excited about programming. 

Of course, I don't recommend Basic. Now we've got Python. And one of the game engines available for Python is PursuedPyBear, a project started by Piper Thunstrom. 

Piper joins us this episode and we talk about PursuedPyBear, learning to code, and learning CS concepts with game development. 

PursuedPyBear, ppb, is a game framework great for learning with, with goals of being fun, education friendly, an example of idiomatic Python, hardware library agnostic, and built on event driven and object oriented concepts.

Special Guest: Piper Thunstrom.

Links:

• PursuedPyBear | Unbearably Fun Game Development
• Piper's Blog
• Making Games With PPB - PyTexas
• Shooter Game by Piper Thunstrom
• shootergame on GitHub
• Briefcase— BeeWare
• game-blink: A tiny emergent behavior toy.
• Combat (Atari 2600) — The tank game I didn't remember the name of.
• Lunar Lander

Links:

• Random Geekery
• Jamstack
• Eleventy
• Netlify
• Plausible Analytics
• pytest
• Beautiful Soup
• pyinvoke - Invoke!
• rsync
• Internet Archive : archive.org
• Rich
• Statamic
• jamstack.org
• A static site generator should be your next language learning project

 PyPy is implemented in Python.
 What does that mean?

 And how do you test something as huge as an alternative implementation of Python?

Special Guest: Carl Friedrich Bolz-Tereick.

Links:

• PyPy
• How is PyPy Tested? 
• PyPy Speed
• Python Speed Center

The story of attrs and dataclasses is actually intertwined. 
 They've built on each other.
 And in the middle of it all, Hynek.

Hynek joins the show today to discuss some history of attrs and dataclasses, and some differences.

If you ever need to create a custom class in Python, you should listen to this episode.

Links:

• attrs documentation
• History of attrs and introduction to attrs namespace
• cattrs: Complex custom class converters for attrs. — python-attrs
• PEP 557 – Data Classes
• PEP 681 – Data Class Transforms

Links:

• rich
• rich-cli
• textual
• Textualize.io
• Rich Gallery
• Textualize Gallery
• Python Bytes Podcast

Karl Stolley believes it's never too early. Let's hear how he incorporates code quality in his courses.

Our discussion includes:

• starting people off with good dev practices and tools
• linting
• html and css validation
• visual regression testing
• using local dev servers, including https
• incorporating testing with git hooks
• testing to aid in css optimization and refactoring
• Backstop
• Nightwatch
• BrowserStack
• the tree legged stool of learning and progressing as a developer: testing, version control, and documentation

Karl is also writing a book on WebRTC, so we jump into that a bit too.

Links:

• Backstop
• Nightwatch
• BrowserStack
• Programming WebRTC: Build Real-Time Streaming Applications for the Web by Karl Stolley

Nick Hodges joins the show to discuss the good and the bad of developer and team productivity, including how we can improve productivity.

Links:

• django-rich · PyPI
• episode 181: Boost Your Django DX - Adam Johnson
• django-crispy-forms
• Python Developers Survey 2020 Results 

Twisted uses a technique that should be usable by other applications, even those using asyncio or other event driven architectures.

Links:

• Community Service Award Recipient Glyph Lefkowitz — The PSF article I was referring to early in the episode 
• Twisted
• twisted/klein
• twisted/treq
• django/daphne — Django Channels HTTP/WebSocket server
• Scrapy
• So Easy You Can Even Do It in JavaScript: Event-Driven Architecture for Regular Programmers - YouTube
• Feature test - twitter engineering
• Python Testing with pytest — Some reading for Glyph if he wants to learn to love pytest
• pytest-twisted
• pygame
• Pyodide
• Brython

Some topics discussed:

• handling code reviews
• asking good questions
• being honest about what you can't do with current resources and data
• discussing tradeoffs and offering solutions that can be completed faster than the ideal solution
• balancing engineering and managing
• making sure documentation happens
• remote teams 
  • encouraging collaboration
  • encouraging non-work-related conversations
  • watching out for overworking

No? Neither does Nathan Aschbacher. 
 So when he started building the team at his company, he decided to do things differently.

Hiring is one of the essential processes for building a great team.
 However, it's a high noise, low signal process.

Nathan Aschbacher has a relatively unorthodox tech hiring approach. 
 He's trying to make it very humane, with a better signal to noise ratio. 

Nathan is not intereseted in bizarre interview processes where the interviewer doesn't know anything about the interviewee beforehand, all people are asked the same questions, and people are asked to code on white boards.

Instead, he states "if the goal is to try to figure out if the person can do the work with your team, and your trying to build the team that you are adding this person to, they need to know what the team is like, and determine if they want to be part of the team, and the team needs to know what the person is like and if they would be addititve to the team.

So what's Nathan's process:

• Screening resumes and CVs, looking for internal motivation to become an expert at something.
• Basic phone screen, very informal.
• A couple 2-3 hour pairings with someone on the team with whatever they are working on.
• Debriefing both the candidate and the team afterwords.
• Giving the candidate an opportunity for a second impression and following up on difficulties during the pairings.

We discuss the process, and also:

• trying to remove the barriers to team integration 
• treating people as humans 

And of course, there's the story of how Nathan ended up interviewing someone with Zoo experience an no technical experience for a technical role. Of course, it was a misunderstanding of a job requirement around experience with ZooKeeper. But it's a good story.

Developer experience includes tools and practices to make developers more effective and efficient, and just plain make software development more fun and satisfying. 
 One of the things I love about this book is that it's not just for Django devs. 
 I'd guess that about half the book is about topics that all Python developers would find useful, from virtual environments to linters to testing. 
 But of course, also tons of tips and tools for working with Django.

Links:

• “Boost Your Django DX” Released — Adam's announcement
• Boost Your Django DX — The book

I've mentioned Lean TDD on the podcast a few times and even tried to do a quick outline at the end of episode 162.

This episode is a more complete outline, or at least a first draft.

If you feel you've got a good understanding of TDD, and it's working awesome for you, that's great. Keep doing what you're doing. There are no problems.

For me, the normal way TDD is taught just doesn't work. So I'm trying to come up with a spin on some old ideas to make it work for me. I'm hoping it works for you as well.

I'm calling the new thing Lean TDD. It's inspired by decades of experience writing software and influence from dozens of sources, including Pragmatic Programmer, Lean Software Development, Test-Driven Development by Example, and many blog posts and wiki articles. 

The main highlights, however, come from the collision of ideas between Lean and TDD and how I've tried to resolve the seemingly opposing processes.

Links:

• Lean TDD | pythontest.com
• episode 162: Flavors of TDD
• What is Test Driven Development (TDD)? | Agile Alliance
• Test First Programming / Test First Development | pythontest.com
• episode 7: The Waterfall Model and “Managing the Development of Large Software Systems”
• episode 51: Feature Testing
• The testing renaissance — Twitter Engineering blog post about Feature Testing
• Lean software development - Wikipedia

This episode discusses the factors.

Sarah's order:

1. Verify the code is working correctly
2. Prevent future regressions
3. Document the code’s behavior
4. Provide design guidance
5. Support refactoring

Brian's order:

1. Verify the code is working correctly
2. Prevent future regressions
3. Support refactoring
4. Provide design guidance
5. Document the code’s behavior

The episode includes reasons why I've re-ordered them.

Links:

• Five Factor Testing - Sarah Mei

Links:

• Simon Willison's Twitter Thread
• The Testing Trophy and Testing Classifications — Kent C Dodds
• Write tests. Not too many. Mostly integration. — Kent C Dodds
• On the Diverse And Fantastical Shapes of Testing — Martin Fowler

 Solve a problem with software. Have a nice looking landing page and website. Get paying customers.

 Eventually have it make enough revenue so you can turn it into your primary source of income. 

There's a lot of software talent out there. We could solve lots of problems. 
 But going from idea to product to first customer is non-trivial. 
 Especially as a side hustle. 
 This episode discusses some of the hurdles from idea to first customer. 

Brandon Braner is building Released.sh. It's a cool idea, but it's not done yet. 

Brandon and I talk about building side projects:

• finding a target audience
• limiting scope to something doable by one person
• building a great looking landing page
• finding time to work on things
• prioritizing and planning
• learning while building
• even utilizing third party services to allow you to launch faster
• and last, but not least, having fun

Special Guest: Brandon Braner.

Links:

• Released
• Tailwind CSS 
• Tailwind UI
• Figma
• Heroku
• Google App Engine

Answering these questions are the goals of this episode.

Links:

• Test Automation - Who Should be Involved? | Thoughtworks

Some strict forms of TDD include something like this:

• write a failing test that demonstrates a lacking feature or defect
• write the source code to get the test to pass
• refactor if necessary
• repeat

In reality, at least for me, the software development process is way more messy than this, and not so smooth and linear.

Pauls workflow allow you to develop non-linearly, but commit cleanly.

Links:

• A pseudo-TDD workflow using expected failures
• episode 171: How and why I use pytest's xfail - Paul Ganssle
• episode 165: pytest xfail policy and workflow
• episode 162: Flavors of TDD

unittest is an xUnit style framework. For me, xUnit style frameworks are fatally flawed for software testing.

That's what this episode is about, my opinion of 

• "Why NOT unittest?", or more broadly, 
• "What are the fatal flaws of xUnit?"

Links:

• Python Testing with pytest, Second Edition
• unittest docs
• unittest assert methods
• xUnit - Wikipedia

Links:

• Selecting a programming language can be a form of premature optimization — Brett Cannon's blog post

Special Guest: Paul Ganssle.

Links:

• How and why I use pytest's xfail — Paul's blog post mentioned in the episode
• Craft Minimal Bug Reports — Matthew Rocklin's article
• episode 111: Subtests in Python with unittest and pytest - Paul Ganssle
• episode 165: pytest xfail policy and workflow
• episode 166: unittest expectedFailure and xfail

Prayson is using pytest for some pretty cool stuff, including:

• unit tests, of course
• testing pipeline stages
• counterfactual testing
• performance testing

All with pytest. So cool.

Special Guest: Prayson Daniel.

Links:

• Python Bytes 250, with Prayson Daniel — Listen to this for more of an introduction to Prayson

Special Guest: Omri Sass.

Special Guest: Paul Everitt.

Links:

• React, TypeScript, and TDD — Paul Everitt's tutorial
• React Testing Library

In this episode, we cover:

• using @unittest.expectedFailure
• the results of passing and failing tests with expectedFailure
• using pytest as a test runner for unittest
• using pytest markers on unittest tests

Docs for expectedFailure:
 https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failures

Some sample code. 
 unittest only:

unittest with pytest markers:

The episode covers:

• What is xfail
• Why I use it
• Using reason effectively by including issue tracking numbers
• Using xfail_strict
• Adding --runxfail when transitioning from development to feature freeze
• What to do about test failures
• How all of this might help with team communication

pytest includes quite a few command-line flags that are useful for debugging. 

We talk about thes flags in this episode.

Flags for selecting which tests to run, in which order, and when to stop:

• -lf / --last-failed: Runs just the tests that failed last.
• -ff / --failed-failed: Runs all the tests, starting with the last failed.
• -x / --exitfirst: Stops the tests session afterEd: after?Author: yep the first failure.
• --maxfail=num: Stops the tests after num failures.
• -nf / --new-first: Runs all the tests, ordered by file modification time.
• --sw / --stepwise: Stops the tests at the first failure. Starts the tests at the last failure next time.
• --sw-skip / --stepwise-skip: Same as --sw, but skips the first failure.

Flags to control pytest output:

• -v / --verbose Displays all the test names, passing or failing.
• --tb=[auto/long/short/line/native/no] Controls the traceback style.
• -l / --showlocals Displays local variables alongside the stacktrace.

Flags to start a command-line debugger:

• --pdb Starts an interactive debugging session at the point of failure.
• --trace Starts the pdb source-code debugger immediately when running each test.
• --pdbcls Uses alternatives to pdb, such as IPython’s debugger with –-pdbcls=IPython.terminal.debugger:TerminalPdb.

This list is also found in Chapter 13 of Python Testing with pytest, 2nd edition.
The chapter is "Debugging Test Failures" and covers way more than just debug flags, while walking through debugging 2 test failures.

Links:

• Python Testing with pytest — The fastest way to get up to speed on pytest.
• all pytest flags in pytest 6.2.x

You can also use pip to install a package in a local directory.
 That's the part I want to jump in and explore with Stéphane Bidoul.
 The way pip installs from a local directory is about to change, and the story is fascinating.

Special Guest: Stéphane Bidoul.

Links:

• The Odoo Community Association
• PEP 610 -- Recording the Direct URL Origin of installed distributions | Python.org
• PEP 660 -- Editable installs for pyproject.toml based builds (wheel based) | Python.org — Bidoul
• pip install --no-index --find-links 
• Solving issues related to out-of-tree builds · Issue #7555 · pypa/pip
• pip list json format

In this episode we talk about:

• Classical vs Mockist TDD
• Detroit vs London (I actually refer to it in the episode as Chicago instead of Detroit. Oh well.)
• Static vs Behavior
• Inside Out vs Outside In
• Double Loop TDD
• BDD
• FDD
• Tracer Bullets
• Rules of TDD
• Team Structure
• Lean TDD 

This is definitely an episode I'd like feedback on. Reach out to me for further questions or if I missed some crucial variant of TDD that you know and love.

Links:

• Mocks Aren't Stubs - Martin Fowler
• Mockists Are Dead. Long Live Classicists.
• Double Loop TDD
• BDD: Behavior-driven development
• FDD: Feature-driven development
• My reaction to “Is TDD Dead?” - pythontest.com
• Test First Programming / Test First Development
• Humorous list of TDD variants — BDD = Buzzword Driven Development, CDD = Calendar Driven Development, etc

I'm going to use the term waste and value in future episodes. I'm using waste in a Lean sense, so we can look at software processes critically, see the value chain, and try to reduce waste.

Lean manufacturing and lean software development caused people to talk about and examine waste and value, even in fields where we didn't really think about waste that much to begin with.

Software is just ones and zeros. Is there waste? 
 When I delete a file, nothing goes into the landfill.

The mistake I'm making here is confusing the common English definition of waste when what we're talking about is the Lean definition of waste.

This episode tries to clear up the confusion.

Links:

• Big Design Up Front
• Lightweight Methodologies
• Manifesto for Agile Software Development
• Extreme programming
• The New Methodology
• Test First Programming / Test First Development
• Test Driven Development
• The Pragmatic Programmer
• Six Sigma
• DMAIC
• Lean software development
• Lean manufacturing
• The Toyota Way
• Lean Six Sigma
• Definition of Waste by Merriam-Webster

We'll get to all of these definitions, and then talk about how it applies to both production code and test code in this episode. 

Links:

• The Pragmatic Programmer, 20th Anniversary Edition
• Don't repeat yourself - Wikipedia
• a-ha - Take On Me
• Rule of three - Wikipedia
• What does “DAMP not DRY” mean when talking about unit tests? - Stack Overflow

In this episode we talk about:

• Looking at your own Twitter analytics data.
• Using Python, pandas, Jupyter for data cleaning and exploratory analysis
• Data visualization
• Machine learning, principal component analysis, clustering
• Model drift and re-running analysis
• What kind of tweets perform well
• And much more

Links:

• Applied Pandas: Twitter Analytics — the course

All of us that care about developing quality code with the help of testing can learn from each other, regardless of programming language.

Today we step outside our normal Python comfort zone and talk with Gio about TDD in Swift.

Gio Lodi, author of TDD in Swift, joins the show to discuss Test Driven Development, software workflows, bottom up vs top down, rapid feedback, developer vs customer facing tests, and more.

Links:

• TDD in Swift with SwiftUI and Combine
• mokacoding - Gio's blog
• XCTest
• Software Design: Tidy First? - Kent Beck on Substack

Anthony Sottile discusses pre-commit, for using locally by developers, and pre-commit.ci, which can run actions during merge requests.

"Git hook scripts are useful for identifying simple issues before submission to code review. We run our hooks on every commit to automatically point out issues in code such as missing semicolons, trailing whitespace, and debug statements. By pointing these issues out before code review, this allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks." - pre-commit.com

"Developers spend a fair chunk of time during their development flow fixing relatively trivial problems in their code. pre-commit.ci both enforces that these issues are discovered, which is opt in for each developer workflow via pre-commit, but also fixes the issues automatically, letting developers focus their time on more valuable problems." - A user of pre-commit.ci

Links:

• pre-commit
• pre-commit supported hooks
• pre-commit.ci

But what does all of that mean?

Anthony Sottile is a maintainer of flake8 and has kindly offered to explain it to us.

Links:

• Flake8: Your Tool For Style Guide Enforcement
• pyflakes
• pycodestyle
• mccabe
• pyflakes/messages.py
• flake8 Error / Violation Codes
• wemake-python-styleguide
• flake8-typing-imports
• tox automation project
• Nox
• pre-commit

They are actually awesome questions to ask during a software project.

We discuss the questions, where they came from, and look at some uses in software.

The questions:

1. What is it that needs to be done?
2. What do we need in order to do it?
3. Where can we get what we need?
4. How can we tell if we’re doing it right?

Bonus question that can be swapped out for #1:

1. What's the most important thing that it doesn't do yet?

Links:

• The Four Questions

In this episode, we talk with Jeff about testing web applications, specifically Django apps, and of course talk about the downsides of database mocking.

Links:

• "🤔 In ~12 years, I have never seen a case where mocking a database was a good idea. ..." 
• Django News
• Moto
• Test & Code 103: Django - Lacey Williams Henschel
• Model Bakery: Smart fixtures for better tests 
• Faker

Links:

• Playwright for Python
• Why Playwright?
• playwright-pytest

Links:

• pytest-srcpaths
• Python Packaging User Guide
• pypa/packaging: Core utilities for Python packages
• flit and requires-python
• Extending and Embedding the Python Interpreter — Python 3.9.2 documentation
• Flit
• setuptools
• PEP 518 -- Specifying Minimum Build System Requirements for Python Projects | Python.org
• PEP 517 -- A build-system independent format for source trees | Python.org
• Flit editable installs with --symlink or --pth-file
• PEP 440 -- Version Identification and Dependency Specification | Python.org
• Specifying requires-python with flit
• wheel
• Universal 1 binary
• Universal 2 binary
• Cython: C-Extensions for Python
• CFFI
• pybind11: Seamless operability between C++11 and Python
• Conda
• conda environments
• conda-forge
• VaultsOfParnassus

What's happening is that I'm playing adventure, which you can pip install thanks to Brandon Rohdes. Adventure is a faithful port to Python 3 from the original 1977 FORTRAN code by Crowther and Woods that lets you explore Colossal Cave, where others have found fortunes in treasure and gold, ...

In this episode, we talk with Brandon Rhodes about this marvelous game.

Links:

• adventure · PyPI
• python-adventure: Original Colossal Caves adventure game, but in Python 3
• Brandon Rhodes Personal Site
• Python Design Patterns
• pyephem: Scientific-grade astronomy routines for Python
• python-skyfield: Elegant astronomy for Python
• Adventure in Python 3 - announcement blog post from 2012
• NetHack
• FTL: Faster Than Light
• PEP 517 -- A build-system independent format for source trees | Python.org

Reason for the interview. Rough summary of a twitter conversation:

• Corey: What podcasts should I try to get an invite onto?
• ToonArmyCaptain: Python Bytes, Test & Code, Talk Python
• Corey: But... I don't test my code, "crappy Python" is all I write, and I'd feel like a giant imposter. So yeah, I'd be game. link
• So here we are. 

This diagram is referenced in the show, the Last Week In AWS Newsletter Production Pipeline.

Special Guest: Corey Quinn.

Links:

• Last Week in AWS

Also, a couple followups from last week's episode on using coverage for single file applications. 

Links:

• episode 147: Testing Single File Python Applications/Scripts with pytest and coverage
• Specifying source files — Coverage.py documentation
• Testing & Packaging - Hynek
• ack

This is the topic of this weeks episode, spurred on by a listener question.

The questions:

• For single file scripts, I'd like to have the test code included right there in the file. Can I do that with pytest?
• If I can, can I use code coverage on it?

The example code discussed in the episode: script.py

Suggestion by @cfbolz if you need to import pytest: