My experience adding type annotations to a 2.5k-line Python library

The wwvb package for Python has been a focus of my recent hobby-time programming. I've used it as a place to educate myself about the ins and outs of maintaining a Python package. In the past, I used it to learn about using pylint, black & code coverage to improve the quality of Python code. Most recently, I added type annotations through the whole package until mypy --strict was happy with the whole wwvb package and uwwvb module.

The annotations were added in two steps: See pull requests #7 and #8. Together, these PRs contained 320 insertions and 223 deletions across 14 python files, plus 6 insertions in 2 other files related to CI. I did the work during a part of a day, probably under 4 hours of time spent. Since the package currently contains exactly 2500 physical lines of Python code, adding type annotations touched or added over 10% of physical lines!

I had to use a small number of Any type annotations and a few type: ignore comments. It's possible that if I were more experienced, I would have been able to avoid these occasions.

For instance, I had to ignore type annotations related to adafruit_datetime, used by the uwwvb low-resource decoder module, by using a type: ignore comment at the site where it was imported. I also used this technique in a few places where I delegated construction by using return cls(**d).

In argument parsing, which uses the click library, I had one function with some ignored arguments which had some click internal type. I used Any as an alternative to figuring out the correct type. Similarly, in a Tkinter event handler I marked an unused event object as having the Any type.

In a couple of functions related to decoding BCD, I had elected to signal a nonsense BCD value to the caller by using a tuple of False, None for a BCD decoding failure and True, int for success. However, this was difficult to express in the type system, and Optional[int] seemed to work better overall.

Probably related to type annotations being missing for adafruit_datetime, in one place where I compare two datetime objects and return the result of the comparison, I get the diagnostic error: Returning Any from function declared to return "bool". While composing this blog post, I discovered additional typing errors in uwwvb that will be addressed in an upcoming pull request. Interestingly, these "new" typing errors weren't due to bugs in the Python code, but due to incorrect type annotations that mypy couldn't warn about while missing type information about adafruit_datetime. Update, 2021-10-06: These have been resolved in #9.

Ultimately, I don't feel I uncovered any interesting bugs in wwvbpy in this process. The closest I came was discovering a docstring bug, in which the docstring informally described the argument types and got it wrong. I probably wouldn't even have supplied a docstring with this function, except that pylint demanded it. Anyhow, I think that having "100%*" test coverage (*some error cases are marked "pragma no coverage") is probably the reason that I was unable to discover any interesting type-related bugs while adding type annotations. Rather, I made small code restructurings to support typing, being able to rely on my coverage tests for confidence I was not introducing any bugs.

As the line between public and private APIs is ill defined in wwvb, this will end up being a major version in the semver sense, due to how the signature of get_am_bcd changed. However, this function should have been a private API in the first place...

My experience adding type annotations to a 2.5k-line Python library

Tags: