Useful Python Tools

I often stumble across and use a number of useful tools for creating Python code.  Thought I'd barf out a blog post documenting a few of them so that my future self will be able to find this info again if need be. :)

coverage.py 

(http://nedbatchelder.com/code/coverage/)

Coverage.py is a Python code coverage tool and is useful for finding out how well your unit tests cover your code.  I've often had it find big deficiencies in my unit test coverage.  Common usage:

$ coverage run somemodule_test.py
$ coverage report -m

Will spit out a coverage report for the tests in somemodule_test.py.  Used in this way, coverage.py isn't particularly handy, but combined with a good unit test runner (see below) it becomes very handy.

Nose 

(http://readthedocs.org/docs/nose/en/latest/)

Is nicer testing for Python.  Nose is an extremely handy unittest runner that has some perks over the standard Python unittest module.  Continuing from the last tool, nose also integrates very nicely with coverage.py.  I commonly use it to produce some nice HTML pages summarzing test coverage for my project:

$ nosetests --with-coverage --cover-inclusive --cover-html --cover-erase

produces a "cover" directory containing an index.html with some nice pretty HTML reports telling me how well my unit tests cover my codebase.

pymetrics 

(http://sourceforge.net/projects/pymetrics/)

pymetrics is a handy tool for spitting out some well, metrics, about your code.  Ex:

$ pymetrics somemodule.py

Spits out a bunch of numbers about somemodule.py including trivial things like how many methods have docstrings, to more interesting things like the McCabe cyclomatic complexity of each method/function within the module.  Handy.

cloc 

(http://cloc.sourceforge.net/)

Is a simple "lines of code" counter that happens to support Python.  In the top directory of a project a:

$ cloc .

will give you summary output for your project like:

-------------------------------------------------------------
Language  files          blank       comment           code
-------------------------------------------------------------
Python       31           3454          9215          14775
-------------------------------------------------------------
SUM:         31           3454          9215          14775
-------------------------------------------------------------

While LOC is generally a meaningless statistic, it can be handy for getting a "ballpark" idea of how big a project is.

The heirarchy of Mystical Arts in Programming

I was responding to an incredibly detailed answer on StackOverflow on Python metaclasses, when I wrote the following on my whiteboard:

I thought this was clever, and gave the (I thought clever) response on StackOverflow which read:

I read this and think of the famous "There are lies, damned lies, and then there is statistics", but instead think of it as "there are hacks, tricks, voodoo magic, dark arts, and then there are Python metaclasses".

This was good, I then left to go for lunch and when I came back a co-worker had modified/added to my list:

Python HTMLParser and super()

So I have a class that inherits from HTMLParser, and I want to call the super class init (the __init__ of HTMLParser), I would think I should do:

class MyParser(HTMLParser):
def __init__(self):
super(MyParser, self).__init__()

But this causes a problem:

myparser = MyParser()
Traceback (most recent call last):
File "", line 1, in
File "", line 3, in __init__
TypeError: must be type, not classobj

What's with that? The super(class, instance).__init__ idiom is the supposed proper way of calling a parent class constructor, and it is -- if the class is a "new-style" Python class (one which inherits from object, or a class which inherits from object).

And therein is the problem: HTMLParser inherits from markupbase.ParserBase, and markupbase.ParserBase is defined as:

class ParserBase:
"""Parser base class which provides some common support methods used
by the SGML/HTML and XHTML parsers."""

That is, as an *old* style class. One definitely wonders why in Python 2.7+ the classes that form part of the standard library wouldn't all be new-style classes, *especially* when the class is intended as being something you inherit from (like HTMLParser). Anywho, to fix:

class MyParser(HTMLParser):
def __init__(self):
# Old style way of doing super()
HTMLParser.__init__(self)