How not to write Python unit tests

In Openstack projects, I see tests like this, entirely too often:

    def test_main_sync_success(self):
        self._test_main_sync()
        self.cmd_sync.sync_address_sets.assert_called_once_with(mock.ANY)
        self.cmd_sync.sync_networks_ports_and_dhcp_opts.\
            assert_called_once_with(mock.ANY)
        self.cmd_sync.sync_acls.assert_called_once_with(mock.ANY)
        self.cmd_sync.sync_routers_and_rports.assert_called_once_with(mock.ANY)
        self.cmd_log.info.assert_called_with('Sync completed')

In the name of 100% unit test coverage, tests are written that just assert that a bunch of functions are called. It doesn't matter that any of those functions could be broken. Or stupid. Or just not needed. The test just makes sure that those functions are called.

Consider the following horrible implementation of an add() function:

def add(x, y):  
    val = 0
    for i in range(x + y):
        val += 1
    return val

This, like a lot of code that makes it into production, is stupid. The only thing that could be worse would be a test that relies on the stupidity:

import mock, unittest  
from adder import add

class TestAdd(unittest.TestCase):  
    def test_add(self):
        x, y = 1, 3
        # for the sake of example, assume range is an external that has to be mocked
        with mock.patch.object(add, 'range') as mock_range:
            add.add(x, y)
            mock_range.assert_called_once_with(x + y)

The truly crazy thing is that I see tests like this all the time. The tests just blindly call assert_called_once_with() over several methods called by the function under test. The internal details of add() are stupid and should change, but now when the function is fixed, the test also has to change. Congrats, you've tested absolutely nothing and created twice the work to fix the problem.

Any time you feel like calling Mock's assert_called* methods, please make sure that you aren't tying the test to a specific implementation. If you can't write a unit test without it, maybe the test you are writing shouldn't be a unit test. Write an integration test.