I think I found a bug in python’s unittest.mock library

Share

Mocking is a pretty common thing to do in unit tests covering OpenStack Nova code. Over the years we’ve used various mock libraries to do that, with the flavor de jour being unittest.mock. I must say that I strongly prefer unittest.mock to the old mox code we used to write, but I think I just accidentally found a fairly big bug.

The problem is that python mocks are magical. Its an object where you can call any method name, and the mock will happily pretend it has that method, and return None. You can then later ask what “methods” were called on the mock.

However, you use the same mock object later to make assertions about what was called. Herein is the problem — the mock object doesn’t know if you’re the code under test, or the code that’s making assertions. So, if you fat finger the assertion in your test code, the assertion will just quietly map to a non-existent method which returns None, and your code will pass.

Here’s an example:

#!/usr/bin/python3

from unittest import mock

class foo(object):
    def dummy(a, b):
        return a + b

@mock.patch.object(foo, 'dummy')
def call_dummy(mock_dummy):
    f = foo()
    f.dummy(1, 2)

    print('Asserting a call should work if the call was made')
    mock_dummy.assert_has_calls([mock.call(1, 2)])
    print('Assertion for expected call passed')

    print()
    print('Asserting a call should raise an exception if the call wasn\'t made')
    mock_worked = False
    try:
        mock_dummy.assert_has_calls([mock.call(3, 4)])
    except AssertionError as e:
        mock_worked = True
        print('Expected failure, %s' % e)

    if not mock_worked:
        print('*** Assertion should have failed ***')

    print()
    print('Asserting a call where the assertion has a typo should fail, but '
          'doesn\'t')
    mock_worked = False
    try:
        mock_dummy.typo_assert_has_calls([mock.call(3, 4)])
    except AssertionError as e:
        mock_worked = True
        print('Expected failure, %s' % e)
        print()

    if not mock_worked:
        print('*** Assertion should have failed ***')
        print(mock_dummy.mock_calls)
        print()

if __name__ == '__main__':
    call_dummy()

If I run that code, I get this:

$ python3 mock_assert_errors.py
Asserting a call should work if the call was made
Assertion for expected call passed

Asserting a call should raise an exception if the call wasn't made
Expected failure, Calls not found.
Expected: [call(3, 4)]
Actual: [call(1, 2)]

Asserting a call where the assertion has a typo should fail, but doesn't
*** Assertion should have failed ***
[call(1, 2), call.typo_assert_has_calls([call(3, 4)])]

So, we should have been told that typo_assert_has_calls isn’t a thing, but we didn’t notice because it silently failed. I discovered this when I noticed an assertion with a (smaller than this) typo in its call in a code review yesterday.

I don’t really have a solution to this right now (I’m home sick and not thinking straight), but it would be interesting to see what other people think.

Share

Starfish Prime

Share

As for Starfish Prime, because there is almost no air at an altitude of 400 kilometres, no fireball formation occurred, although manifold other notable effects did occur. About 1500 kilometres (930 statute miles) away in Hawaii, the Electromagnetic pulse created by the explosion was felt as three hundred street lights failed, television sets and radios malfunctioned, burglar alarms went off and power lines fused. On Kauai, EMP shut down telephone calls to the other islands by burning out the equipment used in a microwave link. Also, the sky in the Pacific region was illuminated by an artificial aurora for more than seven minutes. In part, these effects were predicted by Nicholas Christofilos, a scientist who had earlier worked on the Operation Argus high-altitude nuclear shots.

High altitude nuclear testing leads to interesting EMP effects, which is something I have known about for a while. There is good coverage of Starfish Prime at Wikipedia and some US government hearings on the issue are interesting too.

Wow. Did they really break a bunch of satellites?

While some of the energetic beta particles had followed of the earth’s magnetic field and illuminated the sky, other high-energy electrons became trapped in man-made radiation belts around the earth. There was much uncertainty and debate about the composition, magnitude, and potential adverse effects from this trapped radiation after the detonation. The weaponeers became quite worried when three satellites in low earth orbit were disabled. These man-made radiation belts eventually crippled one-third of all satellites in low orbit. Seven satellites were destroyed as radiation knocked out their solar arrays or electronics, including the first commercial communication satellite ever, Telstar.

Those are from the Wikipedia page. There is also another Wikipedia page on high altitude nuclear testing which discusses Russian efforts as well, including this gem:

The Soviets detonated four high-altitude tests in 1961 and three in 1962. During the Cuban Missile Crisis in October 1962, both the US and the USSR detonated several high-altitude nuclear explosions as a form of sabre-rattling. The Soviet tests were meant to demonstrate their anti-ballistic missile defences which would supposedly protect their major cities in the event of a nuclear war. The worst effects of a Russian high altitude test occurred on 22 October 1962 (during the Cuban missile crisis), in ‘Operation K’ (ABM System A proof tests) when a 300-kt missile-warhead detonated near Dzhezkazgan at 290-km altitude. The EMP fused 570 km of overhead telephone line with a measured current of 2,500 A, started a fire that burned down the Karaganda power plant, and shut down 1,000-km of shallow-buried power cables between Aqmola and Almaty [5]. The Partial Test Ban Treaty was passed the following year, ending atmospheric and exoatmospheric nuclear tests.

Share