So, I just started an interest on python tests and I would like to share what I learned so far.
Let’s skip all that theory about how tests are important for the code, bla bla bla, if you want to learn test, probably you already have your own reason, so, let’s jump to the part you actually write code!
Our application
Here a simple code that calculates fibonacci. Let’s save it in the file fibonacci.py
def fibonacci(number):
if number < 2:
return number
return fibonacci(number - 1) + fibonacci(number - 2)
So, as you can see, it’s a very simple code that calculates the fibonacci number, so, how can we ensure that this is working? Well, the easiest way is just invoke the code with different values, and check if the result will be the expected. So, in other words, you’re doing exactly what doctest intent to do!
so, probably you would write a python file, let’s say, test_fibonacci.py with the following content:
from fibonacci import fibonacci
def test_fibonacci():
expected_result = [0, 1, 1, 2, 3, 5]
for x in range(0, 5):
result = fibonacci(x)
if result != expected_result[x]:
print "Failing on calculate fibonacci %s" % x
if __name__ == '__main__':
print "Testing fibonacci"
test_fibonacci()
Well, it might work, until you want to calculate fibonacci(200), then you gonna need to add a new list, alter the test code, etc.
What if we now only accept positive numbers and negative numberes throws an exception? Try/catch block? Well, soon you’re writing more code, and adding more complexity than you were expecting. Maybe you gonna need to test your test code itself to ensure that it’s working fine (are you feeling the inception here?)
Doctest
So, doctest can help you with some of these problems! Let’s see, first of all, you’re gonna need to create a simple txt file and add the following content:
>>> from fibonacci import fibonacci >>> fibonacci(0) 0 >>> fibonacci(1) 1 >>> fibonacci(2) 1 >>> fibonacci(3) 2 >>> fibonacci(4) 3 >>> fibonacci(7) 12
So, basically, doctest expect every line starting with “>>>” to be a python code (we are importing fibonacci function in the line number 1 for example. And any other line as the result, looking just like the python shell.
Assuming that both fibonacci.py and fibonacci.txt are in the same directory, you can run the doctest with the command:
$ python -m doctest fibonacci.txt
**********************************************************************
File "fibonacci.txt", line 18, in fibonacci.txt
Failed example:
fibonacci(7)
Expected:
12
Got:
13
**********************************************************************
1 items had failures:
1 of 7 in fibonacci.txt
***Test Failed*** 1 failures.
Oops! Something went wrong! Don’t worry, that was expected. As you can see, the doctest shows a very good output when some error occurs, it shows you the line where the error happened, what was the expected result, and what was the result doctest gots. So, after fixing it, and runing again, we got no error
>>> from fibonacci import fibonacci >>> fibonacci(0) 0 >>> fibonacci(1) 1 >>> fibonacci(2) 1 >>> fibonacci(3) 2 >>> fibonacci(4) 3 >>> fibonacci(7) 13
$ python -m doctest fibonacci.txt
Now let’s go further, and say: Hey, I don’t want negative numbers to be allowed, I want it to throw an exception. So, in this case, we just need to add a new like in our fibonacci.txt testing the fibonacci(-1) and of course, checking if the result will be an exception.
If we would do that in our test_fibonacci.py code, we should use a try/catch block, call the fibonacci(-1) inside it, and of course, this woudln’t be in the initial loop, and if something goes wrong, you wouldn’t have the cool output from doctest.
So, let’s change our fibonacci.py to not accept negative numbers:
def fibonacci(number):
if number == 1 or number == 0:
return number
if number < 0:
raise ValueError('Number is negative')
return fibonacci(number - 1) + fibonacci(number - 2)
Even if we run our doctest after this changes, we got no error, but we are not testing the negative number yet. so, let’s add a fibonacci(-1) in our fibonacci.txt test.
Just a heads up, if we run fibonacci(-1) we will get a exception like this:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "fibonacci.py", line 5, in fibonacci
raise ValueError('Number is negative')
ValueError: Number is negative
And you’re probably thinking that you can copy and paste this error on our fibonacci.txt and it will work, but what if the file can change? You can add more code on the file, and the line where the error is being raised (in this case, line 5) might change. Being so, doctest only cares about the first and last lines, so our code in fibonacci.txt file will look like this:
>>> fibonacci(-1) Traceback (most recent call last): ValueError: Number is negative
And we are done! now we have a set of tests to our fibonacci code!
But what if you don’t want to have a txt and still have your tests? You can write doctest directly in your code on the docstring:
def fibonacci(number):
'''
This is a fibonacci function.
Example of usage:
>>> fibonacci(0)
0
>>> fibonacci(1)
1
>>> fibonacci(2)
1
>>> fibonacci(3)
2
>>> fibonacci(4)
3
>>> fibonacci(7)
13
>>> fibonacci(-1)
Traceback (most recent call last):
ValueError: Number is negative
'''
if number == 1 or number == 0:
return number
if number < 0:
raise ValueError('Number is negative')
return fibonacci(number - 1) + fibonacci(number - 2)
and you can run python -m doctest fibonacci.py.
More
Doctest have more features, as you can see below:
ELLIPISIS
According our fibonacci code, if you pass a negative number, it will raise a ValueError exception with the message ‘Number is negative’. There are some cases, that we don’t need to know which message is returning, we just want to know that an exception is being raised. Another example is when the output is the default __repr__ output, every time is a different memory address and you can’t control it. For this kind of situations, we can use the ELLIPISIS like in the example below.
>>> fibonacci(-1) #doctest: +ELLIPSIS Traceback (most recent call last): ValueError: ...
The line containing #doctest: + ELLIPSIS tells to doctest that the ‘…’ can match anything after ‘ValueError: ‘
SKIP
As the name says, it skips a test. This can be usefull when you know that some test is failing due some know bug, but it’s not fixed yet.
>>> fibonacci(-1) #doctest: +ELLIPSIS +SKIP Traceback (most recent call last): ValueError: ...
As you can see, you can use the two options, ELLIPSIS and SKIP together.
Conclusion
Tests can be fun and easy, and it helps to keep you code working properly in the whole cycle of development.
For more information about doctest, visit doctest website