python – The shit is not working

So, these days I was working in a project that uses requests to make some http/https calls:

import requests


URL = 'https://us.api.battle.net/wow/auction/data/{}?locale=en_US'
API = 'myapikey'

def request_with_api(url):
    apikey = {'apikey': API}
    return requests.get(url, params=apikey)


def get_auctions_url(realm):
    url = URL.format(realm)
    r = request_with_api(url)
    if r.status_code == 200:
        data = r.json()
        data_url = data['files'][0]['url']
        return data_url
    return None


def get_auctions(realm):
    url = get_auctions_url(realm)
    if url:
        r = request_with_api(url)
        if r.status_code == 200:
            return r.json()
        else:
            raise Exception('Error in return code: %s' % r.status_code)
    else:
        raise Exception('Auction data not found')


if __name__ == '__main__':
    auctions = get_auctions('medivh')
    print auctions

And as usual, I wanted to have my unit tests, and requests is very tricky, because, you don’t want to keep making requests to the real server, every time you run your unit tests, and here is a couple of reasons:

I’ll depend of internet to access the server all the time
The server has a limit of requests that I don’t want to waste testing
Some servers may block your account if you do too many requests in a small amount of time.

Mocking

So, the easiest way to test would be using mock, and then do something like this:

import unittest
import mock

import wow

from wow import request_with_api

class TestWow(unittest.TestCase):

    @mock.patch('wow.request_with_api')
    def test_get_auctions_url(self, wow_patch):
        wow_patch.return_value.status_code = 200
        wow_patch.return_value.json.return_value = {'files': [
            {'url': ('http://auction-api-us.worldofwarcraft.com/'
                     'auction-data')}
        ]}
        data = wow.get_auctions_url('medivh')
        wow_patch.assert_called_once()
        self.assertEquals(data,
                'http://auction-api-us.worldofwarcraft.com/auction-data')

        wow_patch.reset_mock()
        wow_patch.return_value.status_code = 403
        data = wow.get_auctions_url('medivh')
        wow_patch.assert_called_once()
        self.assertIsNone(data)


if __name__ == '__main__':
    unittest.main()

This looks good, all I have to do is set the mock return values, reset the mock, set the values of the status_code, the return of the r.json() value, and all of this inside our tests. Of course, I could create the setUp method, and configure the values there:

import unittest
import mock

import wow

from wow import request_with_api

class TestWow(unittest.TestCase):
    def setUp(self):
        self.ok_return = ('http://auction-api-us.worldofwarcraft.com/'
                          'auction-data')
        self.json_ok_return = {
            'files': [
                {
                    'url': self.ok_return
                }
            ]
        }

    @mock.patch('wow.request_with_api')
    def test_get_auctions_url(self, wow_patch):
        wow_patch.return_value.status_code = 200
        wow_patch.return_value.json.return_value = self.json_ok_return
        data = wow.get_auctions_url('medivh')
        wow_patch.assert_called_once()
        self.assertEquals(data, self.ok_return)

        wow_patch.reset_mock()

        wow_patch.return_value.status_code = 403
        data = wow.get_auctions_url('medivh')
        wow_patch.assert_called_once()
        self.assertIsNone(data)


if __name__ == '__main__':
    unittest.main()

Now, so far, we are only testing the get_auctions_url method, let’s add the test to get_auctions method, who calls the get_auctions_url method as well:

import unittest
import mock

import wow


class TestWow(unittest.TestCase):
    def setUp(self):
        self.ok_return = ('http://auction-api-us.worldofwarcraft.com/'
                          'auction-data')
        self.json_ok_return = {
            'files': [
                {
                    'url': self.ok_return
                }
            ]
        }

        self.json_auction_data = {
            'content': {
                "realms": [
                    {"name": "Medivh", "slug": "medivh"},
                    {"name": "Exodar", "slug": "exodar"}
                ],
                "auctions": [
                    {
                        "auc": 1283349179, "item": 129158,
                        "owner": "Toiletcow",
                        "ownerRealm": "Medivh", "bid": 8360, "buyout": 8800,
                        "quantity": 1, "timeLeft": "LONG", "rand": 0,
                        "seed": 0, "context": 0
                    }
                ]
            }
        }

    @mock.patch('wow.request_with_api')
    def test_get_auctions_url(self, wow_patch):
        wow_patch.return_value.status_code = 200
        wow_patch.return_value.json.return_value = self.json_ok_return
        data = wow.get_auctions_url('medivh')
        wow_patch.assert_called_once()
        self.assertEquals(data, self.ok_return)

        wow_patch.reset_mock()

        wow_patch.return_value.status_code = 403
        data = wow.get_auctions_url('medivh')
        wow_patch.assert_called_once()
        self.assertIsNone(data)

    @mock.patch('wow.get_auctions_url')
    @mock.patch('wow.request_with_api')
    def test_get_auctions(self, request_with_api_mock,
                          get_auctions_url_mock):
        get_auctions_url_mock.return_value = None
        self.assertRaises(Exception, wow.get_auctions, 'medivh')
        get_auctions_url_mock.assert_called_once()

        get_auctions_url_mock.reset_mock()

        request_with_api_mock.return_value.status_code = 200
        request_with_api_mock.return_value.json.return_value = (
                self.json_auction_data)
        get_auctions_url_mock.return_value = self.ok_return
        get_auctions_url_mock.json.return_value = self.json_auction_data
        data = wow.get_auctions('medivh')
        self.assertDictEqual(data, self.json_auction_data)


if __name__ == '__main__':
    unittest.main()

So, now things are starting to get a little bit messy, here we need to mock both wow.get_auctions_url, and wow.request_with_api, and reset the mocks and assert that the mock is being called once. It works, but still, it’as a mess for the next guy who will check this code.

HTTMock

The coolest part of the HTTMock is that it returns a requests object for you, which means you can call other requests method if you need it without the need of mock these methods. In our example, we only mocked the json() and status_code, but imagine if we had to mock the headers, responses, etc. HTTMock do that for you.

In my opinion, HTTMock keeps your test methods more clean:

import unittest
from httmock import urlmatch, HTTMock

import wow


@urlmatch(netloc=r'(.*\.)?us\.api\.battle\.net.*$')
def auctions_url(url, request):
    not_found = ('http://auction-api-us.worldofwarcraft.com/'
                 'auction-data/notfound')
    found = 'http://auction-api-us.worldofwarcraft.com/auction-data'

    if 'fail' in url.path:
        return {'status_code': 403}

    return {
           'status_code': 200,
           'content':
           {
               'files': [
                   {'url': not_found if 'notfound' in url.path else found}
               ]
           }
       }


class TestAuctions(unittest.TestCase):

    def test_get_auctions_url(self):
        with HTTMock(auctions_url):
            data = wow.get_auctions_url('medivh')
            self.assertEquals(data, ('http://auction-api-us.worldofwarcraft.'
                                     'com/auction-data'))

        with HTTMock(auctions_url):
            data = wow.get_auctions_url('fail')
            self.assertIsNone(data)

if __name__ == '__main__':
    unittest.main()

As you can see, we can set a method with the @urlmatch decorator and all requests that match that particular regular expression will be handle by this method.
Inside that method we also can set some conditionals as I did to test when we have ‘not found’.
Of course, there are other decorators. You can use, for example the @all_requests decorator

import unittest
from httmock import all_requests, HTTMock

import wow


@all_requests
def auctions_url_req(url, request):
    not_found = ('http://auction-api-us.worldofwarcraft.com/'
                 'auction-data/notfound')
    found = 'http://auction-api-us.worldofwarcraft.com/auction-data'
    if 'fail' in url.path:
        return {'status_code': 403}

    return {
           'status_code': 200,
           'content':
           {
               'files': [
                   {'url': not_found if 'notfound' in url.path else found}
               ]
           }
       }


class TestAuctions(unittest.TestCase):

    def test_get_auctions_url(self):
        with HTTMock(auctions_url_req):
            data = wow.get_auctions_url('medivh')
            self.assertEquals(data, ('http://auction-api-us.worldofwarcraft.'
                                     'com/auction-data'))

        with HTTMock(auctions_url_req):
            data = wow.get_auctions_url('fail')
            self.assertIsNone(data)

if __name__ == '__main__':
    unittest.main()

In this case, all the requests you do no matter the URL, will be handled by the auctions_url_req:

import requests

with HTTMock(auctions_url_req):
    data = requests.get('https://www.google.com')
    print(data.status_code)
    print(data.json())

And you get this output:

200
{u'files': [{u'url': u'http://auction-api-us.worldofwarcraft.com/auction-data'}]}

Bottom line

Use or not to use HTTMock is up to you, I believe this can make your tests cleaner, it’s much more easy look to a with HTTMock and say: oh, hey, so, this will return whatever is in the with method, cool!
While using mock only, I have to be like, okay… this object is a mock, and the returned value of this method here is 200, and oh, it’s resetting the mock here, so let’s forget about all of this, and start over with the… well, you get the idea.
And of course, you can use more than one method to handle the requests, as you can see on line 65:

import unittest
from httmock import urlmatch, HTTMock, all_requests

import wow

AUCTION_CONTENT = {
    "realms": [
        {"name": "Medivh", "slug": "medivh"},
        {"name": "Exodar", "slug": "exodar"}
    ],
    "auctions": [
        {
            "auc": 1283349179, "item": 129158,
            "owner": "Toiletcow",
            "ownerRealm": "Medivh", "bid": 8360, "buyout": 8800,
            "quantity": 1, "timeLeft": "LONG", "rand": 0,
            "seed": 0, "context": 0
         }
    ]
}


@urlmatch(netloc=r'(.*\.)?us\.api\.battle\.net.*$')
def auctions_url(url, request):
    not_found = ('http://auction-api-us.worldofwarcraft.com/'
                 'auction-data/notfound')
    found = 'http://auction-api-us.worldofwarcraft.com/auction-data'

    if 'fail' in url.path:
        return {'status_code': 403}

    return {
           'status_code': 200,
           'content':
           {
               'files': [
                   {'url': not_found if 'notfound' in url.path else found}
               ]
           }
       }


@urlmatch(path=r'^\/auction\-data.*$')
def auctions(url, request):
    if 'notfound' in url.path:
        return {'status_code': 404}

    return {'status_code': 200,
            'content': AUCTION_CONTENT}


class TestAuctions(unittest.TestCase):

    def test_get_auctions_url(self):
        with HTTMock(auctions_url_req):
            data = wow.get_auctions_url('medivh')
            self.assertEquals(data, ('http://auction-api-us.worldofwarcraft.'
                                     'com/auction-data'))

        with HTTMock(auctions_url_req):
            data = wow.get_auctions_url('fail')
            self.assertIsNone(data)

    def test_get_auctions(self):
        with HTTMock(auctions, auctions_url):
            data = wow.get_auctions('medivh')
            self.assertDictEqual(data, AUCTION_CONTENT)

        with HTTMock(auctions, auctions_url):
            self.assertRaises(Exception, wow.get_auctions, 'notfound')

if __name__ == '__main__':
    unittest.main()
    import requests

And that’s it! Now you have two choices to test your python code that uses requests.

So, I just started an interest on python tests and I would like to share what I learned so far.

Let’s skip all that theory about how tests are important for the code, bla bla bla, if you want to learn test, probably you already have your own reason, so, let’s jump to the part you actually write code!

Our application

Here a simple code that calculates fibonacci. Let’s save it in the file fibonacci.py

def fibonacci(number):
    if number < 2:
        return number
    return fibonacci(number - 1) + fibonacci(number - 2)

So, as you can see, it’s a very simple code that calculates the fibonacci number, so, how can we ensure that this is working? Well, the easiest way is just invoke the code with different values, and check if the result will be the expected. So, in other words, you’re doing exactly what doctest intent to do!
so, probably you would write a python file, let’s say, test_fibonacci.py with the following content:

from fibonacci import fibonacci

def test_fibonacci():
    expected_result = [0, 1, 1, 2, 3, 5]
    for x in range(0, 5):
        result = fibonacci(x)
        if result != expected_result[x]:
            print "Failing on calculate fibonacci %s" % x

if __name__ == '__main__':
    print "Testing fibonacci"
    test_fibonacci()

Well, it might work, until you want to calculate fibonacci(200), then you gonna need to add a new list, alter the test code, etc.
What if we now only accept positive numbers and negative numberes throws an exception? Try/catch block? Well, soon you’re writing more code, and adding more complexity than you were expecting. Maybe you gonna need to test your test code itself to ensure that it’s working fine (are you feeling the inception here?)

Doctest

So, doctest can help you with some of these problems! Let’s see, first of all, you’re gonna need to create a simple txt file and add the following content:

>>> from fibonacci import fibonacci

>>> fibonacci(0)
0

>>> fibonacci(1)
1

>>> fibonacci(2)
1

>>> fibonacci(3)
2

>>> fibonacci(4)
3

>>> fibonacci(7)
12

So, basically, doctest expect every line starting with “>>>” to be a python code (we are importing fibonacci function in the line number 1 for example. And any other line as the result, looking just like the python shell.

Assuming that both fibonacci.py and fibonacci.txt are in the same directory, you can run the doctest with the command:

$ python -m doctest fibonacci.txt
**********************************************************************
File "fibonacci.txt", line 18, in fibonacci.txt
Failed example:
    fibonacci(7)
Expected:
    12
Got:
    13
**********************************************************************
1 items had failures:
   1 of   7 in fibonacci.txt
***Test Failed*** 1 failures.

Oops! Something went wrong! Don’t worry, that was expected. As you can see, the doctest shows a very good output when some error occurs, it shows you the line where the error happened, what was the expected result, and what was the result doctest gots. So, after fixing it, and runing again, we got no error

>>> from fibonacci import fibonacci

>>> fibonacci(0)
0

>>> fibonacci(1)
1

>>> fibonacci(2)
1

>>> fibonacci(3)
2

>>> fibonacci(4)
3

>>> fibonacci(7)
13

$ python -m doctest fibonacci.txt

Now let’s go further, and say: Hey, I don’t want negative numbers to be allowed, I want it to throw an exception. So, in this case, we just need to add a new like in our fibonacci.txt testing the fibonacci(-1) and of course, checking if the result will be an exception.
If we would do that in our test_fibonacci.py code, we should use a try/catch block, call the fibonacci(-1) inside it, and of course, this woudln’t be in the initial loop, and if something goes wrong, you wouldn’t have the cool output from doctest.

So, let’s change our fibonacci.py to not accept negative numbers:

def fibonacci(number):
    if number == 1 or number == 0:
        return number
    if number < 0:
        raise ValueError('Number is negative')
    return fibonacci(number - 1) + fibonacci(number - 2)

Even if we run our doctest after this changes, we got no error, but we are not testing the negative number yet. so, let’s add a fibonacci(-1) in our fibonacci.txt test.

Just a heads up, if we run fibonacci(-1) we will get a exception like this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fibonacci.py", line 5, in fibonacci
    raise ValueError('Number is negative')
ValueError: Number is negative

And you’re probably thinking that you can copy and paste this error on our fibonacci.txt and it will work, but what if the file can change? You can add more code on the file, and the line where the error is being raised (in this case, line 5) might change. Being so, doctest only cares about the first and last lines, so our code in fibonacci.txt file will look like this:

>>> fibonacci(-1)
Traceback (most recent call last):
ValueError: Number is negative

And we are done! now we have a set of tests to our fibonacci code!

But what if you don’t want to have a txt and still have your tests? You can write doctest directly in your code on the docstring:

def fibonacci(number):
    '''
    This is a fibonacci function.
    Example of usage:
    >>> fibonacci(0)
    0
    >>> fibonacci(1)
    1
    >>> fibonacci(2)
    1
    >>> fibonacci(3)
    2
    >>> fibonacci(4)
    3
    >>> fibonacci(7)
    13
    >>> fibonacci(-1)
    Traceback (most recent call last):
    ValueError: Number is negative
    '''
    if number == 1 or number == 0:
        return number
    if number < 0:
        raise ValueError('Number is negative')
    return fibonacci(number - 1) + fibonacci(number - 2)

and you can run python -m doctest fibonacci.py.

Doctest have more features, as you can see below:

ELLIPISIS

According our fibonacci code, if you pass a negative number, it will raise a ValueError exception with the message ‘Number is negative’. There are some cases, that we don’t need to know which message is returning, we just want to know that an exception is being raised. Another example is when the output is the default __repr__ output, every time is a different memory address and you can’t control it. For this kind of situations, we can use the ELLIPISIS like in the example below.

>>> fibonacci(-1) #doctest: +ELLIPSIS
Traceback (most recent call last):
ValueError: ...

The line containing #doctest: + ELLIPSIS tells to doctest that the ‘…’ can match anything after ‘ValueError: ‘

SKIP

As the name says, it skips a test. This can be usefull when you know that some test is failing due some know bug, but it’s not fixed yet.

>>> fibonacci(-1) #doctest: +ELLIPSIS +SKIP
Traceback (most recent call last):
ValueError: ...

As you can see, you can use the two options, ELLIPSIS and SKIP together.

Conclusion

Tests can be fun and easy, and it helps to keep you code working properly in the whole cycle of development.
For more information about doctest, visit doctest website

Tag: python

Testing http requests with mock or HTTMock?