9 The Importance of Utilizing Different Tests

The first, and often easiest, type of testing to implement is a doctest. A doctest is a test you write in the docstring of your function– in fact in chapter 3, writing one or two docstrings is considered a best practice for designing a function.

def is_high_contrast(lum1: float, lum2: float) -> bool:


"""


Check if the contrast between two luminance values is high enoug, by comparing the

luminance values lum1 and lum2– each which range between (0.0 to 1.0).

The function returns true if the contrast is sufficient, False otherwise.


>>> is_high_contrast(1.0, 0.0)

True


>>> is_high_contrast(0.8, 0.7)


False 

"""


In the sample function above, we can see two doctests. When a line is prefaced by ‘>>>’ the indicates we are testing one line of code. The following line represents the value that should be returned after running the function.

These doctests not only provide a way to test your function. But, they are also a form of documentation. They show other people who will use your function sample function calls.

Best Practices for Creating Doctests

As doctests are tests in your docstring, you don’t want to create 50 tests in your docstring. So, you should only add the most important tests in your docstring. Here are a few general tips– these may not be applicable to all

1. If your function has n possible outputs, then it would be beneficial to have a different doctest for each output
   1. This is only feasible when n is small. If it has more outputs than that, it will start to become a hassle
2. It’s best to have a doctest concerning a singleton or zero of your set. But what do I mean by that. For every data type, you have a zero or a one. For examples
   1. For ints and floats its obviously 0 or 1
   2. For strings the zero is the “” while your one would be a string of length 1
   3. For lists the zero is an empty list while your one would be a list of length 1
   4. The list goes on. Based on your inputs, testing with that zero or one can also be helpful
3. If your function is supposed to give an error for certain input values, then your doctests should mirror that– you should an example input combination it will error for
4. Make sure you don’t try to use print statements in your doctests. It’s redundant because doctest automatically works with whatever your function returns

The Weakness of Doctests

Doctests are useful– but they have a few weaknesses:

1. If a function returns nothing (the NoneType) doctests are helpless at checking if you did something
2. If the order your ouput returns isn’t fixed, then doctests won’t help because doctests always have a fixed order
3. When you need to produce specific inputs for each of the functions– for example when you need inputs that aren’t primitive types or lists

We’ve established above that doctests aren’t suitable for extensive testing. The docstring is supposed to give users a quick overview of how the function works– so you can’t overload it with 50 tests. So, how do you test more cases? You could take up pen and paper, and create cases like that. But, thats inefficient. Instead, Python offers different ways to test your programs.

Method 1: Unittest

Unittest is a unit testing framework built into Python. Unit testing is testing that focuses on making sure each individual function in your program works accurately.

Here is an example:

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

How this module works is that

1. You import the unittest library
2. You create a class of functions of test your programs, and ensure it is a subclass of unittest.TestCase– in the above example that’s TestStringMethods
3. Create functions in the class where each function in your class is designed to test a function you implemented
   1. For example test_upper, test_isupper, and test_split each test the functions String.upper(), String.isupper(), and String.split()
   2. In order to test a function, you use assert statements like assertEqual, assertTrue, and assertFalse. The idea is you assert that something should be true if your function works. When the functions run, Python checks if your claims hold. If they don’t then you have something to fix
4. Then, you can run all the method in your unittest classes if you run unittest.main() in main

How to Create Test Cases

When you write a program, you need to make sure you cover every possible input. However, there are often going to be a infinite amount of possible inputs. You can’t cover everything.

So, instead you try to express all the possible inputs in different classes. For example, consider how you would test String.upper()– a function that given a string, returns the same string in upper case. There are an infinite number of strings you could pass to this function but we can broadly categorize them into five classes:

1. Strings that only contain letters in undercase
2. Strings that only contain letters in uppercase
3. Strings that contain letters in uppercase and undercase
4. Strings that only contain numbers and/or special characters
5. Strings that contain uppercase letters, lowercase letters, numbers and special characters

For cases 1-3, although there are infinite possible strings of these form, you don’t need to test it for all of these strings. If your code works for one string with all lowercase letters, one string with all uppercase letters and one string with both then it is most likely it will work for all of them.

For case 4, as long as the function works for a string with all numbers and special characters, it will work for any other case.

For case 5, you probably only need to check one string with uppercase letters, lowercase letters, numbers and special characters– if all the other cases work, then you should cover all strings of this type.

In that way, your approach to testing should be:

1. Consider all the different classes for your input
2. Make tests to address each input class

How to identify different classes?

There are two approaches to finding these classes:

1. Closed Box Approach: In this case, you don’t know how the function you are testing works. You just know what its supposed to do. This approach is best suited for creating doctests, because when you should be writing doctests before you write your dunction
2. Open Box Approach: In this case, you know how the function you are testing is implemented. So, you try to identify input classes based on what the function is supposed to do and how the function is implemented. This is the approach you would be approaching unittesting with– after you are done writing your function, and need to make sure your implementation works.

The input classes for each function is dependent upon what the function does. However, there are some generic classes you can consider based on the type of input the function accepts.  For example

1. When your input is a string, then some classes to consider are
   1. When string is empty
   2. When string is non-empty
2. When your input is an int or a float, then you can consider
   1. When the number < -1
   2. When the number = -1
   3. when -1 < number < 1
   4. When the number = 0
   5. When the number = 1
   6. When number > 0
3. When you’re input is a boolean, you can consider
   1. When the boolean is true
   2. When the boolean is false
4. When you’re input is a list, you can consider
   1. When the list is empty
   2. When the list is non-empty

There are also other fields you can consider, depending on the condition:

1. The size/length of an object (String, list, etc): 0, 1, larger, even odd
2. The position of a value in a sequence: start, middle, end, not there
3. The relative position between two values in a sequence: adjacent, separated, one/both values not there
4. Presence of duplicates: True or False
5. Ordered: ascending, descending, else unsorted
6. For ints/floats: large, small, even, odd, everything in table, etc
7. Value of string: is it strictly alphanumeric, has special characters, only alpha, only numeric, etc
8. Location of whitespace in a string– beginning, end, middle, multiple occurrences, different types of white space

When you start to consider what to test, you can consider the above to get started.

Pytest

Now, unittesting is useful– but let’s say you have 50 functions in your project. So that means in your unit testing class you need to add at least 50 additional functions in the same file. That makes it really annoying to look at. So, another approach you use for extended testing is Pytest.

Pytest is a module on Python that you can use to test functions. The highlight: you can put all your testing files on a sperate file away from your code. Here is how you create a pytest file.

1. Create a python file, just like usual. Technically, you can name it whatever you want, but usually you should name is test_x.py, where x = file whose functions you want to test.
2. On the file, import Pytest at the top. Then import the functions you want to test from your file
3. Now, you define each test case as a function as so:

def test_y():

assert [function call] == [supposed return value], y

Here, y = specific name of a test case. The assert works the same as how it does with Unittest

A whole pytest file looks something like:

import pytest

import vowels

from vowels import num_vowels, any_vowels

def test_any_vowels_one_vowel():

   assert any_vowels('qwert'), 'string of one vowel'



def test_any_vowels_no_vowels():

   assert not any_vowels('sdfghjkl'), 'string of no vowels'

pytest.main(["test_vowels.py"])