Creating a New Test

This section describes a way of creating new regression tests.

Basics

To create a new test, perform the following steps:

1. Create a new subdirectory inside the root directory. Example:

regression-tests
└── integration
    └── ack

The naming conventions are described in General Overview.

2. Create a test.py file inside that directory. Example:

regression-tests
└── integration
    └── ack
        └── test.py

The structure of this file is described later.

3. Add all the necessary input files to the created directory. Example:

regression-tests
└── integration
    └── ack
        ├── ack.exe
        └── test.py

General Structure of test.py Files

A test.py file has the following structure:

from regression_tests import *

# Test class 1
# Test class 2
# ...

The first line imports all the needed class names from the regression_tests package. You can import them explicitly if you want, but the import * is just fine in our case [1]. More specifically, it automatically imports the following classes, discussed later in this documentation:

• Test
• TestSettings

Also, other useful functions are automatically imported, which we will use later.

Every test class is of the following form:

class TestName(Test):
    # Settings 1
    # Settings 2
    # ...

    # Test method 1
    # Test method 2
    # ...

The name of a class can be any valid Python identifier, written by convention in CamelCase. Furthermore, it has to inherit from Test, which represents the base class of all tests. Example:

class AckTest(Test):
    # ...

Every test setting is of the following form:

settings_name = TestSettings(
    # Decompilation arguments.
    # ...
)

The arguments specify the input file(s), architecture(s), file format(s), and other settings to be used in the test cases.

Finally, there have to be some test methods to check that the decompiled code satisfies the needed properties. Every test method is of the following form:

def test_description_of_the_test(self):
    # Assertions
    # ...

A method has to start with the prefix test_ and its name should contain a description of what exactly is this method testing. This is a usual convention among unit test frameworks. The reason is that when a test fails, its method name reveals the purpose of the test.

The assertions can be assert statements, assertions from the standard unittest module (self.assertXYZ()), or specific assertions provided by the classes of the regression tests framework. Example:

def test_ack_has_correct_signature(self):
    # The following statements assert that in the decompiled C code, there
    # is an ack() function, its return type is int, and that it has two
    # parameters of type int.
    self.assertTrue(self.out_c.funcs['ack'].return_type.is_int(32))
    self.assertEqual(self.out_c.funcs['ack'].param_count, 2)
    self.assertTrue(self.out_c.funcs['ack'].params[0].type.is_int(32))
    self.assertTrue(self.out_c.funcs['ack'].params[1].type.is_int(32))

The above example uses assertions from the standard unittest module [2].

See section Writing Test Methods for more details on how to write test methods and assertions.

Example of a test.py File

The following code is a complete example of a test.py file:

from regression_tests import *

class AckTest(Test):
    settings = TestSettings(
        input='ack.exe'
    )

    def test_ack_has_correct_signature(self):
        # The following statements assert that in the decompiled C code,
        # there is an ack() function, its return type is int, and that it
        # has two parameters of type int.
        self.assertTrue(self.out_c.funcs['ack'].return_type.is_int(32))
        self.assertEqual(self.out_c.funcs['ack'].param_count, 2)
        self.assertTrue(self.out_c.funcs['ack'].params[0].type.is_int(32))
        self.assertTrue(self.out_c.funcs['ack'].params[1].type.is_int(32))

Next, we will learn the details of specifying test settings.

[1]	Do not use the construct import * in real-world projects though because of namespace pollution and other reasons.

[2]	The assertions in the standard unittest module are historically named by using CamelCase instead of snake_case.