I tested both pytest and nose for testing my hy code. Unfortunately I couldn’t get nose to work. Pytest did work after some weird configuration and monkey patching.

Update: I gave up on hy so this might stop working or something

Pytest

Pytest is used to test the hy code that is part of the hy language itself. Unfortunately, the support is not built into pytest and it’s all a bit hacky. Here’s what I had to do:

Create a pytest.ini file with some configuration options:

[pytest]
testpaths = tests
addopts = --assert=plain

The most important option here is –assert=plain. This stops pytest from attempting to parse the .hy files as python to replace the calls to assert with better error messages. Since we’re in hy we can just write a macro to do the same thing :)

Create a conftest.py file that monkey patches pytest to enable loading hy files. This file is lightly modified but mainly copied from the hy repository.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# copying hacks from https://github.com/hylang/hy/blob/master/conftest.py

import hy, importlib, py, pytest

_fspath_pyimport = py.path.local.pyimport

def hyimport(self):
    pkgpath = self.pypkgpath()
    if pkgpath is None:
        pkgroot = self.dirpath()
        modname = self.purebasename
    else:
        pkgroot = pkgpath.dirpath()
        names = self.new(ext="").relto(pkgroot).split(self.sep)
        if names[-1] == "__init__":
            names.pop()
        modname = ".".join(names)

    res = importlib.import_module(modname)
    return res

def pyimport_patch_mismatch(self, **kwargs):
    """hacky fix for https://github.com/pytest-dev/py/issues/195"""
    if (self.ext == '.hy'):
        return hyimport(self)
    try:
        return _fspath_pyimport(self, **kwargs)
    except py.path.local.ImportMismatchError:
        hyimport(self)

def pytest_collect_file(parent, path):
    if (path.ext == ".hy" and path.basename != "__init__.hy"):
        pytest_mod = pytest.Module(path, parent)
        return pytest_mod

py.path.local.pyimport = pyimport_patch_mismatch

Note that this also works around an issue in python itself.

Finally, it’s necessary to put a __init__.py file in the tests/ directory, to make pytest import it as a module.

Once those things above are done pytest can be used to run all tests or just tests from a specific file or whatever!

assert_eq macro

Since pytest’s assert-rewriting is turned off, the assert statements will give opaque error messages that don’t include the tested values. By writing an assert_eq macro, the same thing can be had with hy’s macro system!

(defmacro! assert_eq [o!l o!r]
  `(assert (= ~g!l ~g!r) (.format "{} != {}" ~g!l ~g!r)))

Note that defmacro! and the o! prefix are used to ensure that the left and right sides are only evaluated once.

I think maybe at some point I would like to make a package that has some hy macros for testing, and maybe also a package that enables hy support in pytest, like with the code above.

Nose

Ultimately, I found that Nose wasn’t going to work and it would take too much effort to get it working in a nice way. This is because although you an modify what files are selected, and how tests are found in them, the code which implements special nose features to find tests cannot be incorporated into a plugin (unless you copy-paste it) and will only ever run on .py files.

The documentation describes how to load custom tests:

Writing a plugin that loads tests from files other than python modules

Implement wantFile and loadTestsFromFile. In wantFile, return True for files that you want to examine for tests. In loadTestsFromFile, for those files, return an iterable containing TestCases (or yield them as you find them; loadTestsFromFile may also be a generator).

Unfortunately, there’s no easy way to implement loadTestsFromFile in a way that uses the existing test discovery methods in nose.