Post History

How to use Python/pytest to organize unit and system tests?

I have a big system which I want to test using Python scripts: using `pytest` or any other infrastructure.

Some of my tests are of the "pass/fail" type; `pytest` has great support for these, like collecting, executing, summarizing, etc. Let's call them unit tests.

However, I have different tests, which produce some data about the system, like speed, quality, etc. I want to run these tests manually and examine the results (which are in csv or HTML format). Let's call them system tests.

Currently, I use shell-scripts for them. That means that any common part between these and unit tests has two versions: one written in shell language and another in Python.

How could I organize all my tests using Python?

I run my shell-scripts like this:

```
test/speed_test_1/run.sh
test/speed_test_2/run.sh
test/quality_test/run.sh
```

Naively, I'd like to rewrite them in Python and run like this:

```
test/speed_test_1/run.py
test/speed_test_2/run_this.py
test/speed_test_2/run_that.py
test/quality_test/run.py
```

However, running Python scripts in a subdirectory is not recommended because of problems with imports. That is — where would I put `pytest` fixtures and how would the `run` scripts access them? Not sure there is a good solution for that.

Could I use `pytest` to run my system tests? I imagine I'd need to change my command lines to something like this:

```
python -m pytest -k speed_test_1
python -m pytest -k speed_test_2_this
python -m pytest -k speed_test_2_that
python -m pytest -k quality_test
```

which looks ugly (e.g. there is no tab-completion for test names). And in addition, does pytest really support tests which have output other than pass/fail? I imagine there is something equally ugly I need to do to access my generated reports.

Is there a way I could use `pytest` or just any Python code to write system tests which could be invoked manually using shell-like syntax?