This makes it possible to run the execution-spec-tests (a.k.a. pyspec) in CI.

---------
Co-authored-by: Felix Lange <fjl@twurst.com>