This re-enables CI tests on every PR 🎉.
This uses make test-integration, which runs tests sequentially.
In theory, we also have test-integration-parallel but it seems to be very flaky: many tests randomly timeout. I didn't spend too much time investigating this, it's probably worth its own investigation in a separate PR, but for now it's important to re-enable CI tests, even if they are a bit slower.
This PR re-enables tests on `worker`s. Highlights:
* by default, each test is run twice: the main thread version uses `<script type="py">`, the worker version automatically turn the tags into `<script type="py" worker>`
* you can tweak the settings per-class by using the `@with_execution_thread` decorator. In particular, `@with_execution_thread(None)` is for those tests which don't care about it (e.g., `test_py_config.py`)
* inside each class, there might be some test which should be run only in the main thread (because it doesn't make sense to test it in a worker). For those, I introduced the `@only_main` decorator
* we might introduce `@only_worker` in the future, if needed
* `@skip_worker` is for those tests which currently pass on main but not on workers. These are meant to be temporary, and eventually they should all be fixed
During the process, I tweaked/improved/fixed/deleted some of the existing tests. Some of them were at risk of being flaky and I made them more robust, others depended on some very precise implementation detail, and I made them more generic (for example, `test_image_renders_correctly` relied on pillow to render an image with a very specific string of bytes, and it broke due to the recent upgrade to pyodide 0.24.1)
I also renamed all the skip messages to start with `NEXT`, so that they are easier to grep.
Clean up a bit the testsuite and integration tests.
Highlights:
- Some of the @skipped tests just worked -- I unskipped them
- some worked after some small tweak to adapt to the new pyscript next
- some are still skipped, but I tweaked the skip message to be more precise and descriptive
- Moreover, I killed/removed the ones which no longer make sense in the context of pyscript next; in particular, I removed all the ones which tested Element (which is now gone) and the one which tested py-config features which are no longer needed (e.g., multiple interpreters).
The testsuite passes locally.
* move integration tests pyscriptjs/tests/integration ->pyscript.core/tests/integration
* add information in regards to how to run integration tests to README
* fix fake server build paths
* fix paths to build and run tests. The change of path before integration tests is a glitch maybe due to pytest cache?
* remove test files created by mistake
* update readme with latest changes
---------
Co-authored-by: Fabio Pliger <fpliger@anaconda.com>
