This PR move codes from main.ts into two new plugins:
- splashscreen (formerly known as py-loader)
- importmap
The old setting config.autoclose_loader is still supported but deprecated; the new setting is config.splashscreen.autoclose.
Moreover, it does a small refactoring around UserError: now UserErrors are correctly caught even if they are raised from within afterRuntimeLoad.
* use pytest-cache to cache HTTP requests, so that it survives across multiple runs
* add an option to clear the HTTP cache
* add some docs about pytest
Until now, we didn't have a nice way to check that we expect a specific JS error in the web page.
This PR improves check_js_errors() so that now you can pass a list of error messages that you expect.
It is tricky because we need to handle (and test!) all various combinations of cases:
- errors expected and found / expected but not found
- unexpected errors found / not found
Moreover, JS exceptions now are logged in the special category console.js_error, which means that the printed text is also available using e.g. self.console.js_error.text or self.console.all.text. However, this should never be required and it's preferred to use self.check_js_errors to check for exceptions. This fixes#795 .
Finally, use the new logic to improve test_no_implicit_target.
Yet another refactoring to untangle the old mess.
Highlights:
base.ts, pyscript.ts and pyrepl.ts were a tangled mess of code, in which each of them interacted with the others in non-obvious ways. Now PyScript is no longer a subclass of BaseEvalElement and it is much simpler. I removed code for handling the attributes std-out and std-err because they are no longer needed with the new display() logic.
The logic for executing python code is now in pyexec.ts: so we are decoupling the process of "finding" the python code (handled by the py-script web component) and the logic to actually execute it. This has many advantages, including the fact that it will be more easily usable by other components (e.g. pyrepl). Also, note that it's called pyexec and not pyeval: in the vast majority of cases in Python you have statements to execute, and almost never expressions to evaluate.
I killed the last remaining global store, scriptQueue tada. As a bonus effect, now we automatically do the correct thing when a <py-script> tag is dynamically added to the DOM (I added a test for it). I did not remove svelte from packages.json, because I don't fully understand the implications: there are various options which mention svelte in rollup.js and tsconfig.json, so it's probably better to kill it in its own PR.
pyexec.ts is also responsible of handling the default target for display() and correct handling/visualization of exceptions. I fixed/improved/added display/output tests in the process.
I also found a problem though, see issue #878, so I improved the test and marked it as xfail.
I removed BaseEvalElement as the superclass of most components. Now the only class which inherits from it is PyRepl. In a follow-up PR, I plan to merge them into a single class and do more cleanup.
During the refactoring, I killed guidGenerator: now instead of generating random py-* IDs which are very hard to read for humans, we generated py-internal-X IDs, where X is 0, 1, 2, 3, etc. This makes writing tests and debugging much easier.
I improved a lot our test machinery: it turns out that PR #829 broke the ability to use/view sourcemaps inside the playwright browser (at least on my machine).
For some reason chromium is unable to find sourcemaps if you use playwrights internal routing. So I reintroduced the http_server fixture which was removed by that PR, and added a pytest option --no-fake-server to use it instead, useful for debugging. By default we are still using the fakeserver though (which is faster and parallelizable).
Similarly, I added --dev which implies --headed and also automatically open chrome dev tools.
* Add cache, fetch, retry logic to tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* run in parallel
* add pytest-xdist
* undo parallelism. Need to remove http server to enable.
* woops a extra space
* Pass flake8
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* spell fulfill
* use decorator for fetch if not in cache
* Fix --headed and limit to PlaywrightRequestError
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docs on cache
* CICD caching of conda on unstable builds
* fix config issues
* empty commit to trigger gh-actions
* restore build-unstable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove http server, add parallel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* temp: Bypass zip runtime test and point to v0.21.3 on CDN
* suport for files in zip under /pyodide
* remove test-one
* self.http_server and remove content_type
* domcontentloaded w no timeout on base url + longer timeout on wait_for_pyscript
* Fixed#678
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* set default timeout to 60000
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* seamless --headed support
* add test-integration-parallel and default for GHActions
* simplify the code. Use http://fakeserver instead of localhost:8080 so that it's clearer that the browser is NOT hitting a real server, and use urllib to parse the url. Moreover, the special case for pyodide is no longer needed, it's automatically handled by the normal 'fakeserver' logic
* The page-routing logic is becoming too much complicated to stay as an inner function. Move it to its own class, and add some logic to workaround a limitation of playwright which just hangs if a Python exception is raised inside it
* no need to use a hash, we can use the url as the key
* re-implement the retry logic. The old @retry decorator was nice but a bit too over-engineered and most importantly failed silently in case of exceptions. This new approach is less powerful but since we want to retry only two times, simple is better than complex -- and in case of exception, the exception is actually raised
* improve logging
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Madhur Tandon <madhurtandon23@gmail.com>
Co-authored-by: Antonio Cuni <anto.cuni@gmail.com>
This is a follow-up of PR #806 and it's a big step forward towards solving issue #763.
The basic idea is that at this stage I want to streamline the execution logic
and make it as linear and easy to read/understand as possible. The diff is
relatively big, but for the most part is just "shuffling code around".
Svelte stores:
the idea is to eventually kill of them, so that we can remove the dependency
on svelte but also to avoid relying on so much global state, which makes
things more complicated and error-prone (e.g., we have several issues related
to using runtime when it's not ready yet).
I killed addInitializer, addPostInitializer and the corresponding svelte
stores tada. They are no longer needed since the relevant code is called
directly from main.ts.
I started to kill the usage of the runtimeLoaded svelte store: instead of
relying on a global variable, I want to arrive at the point in which the
runtime is passed as a parameter in all places where it's needed: pyscript.ts
is now free of global state, but I couldn't kill it yet because it's used
heavily by base.ts and pyrepl.ts. I will do it in another PR.
Other misc changes:
I added sanity checks (and corresponding tests!) which complain if you specify
0 or multiple runtimes. Currently we support having one and only one, so there
is no point to pretend otherwise
I modified the messages displayed by the loader, to be more informative from
the user point of view.
* Create custom elements when the runtime finishes loading
* Remove xfails and fix repl integration test
* Fix commented ignore
* Address Antonio's comments
* Fix bad rebase
* Make ure to wait for repl to be in attached state before asserting content
* Move createCustomeElement up so it runs before we close the loader, xfail flaky d3 test
* Fix xfail
This PR tries to improve and rationalize what we log. Key points:
- introduce `logger.ts`: each file/component is encouraged to use the logger instead of writing directly to `console.*`
* the logger automatically prepend a prefix like `[py-config]`, `[py-env]` which make it easier to understand where a certain message is printed from
* it provide a central place where to add more features in the future. E.g., I can imagine having a config setting to completely silence the logs (not implemented yet)
- use the new loggers everywhere
- write to `.info()` instead of `.log()`. The idea is to keep `console.log` free, so that for the users it's easier to tell apart their own messages and the pyscript ones
- generally improve what we log. This is an endless exercise, but I tried to print more things which are useful to understand what's going on and in which order the various things are executed, and remove prints which were clearly debugging leftovers
* add integration test for py-config
* fix bug
* fix test
* remove indexURL altogether
* make jest happy
* fix create_proxy import
* check that py-config loads an older version
* add unit test
* suggested changes
* don't use /tmp because of bandit
* move current integration tests to the integration folder
* move pyscript.py into its own python folder
* change the path for python unit testing files
* change pyscript.py path
* Update Makefile
* remove echo
* replace conda run with pytest directly
* oops, add python test files I embarrassingly forgot to add
Co-authored-by: Peter W <34256109+pww217@users.noreply.github.com>
