* test for no cors headers
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix tests
* suggested changes
* disable directly
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add error message
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* improve test
* improve error message
* remove py-config tag from cors test
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This PR adds support for optionally running pyodide in a web worker:
- add a new option config.execution_thread, which can be `main` or `worker`. The default is `main`
- improve the test machinery so that we run all tests twice, once for `main` and once for `worker`
- add a new esbuild target which builds the code for the worker
The support for workers is not complete and many features are still missing: there are 71 tests which are marked as `@skip_worker`, but we can fix them in subsequent PRs.
The vast majority of tests fail because js.document is unavailable: for it to run transparently, we need the "auto-syncify" feature of synclink.
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Co-authored-by: Madhur Tandon <20173739+madhur-tandon@users.noreply.github.com>
In the future this should help us leak fewer names into the pyscript
namespace.
Rather than assigning to the pyscript module from JavaScript, we
mount a separate private JS module with the extra names needed by
PyScript. I moved a bit more interpeter intialization into
remote_interpreter.
I added a deprecation warning for `pyscript.js`: the proper way to
access `js` is `import js`.
---------
Co-authored-by: Antonio Cuni <anto.cuni@gmail.com>
Switch to using the new version of synclink with support for same thread syncify (and also correct types).
Uses syncify to replace one use of `_unwrapped_remote`.
* remove pys-on* and py-on* attributes
* update changelog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix eslint
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Resolves pyscript#1313 and pyscript#1314. On top of pyscript#1318.
The point of these tests is to define the execution order of Tasks
that are scheduled in <py-script> tags: first all the py-script tags
are executed and their related lifecycle events. Once all of this
is done, we schedule any enqueued tasks.
To delay the execution of these tasks, we use a custom event loop for
pyExec with this defer behavior. Until schedule_deferred_tasks is called,
we defer tasks started by user code. schedule_deferred_tasks starts all
deferred user tasks and switches to immediately scheduling any further
user tasks.
Turns off:
- no-explicit-any (we need to handle any return values from runPython)
- no-unsafe-assignment (noisy and pointless)
And resolves all but two remaining ones. The last two lints regard access to private Pyodide variables and so:
- they are not easy to work around without an upstream Pyodide patch
- they should trigger linter and require explicit override
This is some refactoring I did on the way towards resolving pyscript#1313.
I added a new _run_pyscript Python function which executes the code
inside a context manager that sets the display target. We can then
return a JS object wrapper directly from Python.
I moved the "installation" of the pyscript module to loadInterpreter,
and pyimport pyscript_py there and give it a type. This avoids a bunch
of creating and deleting of proxies for pyscript_py and allows us to
give it a type once and for all.
I also did some minor logic cleanup in a few places.
This is a first step towards loading more stuff simultaneously rather
than sequentially.
The functional part of this is pretty small: call `calculateFetchPaths` and
then `Promise.all(fetchPaths.map(loadFileFromURL));`. I also transposed the
return type of `calculateFetchPaths` since it's more convenient to consume
this way.
I redid the logic in `calculateFetchPaths` a bit. I renamed `src/plugins/fetch.ts`
to `calculateFetchPaths.ts` since the file performs no fetching. I also
renamed `loadFromFile` to `loadFileFromURL`.
Followup to pyscript#1232. Closes pyscript#1226.
Use node to make a manifest of the src/python dir and then use an esbuild
plugin to resolve an import called `pyscript_python_package.esbuild_injected.json`
to an object indicating the directories and files in the package folder.
This object is then used to govern runtime installation of the package.
* Unvendor toml package
* Fix many ESlint errors
For mysterious reasons, these errors appear on my branch #1262 even
though they are not related to changes there. The eslint config seems
a bit unstable.
Anyways this fixes them.
* Put back Record
* Fix typescript compilation
* Fix lints
* Try @iarna/toml instead
* Fix import
* Use @ltd/j-toml
* Update test
* Use toml-j0.4
* Some changes
* Fix toml import
* Try adding eslint gha job
* Add forgotten checkout action
* Force CI to run
* Blah
* Fix
* Revert changes to github workflow
* Fix lints
* wget toml-j0.4 type definitions
* Add toml-j types workaround to eslint workflow
* Apply formatter
* Use @hoodmane/toml-j0.4
* Import from @hoodmane/toml-j0.4
* Apply prettier to css, js, html, md, ts, and yml
As a followup I will add prettier to the .pre-commit config.
This patch is 100% generated by prettier.
I used a forked version of prettier that understands the
py-script tag.
See https://github.com/hoodmane/pyscript-prettier-precommit
for more info.
* Apply old pre-commit
* Revert some problems
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert some changes
* More changes
* Fix pre-commit
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* split interpreter class
* add new files
* add newlines
* disable eslint for run
* remove usage of interpreter from unit test
* delete fakeinterpreter class
* fix unit tests
* add comments
* remove interpreter.ts and pyodide.ts files
* suggested changes
* Add beforePyScriptExec, afterPyScriptExec lifecycle hooks
* Add stdiodirector plugin for `output`, `stderr` attributes of py-script tag
* Add docs on `output` and `stderr` attributes of py-script tag
* Tests
* Removed output deprecation warning for `output` attribute
* Add createSingularWarning(), with createDeprecationWarning as alias
* kill the PyScript class and the weird pyscript instance; from the user point of view its functionalities are still available as pyscript.*, but pyscript is not the module, not the instance of PyScript
* simplify the code in _set_version_info, while I'm at it
* start to implement DeprecatedGlobal
* DeprecatedGlobal.__getattr__
* don't show the same warning twice
* DeprecatedGlobal.__call__
* make it possible to specify a different warning message for every global
* WIP: carefully use DeprecatedGlobal to show reasonable warning messages depending on which name you are accessing to. More names to follow
* deprecate more names
* deprecate private names
* depreacte direct usage of console and document
* deprecate the PyScript class
* use a better error message
* fix test_pyscript.py
* introduce a __repr__ for DeprecatedGlobal
* add an helper to ensure that we don't show any error or warning on the page
* WIP: ensure that examples don't use depreacted features. Many tests are failing
* don't deprecate Element
* don't use the global micropip to install packages, else we trigger a warning
* use a better error message for micropip
* fix test_todo_pylist to avoid using deprecated globals
* fix test_webgl_raycaster
* fix tests
* make HTML globally available
* add MIME_RENDERERS and MIME_METHODS
* fix the typing of Micropip, thanks to @FabioRosado
* Add runtime.invalidate_module_path_cache() to clear importlib's cache of modules
* Clear cache after [[fetch]], after plugin files download, after plugins setup, and prior to py-script tag execution.
* add test and example files
* update config to include python plugins in build
* add markdown plugin
* remove full pyscript execution from pyodide
* move loading of pyscript.py from pyodide loagInterpreter to main setupVirtualEnv and add function to create python CE plugins
* add plugin class to pyscript.py
* add missing import
* fix plugin path
* add fetchPythonPlugins to PyScriptApp
* remove old comments
* fix test
* add support for python plugins beyond custom elements and add app to python namespace in main
* inject reference to PyScript app onto python plugins
* add example hook onto markdown plugin
* change plugin events logs
* remove unused PyPlugin
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix type import
* add docstring to fetchPythonPlugins
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* rename addPythonPlugin method
* address PR comment
* call python plugins on hooks after the interpreted is ready
* add test for event hooks and split the test in 2 separate plugins to isolte type of plugins tests
* change python plugins initialization and registration, to inject the app from app itself instead of on the plugins themselves
* handle case when plugin cannot load due to missing plugin attribute
* add test for fail scenario when a plugin module does not have a plugin attribute
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add deprecation warning for pyscript objects loaded in global namespace
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove all from global scope
* remove create_custom_element from global scope
* rename create_custom_element to define_custom_element
* rename attributes in define_custom_element and add docstrings
* better handle connect event output
* add warning to py_markdown plugin
* remove debugging logs
* improve tests
* remove debugging log
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove unused import
* add executable shebang
* add pyodide mock module
* fmt and lint
* Update to pyodide.ffi.create_proxy per pyodide v21 api change
* Mock pyodide as package instead of mdoule
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add __init__ to pyodide package
* Update pyscriptjs/src/plugin.ts
fix logger name
Co-authored-by: Antonio Cuni <anto.cuni@gmail.com>
* fix pyodide import but handling the diff in their API change
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* oops, conflict resolution blooper
* Fix failing integration tests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jeff Glass <glass.jeffrey@gmail.com>
Co-authored-by: Antonio Cuni <anto.cuni@gmail.com>
Co-authored-by: FabioRosado <fabiorosado@outlook.com>
This PR move codes from main.ts into two new plugins:
- splashscreen (formerly known as py-loader)
- importmap
The old setting config.autoclose_loader is still supported but deprecated; the new setting is config.splashscreen.autoclose.
Moreover, it does a small refactoring around UserError: now UserErrors are correctly caught even if they are raised from within afterRuntimeLoad.
* Raise error if we get a non 200 status from fetch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add custom exception
* Add check for TypeError as well
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This PR does two things:
1. introduce the first minimal version of a Plugin system. Plugins are subclasses of the Plugin class, and pyscript will call the relevant methods/hooks at the right time. Currently, I have added only the minimal sets of hooks which were needed for this PR.
2. Implement <py-terminal> as a plugin.
* implement proposal
* update docs and replace py-env
* more docs
* suggested proposal
* update docs
* add to_file parameter
* remove comment from Makefile
* suggested improvements
* move tests from basic to py_config
* retain leading slash from the first path
Yet another refactoring to untangle the old mess.
Highlights:
base.ts, pyscript.ts and pyrepl.ts were a tangled mess of code, in which each of them interacted with the others in non-obvious ways. Now PyScript is no longer a subclass of BaseEvalElement and it is much simpler. I removed code for handling the attributes std-out and std-err because they are no longer needed with the new display() logic.
The logic for executing python code is now in pyexec.ts: so we are decoupling the process of "finding" the python code (handled by the py-script web component) and the logic to actually execute it. This has many advantages, including the fact that it will be more easily usable by other components (e.g. pyrepl). Also, note that it's called pyexec and not pyeval: in the vast majority of cases in Python you have statements to execute, and almost never expressions to evaluate.
I killed the last remaining global store, scriptQueue tada. As a bonus effect, now we automatically do the correct thing when a <py-script> tag is dynamically added to the DOM (I added a test for it). I did not remove svelte from packages.json, because I don't fully understand the implications: there are various options which mention svelte in rollup.js and tsconfig.json, so it's probably better to kill it in its own PR.
pyexec.ts is also responsible of handling the default target for display() and correct handling/visualization of exceptions. I fixed/improved/added display/output tests in the process.
I also found a problem though, see issue #878, so I improved the test and marked it as xfail.
I removed BaseEvalElement as the superclass of most components. Now the only class which inherits from it is PyRepl. In a follow-up PR, I plan to merge them into a single class and do more cleanup.
During the refactoring, I killed guidGenerator: now instead of generating random py-* IDs which are very hard to read for humans, we generated py-internal-X IDs, where X is 0, 1, 2, 3, etc. This makes writing tests and debugging much easier.
I improved a lot our test machinery: it turns out that PR #829 broke the ability to use/view sourcemaps inside the playwright browser (at least on my machine).
For some reason chromium is unable to find sourcemaps if you use playwrights internal routing. So I reintroduced the http_server fixture which was removed by that PR, and added a pytest option --no-fake-server to use it instead, useful for debugging. By default we are still using the fakeserver though (which is faster and parallelizable).
Similarly, I added --dev which implies --headed and also automatically open chrome dev tools.
* export 'pyscript' JS module with runtime attribute, to allow accessing (Pyodide) runtime and globals from JS.
* add docs to explain the js module
* add integration tests for the js module
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
As the title stays, the main goal of the branch is to kill the infamous runtimeLoaded global store and all the complications, problems and bugs caused by the fact that in many places we needed to ensure/wait that the global runtime was properly set before being able to execute code.
The core idea is that runtime is never a global object and that it's passed around explicitly, which means that when a function receives it, it is guaranteed to be initialized&ready.
This caused a bit of complications in pybutton.ts, pyinputbox.ts and pyrepl.ts, because they indirectly want to call runtime.run from connectedCallback, which is the only place where we cannot explicitly pass the runtime because it's automatically called by the browser.
But also, it is also a sign of a bad design, because it were entirely possible that connectedCallback was called before the runtime was ready, which probably caused many bugs, see e.g. #673 and #747.
The solution to is use dependency injection and create the class later on: so instead of having a global PyButton class which relies on a global runtime (whose state is uncertain) we have a make_PyButton function which takes a runtime and make a PyButton class which is tied to that specific runtime (whose state is certainly ready, because we call make_PyButton only when we know that the runtime is ready).
Similar for PyInputBox and PyRepl.
Other highlights: thanks to this, I could kill the also infamous runAfterRuntimeInitialized and a couple of smelly lines which used setTimeout to "wait" for the runtime.
While I was at it, I also called a lot of other stores which were completely unused and where probably leftovers from a past universe.
This is a follow-up of PR #806 and it's a big step forward towards solving issue #763.
The basic idea is that at this stage I want to streamline the execution logic
and make it as linear and easy to read/understand as possible. The diff is
relatively big, but for the most part is just "shuffling code around".
Svelte stores:
the idea is to eventually kill of them, so that we can remove the dependency
on svelte but also to avoid relying on so much global state, which makes
things more complicated and error-prone (e.g., we have several issues related
to using runtime when it's not ready yet).
I killed addInitializer, addPostInitializer and the corresponding svelte
stores tada. They are no longer needed since the relevant code is called
directly from main.ts.
I started to kill the usage of the runtimeLoaded svelte store: instead of
relying on a global variable, I want to arrive at the point in which the
runtime is passed as a parameter in all places where it's needed: pyscript.ts
is now free of global state, but I couldn't kill it yet because it's used
heavily by base.ts and pyrepl.ts. I will do it in another PR.
Other misc changes:
I added sanity checks (and corresponding tests!) which complain if you specify
0 or multiple runtimes. Currently we support having one and only one, so there
is no point to pretend otherwise
I modified the messages displayed by the loader, to be more informative from
the user point of view.
