Use registerJSModule when available (#1573)

pyscript.core/.gitignore

@@ -4,6 +4,7 @@ node_modules/
 cjs/
 !cjs/package.json
 core.js
+pyscript.js
 esm/worker/xworker.js
 esm/worker/__template.js
 types/

pyscript.core/docs/README.md

@@ -229,11 +229,11 @@ Please read the [Terminology](#terminology) **target** dedicated details to know
   <summary><strong>XWorker</strong></summary>
   <div>
 
-With or without access to the `document`, every (*non experimental*) interpreter will have defined, at the global level, a reference to the `XWorker` "_class_" (it's just a *function*!), which goal is to enable off-loading heavy operations on a worker, without blocking the main / UI thread (the current page) and allowing such worker to even reach the `document` or anything else available on the very same main / UI thread.
+With or without access to the `document`, every (*non experimental*) interpreter will have defined, either at the global level or after an import (i.e.`from xworker import XWorker` in *Python* case), a reference to the `XWorker` "_class_" (it's just a *function*!), which goal is to enable off-loading heavy operations on a worker, without blocking the main / UI thread (the current page) and allowing such worker to even reach the `document` or anything else available on the very same main / UI thread.
 
 ```html
 <script type="micropython">
-    # XWorker is globally defined
+    from xworker import XWorker
     print(XWorker != None)
 </script>
 ```
@@ -314,9 +314,9 @@ Whenever computing relatively expensive stuff, such as a *matplot* image, or lit
 
 `@pyscript/core` adds a functionality called `XWorker` to all of the interpreters it offers, which works in each language the way `Worker` does in JavaScript.
 
-In each Interpreter, `XWorker` is a global reference, with a counter `xworker` (lower case) global reference within the worker code.
+In each Interpreter, `XWorker` is either global reference or an import (i.e.`from xworker import XWorker` in *Python* case) module's utility, with a counter `xworker` (lower case) global reference, or an import (i.e.`from xworker import xworker` in *Python* case) module's utility, within the worker code.
 
-In short, the `XWorker` global goal is to help, without much thinking, to run any desired interpreter out of a *Worker*, enabling extra features on the *worker*'s code side.
+In short, the `XWorker` utility is to help, without much thinking, to run any desired interpreter out of a *Worker*, enabling extra features on the *worker*'s code side.
 
 
 ### Enabling XWorker
@@ -348,7 +348,7 @@ The returning *JS* reference to any `XWorker(...)` call is literally a `Worker`
 
 | name      | example                            | behavior |
 | :-------- | :--------------------------------- | :--------|
-| sync      | `sync = XWorker('./file.py').sync` | Allows exposure of callbacks that can be run synchronously from the worker file, even if the defined callback is *asynchronous*. This property is also available in the global `xworker` reference. |
+| sync      | `sync = XWorker('./file.py').sync` | Allows exposure of callbacks that can be run synchronously from the worker file, even if the defined callback is *asynchronous*. This property is also available in the `xworker` reference. |
 
 ```python
 
@@ -370,9 +370,9 @@ In the `xworker` counter part:
 xworker.sync.from_main(1, "two")
 ```
 
-### The xworker global reference
+### The xworker reference
 
-The content of the file used to initialize any `XWorker` on the main thread can always reach the `xworker` counter part as globally available (that means: no *import ... form ...* is necessary, it is already there).
+The content of the file used to initialize any `XWorker` on the main thread can always reach the `xworker` counter part as globally available or as import (i.e.`from xworker import xworker` in *Python* case) module's utility.
 
 Within a *Worker* execution context, the `xworker` exposes the following features:
 
@@ -380,7 +380,7 @@ Within a *Worker* execution context, the `xworker` exposes the following feature
 | :------------ | :------------------------------------------| :--------|
 | sync          | `xworker.sync.from_main(1, "two")`         | Executes the exposed `from_main` function in the main thread. Returns synchronously its result, if any. |
 | window        | `xworker.window.document.title = 'Worker'` | Differently from *pyodide* or *micropython* `import js`, this field allows every single possible operation directly in the main thread. It does not refer to the local `js` environment the interpreter might have decided to expose, it is a proxy to handle otherwise impossible operations in the main thread, such as manipulating the *DOM*, reading `localStorage` otherwise not available in workers, change location or anything else usually possible to do in the main thread. |
-| isWindowProxy | `xworker.isWindowProxy(ref)`               | **Advanced** - Allows introspection of *JS* references, helping differentiating between local worker references, and main thread global references. This is valid both for non primitive objects (array, dictionaries) as well as functions, as functions are also enabled via `xworker.window` in both ways: we can add a listener from the worker or invoke a function in the main. Please note that functions passed to the main thread will always be invoked asynchronously.
+| isWindowProxy | `xworker.isWindowProxy(ref)`               | **Advanced** - Allows introspection of *JS* references, helping differentiating between local worker references, and main thread global JS references. This is valid both for non primitive objects (array, dictionaries) as well as functions, as functions are also enabled via `xworker.window` in both ways: we can add a listener from the worker or invoke a function in the main. Please note that functions passed to the main thread will always be invoked asynchronously.
 
 ```python
 print(xworker.window.document.title)
@@ -472,7 +472,7 @@ In few words, while every *interpreter* is literally passed along to unlock its
 | :------------------------ | :-------------------------------------------- | :--------|
 | type                      | `wrap.type`                                   | Return the current `type` (interpreter or custom type) used in the current code execution. |
 | interpreter               | `wrap.interpreter`                            | Return the *interpreter* _AS-IS_ after being bootstrapped by the desired `config`. |
-| XWorker                   | `wrap.XWorker`                                | Refer to the global `XWorker` class available to the main thread code while executing. |
+| XWorker                   | `wrap.XWorker`                                | Refer to the `XWorker` class available to the main thread code while executing. |
 | io                        | `wrap.io`                                     | Allow to lazily define different `stdout` or `stderr` via the running *interpreter*. This `io` field can be lazily defined and restored back for any element currently running the code. |
 | config                    | `wrap.config`                                 | It is the resolved *JSON* config and it is an own clone per each element running the code, usable also as "_state_" reference for the specific element, as changing it at run time will never affect any other element. |
 | run                       | `wrap.run(code)`                              | It abstracts away the need to know the exact method name used to run code synchronously, whenever the *interpreter* allows such operation, facilitating future migrations from an interpreter to another. |

pyscript.core/esm/custom.js

@@ -107,7 +107,9 @@ export const handleCustomType = (node) => {
                         };
                     }
 
-                    module.setGlobal(interpreter, "XWorker", XWorker);
+                    module.registerJSModule(interpreter, "xworker", {
+                        XWorker,
+                    });
 
                     const resolved = {
                         type,

pyscript.core/esm/interpreter/_python.js

@@ -15,6 +15,10 @@ export const deleteGlobal = (interpreter, name) => {
     interpreter.globals.delete(name);
 };
 
+export const registerJSModule = (interpreter, name, value) => {
+    interpreter.registerJsModule(name, value);
+};
+
 export const writeFile = ({ FS }, path, buffer) =>
     writeFileUtil(FS, path, buffer);
 /* c8 ignore stop */

pyscript.core/esm/interpreter/_utils.js

@@ -1,7 +1,7 @@
 import "@ungap/with-resolvers";
 
 import { getBuffer } from "../fetch-utils.js";
-import { absoluteURL } from "../utils.js";
+import { absoluteURL, entries } from "../utils.js";
 
 /**
  * Trim code only if it's a single line that prettier or other tools might have modified.
@@ -131,3 +131,12 @@ export const fetchPaths = (module, interpreter, config_fetch) =>
                 .then((buffer) => module.writeFile(interpreter, path, buffer)),
         ),
     );
+
+// this is a fallback for interpreters unable to register JS modules
+// all defined keys will end up as globally available references
+// REQUIRES INTEGRATION TEST
+/* c8 ignore start */
+export function registerJSModule(interpreter, _, value) {
+    for (const [k, v] of entries(value)) this.setGlobal(interpreter, k, v);
+}
+/* c8 ignore stop */

pyscript.core/esm/interpreter/micropython.js

@@ -1,5 +1,11 @@
 import { fetchPaths, stdio } from "./_utils.js";
-import { run, setGlobal, deleteGlobal, writeFile } from "./_python.js";
+import {
+    run,
+    setGlobal,
+    deleteGlobal,
+    registerJSModule,
+    writeFile,
+} from "./_python.js";
 
 const type = "micropython";
 
@@ -18,6 +24,7 @@ export default {
     },
     setGlobal,
     deleteGlobal,
+    registerJSModule,
     run,
     // TODO: MicroPython doesn't have a Pyodide like top-level await,
     //       this method should still not throw errors once invoked

pyscript.core/esm/interpreter/pyodide.js

@@ -4,6 +4,7 @@ import {
     runAsync,
     setGlobal,
     deleteGlobal,
+    registerJSModule,
     writeFile,
 } from "./_python.js";
 
@@ -32,6 +33,7 @@ export default {
     },
     setGlobal,
     deleteGlobal,
+    registerJSModule,
     run,
     runAsync,
     writeFile,

pyscript.core/esm/interpreter/ruby-wasm-wasi.js

@@ -1,4 +1,4 @@
-import { clean, fetchPaths } from "./_utils.js";
+import { clean, fetchPaths, registerJSModule } from "./_utils.js";
 
 const type = "ruby-wasm-wasi";
 
@@ -23,6 +23,7 @@ export default {
         if (config.fetch) await fetchPaths(this, interpreter, config.fetch);
         return interpreter;
     },
+    registerJSModule,
     setGlobal(interpreter, name, value) {
         const id = `__pyscript_ruby_wasm_wasi_${name}`;
         globalThis[id] = value;

pyscript.core/esm/interpreter/wasmoon.js

@@ -1,7 +1,16 @@
-import { clean, fetchPaths, stdio, writeFileShim } from "./_utils.js";
+import {
+    clean,
+    fetchPaths,
+    stdio,
+    registerJSModule,
+    writeFileShim,
+} from "./_utils.js";
 
 const type = "wasmoon";
 
+// MISSING:
+//  * I've no idea how to import packages
+
 // REQUIRES INTEGRATION TEST
 /* c8 ignore start */
 export default {
@@ -18,6 +27,7 @@ export default {
         if (config.fetch) await fetchPaths(this, interpreter, config.fetch);
         return interpreter;
     },
+    registerJSModule,
     setGlobal(interpreter, name, value) {
         interpreter.global.set(name, value);
     },

pyscript.core/esm/script-handler.js

@@ -58,11 +58,10 @@ const execute = async (script, source, XWorker, isAsync) => {
             configurable: true,
             get: () => script,
         });
-        module.setGlobal(interpreter, "XWorker", XWorker);
+        module.registerJSModule(interpreter, "xworker", { XWorker });
         return module[isAsync ? "runAsync" : "run"](interpreter, content);
     } finally {
         delete document.currentScript;
-        module.deleteGlobal(interpreter, "XWorker");
     }
 };
 

pyscript.core/esm/utils.js

@@ -1,6 +1,6 @@
 const { isArray } = Array;
 
-const { assign, create, defineProperties, defineProperty } = Object;
+const { assign, create, defineProperties, defineProperty, entries } = Object;
 
 const { all, resolve } = new Proxy(Promise, {
     get: ($, name) => $[name].bind($),
@@ -14,6 +14,7 @@ export {
     create,
     defineProperties,
     defineProperty,
+    entries,
     all,
     resolve,
     absoluteURL,

pyscript.core/esm/worker/_template.js

@@ -98,7 +98,7 @@ add("message", ({ data: { options, code, hooks } }) => {
             }
         }
         // set the `xworker` global reference once
-        details.setGlobal(interpreter, "xworker", xworker);
+        details.registerJSModule(interpreter, "xworker", { xworker });
         // simplify run calls after possible patches
         run = details[name].bind(details, interpreter);
         // execute the content of the worker file

pyscript.core/package.json

@@ -66,6 +66,6 @@
         "coincident": "^0.8.3"
     },
     "worker": {
-        "blob": "sha256-eWNZbyS06lxxlUW/bkU7/fl/Levxxxfiv/+frsgl/fA="
+        "blob": "sha256-71McgT96jsjEBqY19EQRYQYwwoZQ99VTYu60UU6i03w="
     }
 }

pyscript.core/test/doc-intro.html

@@ -15,7 +15,7 @@
             print(document.currentScript.id == my_target.id)
         </script>
         <script type="micropython">
-            # XWorker is globally defined
+            from xworker import XWorker
             print(XWorker != None)
         </script>
         <script type="micropython">

pyscript.core/test/matplot.py

@@ -1,5 +1,6 @@
 import js
 import matplotlib
+from xworker import xworker
 
 try:
     js.document

pyscript.core/test/matplot.worker.html

@@ -10,6 +10,7 @@
     </head>
     <body>
         <script type="micropython">
+            from xworker import XWorker
             w = XWorker('./matplot.py', **{'type': 'pyodide', 'config': './config.toml'})
 
             # xworker.window made the following completely unnecessary

pyscript.core/test/mocked/micropython.mjs

@@ -1,5 +1,8 @@
 export const python = { content: "", target: null };
 export const loadMicroPython = () => ({
+    registerJsModule() {
+
+    },
     runPython(content) {
         if (document.currentScript?.target) {
             python.content = content;

pyscript.core/test/mocked/pyodide.mjs

@@ -7,6 +7,9 @@ export const setTarget = (value) => {
 export const python = { content: "", target: null, packages: null };
 export const loadPyodide = () => ({
     loadPackage() {},
+    registerJsModule() {
+
+    },
     pyimport() {
         return {
             install(packages) {

pyscript.core/test/plugins/py-script.html

@@ -43,6 +43,7 @@
             display('second <py-script>')
         </py-script>
         <py-script>
+            from xworker import XWorker
             # note this is late to the party simply because
             # pyodide needs to be bootstrapped in the Worker too
             XWorker('../a.py')

pyscript.core/test/worker/index.html

@@ -22,7 +22,7 @@
         <script type="module">
             import { XWorker } from "@pyscript/core";
 
-            const w = new XWorker("./worker.py", { type: "micropython" });
+            const w = new XWorker("./worker.py", { type: "micropython", config: "../fetch.toml" });
             w.postMessage("JavaScript: Hello MicroPython 👋");
             w.onmessage = (event) => {
                 console.log(event.data);
@@ -31,26 +31,32 @@
 
         <!-- XWorker - MicroPython to MicroPython -->
         <script type="micropython">
+            from xworker import XWorker
+
             def handle_message(event):
               print(event.data)
 
-            w = XWorker('./worker.py')
+            w = XWorker('./worker.py', **{'config': '../fetch.toml'})
             w.postMessage('MicroPython: Hello MicroPython 👋')
             w.onmessage = handle_message
         </script>
 
         <!-- XWorker - MicroPython to Pyodide -->
-        <script type="micropython">
+        <script type="pyodide">
+            from xworker import XWorker
+
             def handle_message(event):
               print(event.data)
 
-            w = XWorker('./worker.py', **{'type': 'pyodide', 'async': True, 'config': '../fetch.toml'})
+            w = XWorker('./worker.py', **{'type': 'pyodide', 'async_': True, 'config': '../fetch.toml'})
             w.postMessage('MicroPython: Hello Pyodide 👋')
             w.onmessage = handle_message
         </script>
 
         <!-- XWorker - MicroPython to Lua -->
         <script type="micropython">
+            from xworker import XWorker
+
             def handle_message(event):
               print(event.data)
 

pyscript.core/test/worker/input.html

@@ -22,6 +22,7 @@
     <body>
         <script type="micropython">
             from js import Promise, document
+            from xworker import XWorker
 
             deferred = Promise.withResolvers()
 

pyscript.core/test/worker/input.py

@@ -1,2 +1,4 @@
+from xworker import xworker
+
 print("What is 2 + 3?")
 print("Answer: " + xworker.sync.input("What is 2 + 3?"))

pyscript.core/test/worker/worker.py

@@ -1,4 +1,6 @@
 import re
+import a, b
+from xworker import xworker
 
 
 def on_message(event):