run on isolate

CHANGELOG.md

@@ -3,8 +3,12 @@
  * @Author: ekibun
  * @Date: 2020-08-08 08:16:50
  * @LastEditors: ekibun
- * @LastEditTime: 2020-09-21 23:19:02
+ * @LastEditTime: 2020-10-03 00:28:18
 -->
+## 0.1.1
+
+* run on isolate.
+
 ## 0.1.0
 
 * refactor with ffi.

README.md

@@ -3,7 +3,7 @@
  * @Author: ekibun
  * @Date: 2020-08-08 08:16:50
  * @LastEditors: ekibun
- * @LastEditTime: 2020-09-22 00:03:48
+ * @LastEditTime: 2020-10-03 00:36:36
 -->
 # flutter_qjs
 
@@ -17,7 +17,7 @@ Event loop of `FlutterQjs` should be implemented by calling `FlutterQjs.dispatch
 
 ES6 module with `import` function is supported and can be managed in dart with `setModuleHandler`.
 
-A global function `convert` is presented to invoke dart function. Data conversion between dart and js are implemented as follow:
+A global function `channel` is presented to invoke dart function. Data conversion between dart and js are implemented as follow:
 
 | dart | js |
 | --- | --- |
@@ -35,6 +35,8 @@ A global function `convert` is presented to invoke dart function. Data conversio
 
 ## Getting Started
 
+### Run on main thread
+
 1. Create a `FlutterQjs` object. Call `dispatch` to dispatch event loop.
 
 ```dart
@@ -42,7 +44,7 @@ final engine = FlutterQjs();
 await engine.dispatch();
 ```
 
-2. Call `setMethodHandler` to implement `dart` interaction. For example, you can use `Dio` to implement http in js:
+2. Call `setMethodHandler` to implement js-dart interaction. For example, you can use `Dio` to implement http in js:
 
 ```dart
 await engine.setMethodHandler((String method, List arg) {
@@ -55,15 +57,17 @@ await engine.setMethodHandler((String method, List arg) {
 });
 ```
 
-and in javascript, call `convert` function to get data, make sure the second memeber is a list:
+and in javascript, call `channel` function to get data, make sure the second parameter is a list:
 
 ```javascript
-convert("http", ["http://example.com/"]);
+channel("http", ["http://example.com/"]);
 ```
 
 3. Call `setModuleHandler` to resolve the js module.
 
-**important:** I cannot find a way to convert the sync ffi callback into an async function. So the assets files received by async function `rootBundle.loadString` cannot be used in this version. I will appreciate it if you can provide me a solution to make `ModuleHandler` async.
+~~I cannot find a way to convert the sync ffi callback into an async function. So the assets files received by async function `rootBundle.loadString` cannot be used in this version. I will appreciate it if you can provide me a solution to make `ModuleHandler` async.~~
+
+To use async function in module handler, try [Run on isolate thread](#isolate)
 
 ```dart
 await engine.setModuleHandler((String module) {
@@ -90,6 +94,56 @@ try {
 
 5. Method `recreate` can destroy quickjs runtime that can be recreated again if you call `evaluate`, `recreat` can be used to reset the module cache. Call `close` to stop `dispatch` when you do not need it.
 
+### <span id="isolate">Run on isolate thread</span>
+
+1. Create a `IsolateQjs` object, pass a handler to implement js-dart interaction. The handler is used in isolate, so the function must be a top-level function or a static method.
+
+```dart
+dynamic methodHandler(String method, List arg) {
+  switch (method) {
+    case "http":
+      return Dio().get(arg[0]).then((response) => response.data);
+    default:
+      throw Exception("No such method");
+  }
+}
+final engine = IsolateQjs(methodHandler);
+// not need engine.dispatch();
+```
+
+and in javascript, call `channel` function to get data, make sure the second parameter is a list:
+
+```javascript
+channel("http", ["http://example.com/"]);
+```
+
+2. Call `setModuleHandler` to resolve the js module. Async function such as `rootBundle.loadString` can be used now to get module. The handler is called in main thread.
+
+```dart
+await engine.setModuleHandler((String module) async {
+  return await rootBundle.loadString(
+      "js/" + module.replaceFirst(new RegExp(r".js$"), "") + ".js");
+});
+```
+
+and in javascript, call `import` function to get module:
+
+```javascript
+import("hello").then(({default: greet}) => greet("world"));
+```
+
+3. Same as run on main thread, use `evaluate` to run js script:
+
+```dart
+try {
+  print(await engine.evaluate(code ?? '', "<eval>"));
+} catch (e) {
+  print(e.toString());
+}
+```
+
+4. Method `close` (same as `recreate` in main thread) can destroy quickjs runtime that can be recreated again if you call `evaluate`.
+
 [This example](example/lib/main.dart) contains a complete demonstration on how to use this plugin.
 
 ## For Mac & IOS developer

example/js/hello.js

@@ -0,0 +1,8 @@
+/*
+ * @Description: module example
+ * @Author: ekibun
+ * @Date: 2020-10-03 00:29:45
+ * @LastEditors: ekibun
+ * @LastEditTime: 2020-10-03 00:32:37
+ */
+export default (name) => `hello ${name}!`;

example/lib/main.dart

@@ -3,14 +3,14 @@
  * @Author: ekibun
  * @Date: 2020-08-08 08:16:51
  * @LastEditors: ekibun
- * @LastEditTime: 2020-09-21 23:54:55
+ * @LastEditTime: 2020-10-03 00:38:41
  */
 import 'package:flutter/material.dart';
 import 'dart:typed_data';
 
 import 'package:dio/dio.dart';
 import 'package:flutter/services.dart';
-import 'package:flutter_qjs/flutter_qjs.dart';
+import 'package:flutter_qjs/isolate.dart';
 
 import 'highlight.dart';
 
@@ -44,41 +44,43 @@ class TestPage extends StatefulWidget {
   State<StatefulWidget> createState() => _TestPageState();
 }
 
+dynamic methodHandler(String method, List arg) {
+  switch (method) {
+    case "http":
+      return Dio().get(arg[0]).then((response) => response.data);
+    case "test":
+      return arg[0]([
+        true,
+        1,
+        0.5,
+        "str",
+        {"key": "val", 0: 1},
+        Uint8List(2),
+        Int32List(2),
+        Int64List(2),
+        Float64List(2),
+        Float32List(2)
+      ]);
+    default:
+      throw Exception("No such method");
+  }
+}
+
 class _TestPageState extends State<TestPage> {
   String resp;
-  FlutterQjs engine;
+  IsolateQjs engine;
 
-  CodeInputController _controller = CodeInputController();
+  CodeInputController _controller = CodeInputController(
+      text: 'import("hello").then(({default: greet}) => greet("world"));');
 
-  _createEngine() async {
+  _ensureEngine() {
     if (engine != null) return;
-    engine = FlutterQjs();
+    engine = IsolateQjs(methodHandler);
-    engine.setMethodHandler((String method, List arg) {
+    engine.setModuleHandler((String module) async {
-      switch (method) {
+      if (module == "test") return "export default '${new DateTime.now()}'";
-        case "http":
+      return await rootBundle.loadString(
-          return Dio().get(arg[0]).then((response) => response.data);
+          "js/" + module.replaceFirst(new RegExp(r".js$"), "") + ".js");
-        case "test":
-          return arg[0]([
-            true,
-            1,
-            0.5,
-            "str",
-            {"key": "val", 0: 1},
-            Uint8List(2),
-            Int32List(2),
-            Int64List(2),
-            Float64List(2),
-            Float32List(2)
-          ]);
-        default:
-          throw Exception("No such method");
-      }
     });
-    engine.setModuleHandler((String module) {
-      if (module == "hello") return "export default '${new DateTime.now()}'";
-      return "Module Not found";
-    });
-    engine.dispatch();
   }
 
   @override
@@ -96,15 +98,10 @@ class _TestPageState extends State<TestPage> {
               scrollDirection: Axis.horizontal,
               child: Row(
                 children: [
-                  FlatButton(
-                      child: Text("create engine"), onPressed: _createEngine),
                   FlatButton(
                       child: Text("evaluate"),
                       onPressed: () async {
-                        if (engine == null) {
+                        _ensureEngine();
-                          print("please create engine first");
-                          return;
-                        }
                         try {
                           resp = (await engine.evaluate(
                                   _controller.text ?? '', "<eval>"))

example/pubspec.lock

@@ -82,7 +82,7 @@ packages:
       path: ".."
       relative: true
     source: path
-    version: "0.1.0"
+    version: "0.1.1"
   flutter_test:
     dependency: "direct dev"
     description: flutter

example/pubspec.yaml

@@ -40,8 +40,8 @@ flutter:
   uses-material-design: true
 
   # To add assets to your application, add an assets section, like this:
-  # assets:
+  assets:
-  #   - images/a_dot_burr.jpeg
+    - js/
   #   - images/a_dot_ham.jpeg
 
   # An image asset can refer to one or more resolution-specific "variants", see

lib/flutter_qjs.dart

@@ -3,7 +3,7 @@
  * @Author: ekibun
  * @Date: 2020-08-08 08:29:09
  * @LastEditors: ekibun
- * @LastEditTime: 2020-09-27 01:08:14
+ * @LastEditTime: 2020-10-03 00:18:49
  */
 import 'dart:async';
 import 'dart:ffi';
@@ -61,7 +61,7 @@ class FlutterQjs {
     _ctx = jsNewContextWithPromsieWrapper(_rt);
   }
 
-  /// Set a handler to manage js call with `dart(method, ...args)` function.
+  /// Set a handler to manage js call with `channel(method, args)` function.
   setMethodHandler(JsMethodHandler handler) {
     methodHandler = handler;
   }

lib/isolate.dart

@@ -0,0 +1,247 @@
+/*
+ * @Description: 
+ * @Author: ekibun
+ * @Date: 2020-10-02 13:49:03
+ * @LastEditors: ekibun
+ * @LastEditTime: 2020-10-03 00:18:40
+ */
+import 'dart:async';
+import 'dart:ffi';
+import 'dart:io';
+import 'dart:isolate';
+
+import 'package:ffi/ffi.dart';
+import 'package:flutter_qjs/flutter_qjs.dart';
+import 'package:flutter_qjs/wrapper.dart';
+
+class IsolateJSFunction {
+  int val;
+  int ctx;
+  SendPort port;
+  IsolateJSFunction(this.ctx, this.val, this.port);
+
+  Future<dynamic> invoke(List<dynamic> arguments) async {
+    if (0 == val ?? 0) return;
+    var evaluatePort = ReceivePort();
+    port.send({
+      'type': 'call',
+      'ctx': ctx,
+      'val': val,
+      'args': _encodeData(arguments),
+      'port': evaluatePort.sendPort,
+    });
+    var result = await evaluatePort.first;
+    if (result['data'] != null)
+      return _decodeData(result['data'], port);
+    else
+      throw result['error'];
+  }
+
+  @override
+  noSuchMethod(Invocation invocation) {
+    return invoke(invocation.positionalArguments);
+  }
+}
+
+dynamic _encodeData(dynamic data, {Map<dynamic, dynamic> cache}) {
+  if (cache == null) cache = Map();
+  if (cache.containsKey(data)) return cache[data];
+  if (data is List) {
+    var ret = [];
+    cache[data] = ret;
+    for (int i = 0; i < data.length; ++i) {
+      ret.add(_encodeData(data[i], cache: cache));
+    }
+    return ret;
+  }
+  if (data is Map) {
+    var ret = {};
+    cache[data] = ret;
+    for (var entry in data.entries) {
+      ret[_encodeData(entry.key, cache: cache)] =
+          _encodeData(entry.value, cache: cache);
+    }
+    return ret;
+  }
+  if (data is JSFunction) {
+    return {
+      '__js_function_ctx': data.ctx.address,
+      '__js_function_val': data.val.address,
+    };
+  }
+  if (data is IsolateJSFunction) {
+    return {
+      '__js_function_ctx': data.ctx,
+      '__js_function_val': data.val,
+    };
+  }
+  if (data is Future) {
+    // Not support
+    return {};
+  }
+  return data;
+}
+
+dynamic _decodeData(dynamic data, SendPort port,
+    {Map<dynamic, dynamic> cache}) {
+  if (cache == null) cache = Map();
+  if (cache.containsKey(data)) return cache[data];
+  if (data is List) {
+    var ret = [];
+    cache[data] = ret;
+    for (int i = 0; i < data.length; ++i) {
+      ret.add(_decodeData(data[i], port, cache: cache));
+    }
+    return ret;
+  }
+  if (data is Map) {
+    if (data.containsKey('__js_function_val')) {
+      int ctx = data['__js_function_ctx'];
+      int val = data['__js_function_val'];
+      if (port != null) {
+        return IsolateJSFunction(ctx, val, port);
+      } else {
+        return JSFunction.fromAddress(ctx, val);
+      }
+    }
+    var ret = {};
+    cache[data] = ret;
+    for (var entry in data.entries) {
+      ret[_decodeData(entry.key, port, cache: cache)] =
+          _decodeData(entry.value, port, cache: cache);
+    }
+    return ret;
+  }
+  return data;
+}
+
+void _runJsIsolate(Map spawnMessage) async {
+  var qjs = FlutterQjs();
+  SendPort sendPort = spawnMessage['port'];
+  JsMethodHandler methodHandler = spawnMessage['handler'];
+  ReceivePort port = ReceivePort();
+  sendPort.send(port.sendPort);
+  qjs.setMethodHandler(methodHandler);
+  qjs.setModuleHandler((name) {
+    var ptr = allocate<Int64>();
+    sendPort.send({
+      'type': 'module',
+      'name': name,
+      'ptr': ptr.address,
+    });
+    ptr.value = 0;
+    while (ptr.value == 0) sleep(Duration.zero);
+    print(ptr.value);
+    if (ptr.value == -1) throw Exception("Module Not found");
+    var strptr = Pointer<Utf8>.fromAddress(ptr.value);
+    var ret = Utf8.fromUtf8(strptr);
+    return ret;
+  });
+  qjs.dispatch();
+  await for (var msg in port) {
+    var data;
+    SendPort msgPort = msg['port'];
+    try {
+      switch (msg['type']) {
+        case 'evaluate':
+          data = await qjs.evaluate(msg['command'], msg['name']);
+          break;
+        case 'call':
+          data = JSFunction.fromAddress(
+            msg['ctx'],
+            msg['val'],
+          ).invoke(_decodeData(msg['args'], null));
+          break;
+        case 'close':
+          qjs.close();
+          port.close();
+          break;
+      }
+      if (msgPort != null)
+        msgPort.send({
+          'data': _encodeData(data),
+        });
+    } catch (e, stack) {
+      if (msgPort != null)
+        msgPort.send({
+          'error': e.toString() + "\n" + stack.toString(),
+        });
+    }
+  }
+}
+
+typedef JsAsyncModuleHandler = Future<String> Function(String name);
+typedef JsIsolateSpawn = void Function(SendPort sendPort);
+
+class IsolateQjs {
+  SendPort _sendPort;
+  JsMethodHandler _methodHandler;
+  JsAsyncModuleHandler _moduleHandler;
+
+  /// Set a handler to manage js call with `channel(method, args)` function.
+  /// The function must be a top-level function or a static method
+  IsolateQjs(this._methodHandler);
+
+  Future<void> _ensureEngine() async {
+    if (_sendPort != null) return;
+    ReceivePort port = ReceivePort();
+    Isolate.spawn(
+      _runJsIsolate,
+      {
+        'port': port.sendPort,
+        'handler': _methodHandler,
+      },
+      errorsAreFatal: true,
+    );
+    var completer = Completer();
+    port.listen((msg) async {
+      if (msg is SendPort && !completer.isCompleted) {
+        _sendPort = msg;
+        completer.complete();
+        return;
+      }
+      switch (msg['type']) {
+        case 'module':
+          var ptr = Pointer<Int64>.fromAddress(msg['ptr']);
+          try {
+            ptr.value = Utf8.toUtf8(await _moduleHandler(msg['name'])).address;
+          } catch (e) {
+            ptr.value = -1;
+          }
+          break;
+      }
+    }, onDone: () {
+      close();
+      if (!completer.isCompleted) completer.completeError('isolate close');
+    });
+    await completer.future;
+  }
+
+  /// Set a handler to manage js module.
+  setModuleHandler(JsAsyncModuleHandler handler) {
+    _moduleHandler = handler;
+  }
+
+  close() {
+    _sendPort.send({
+      'type': 'close',
+    });
+    _sendPort = null;
+  }
+
+  Future<dynamic> evaluate(String command, String name) async {
+    await _ensureEngine();
+    var evaluatePort = ReceivePort();
+    _sendPort.send({
+      'type': 'evaluate',
+      'command': command,
+      'name': name,
+      'port': evaluatePort.sendPort,
+    });
+    var result = await evaluatePort.first;
+    if (result['data'] != null)
+      return _decodeData(result['data'], _sendPort);
+    else
+      throw result['error'];
+  }
+}

lib/wrapper.dart

@@ -3,7 +3,7 @@
  * @Author: ekibun
  * @Date: 2020-09-19 22:07:47
  * @LastEditors: ekibun
- * @LastEditTime: 2020-09-24 13:38:08
+ * @LastEditTime: 2020-10-02 16:37:16
  */
 import 'dart:async';
 import 'dart:ffi';
@@ -22,6 +22,11 @@ class JSRefValue implements JSRef {
     runtimeOpaques[rt]?.ref?.add(this);
   }
 
+  JSRefValue.fromAddress(int ctx, int val) {
+    this.ctx = Pointer.fromAddress(ctx);
+    this.val = Pointer.fromAddress(val);
+  }
+
   @override
   void release() {
     if (val != null) {
@@ -65,10 +70,11 @@ class JSPromise extends JSRefValue {
 class JSFunction extends JSRefValue {
   JSFunction(Pointer ctx, Pointer val) : super(ctx, val);
 
-  @override
+  JSFunction.fromAddress(int ctx, int val) : super.fromAddress(ctx, val);
-  noSuchMethod(Invocation invocation) {
+
+  invoke(List<dynamic> arguments) {
     if (val == null) return;
-    List<Pointer> args = invocation.positionalArguments
+    List<Pointer> args = arguments
         .map(
           (e) => dartToJs(ctx, e),
         )
@@ -85,6 +91,11 @@ class JSFunction extends JSRefValue {
     }
     return ret;
   }
+
+  @override
+  noSuchMethod(Invocation invocation) {
+    return invoke(invocation.positionalArguments);
+  }
 }
 
 Pointer jsGetPropertyStr(Pointer ctx, Pointer val, String prop) {

pubspec.yaml

@@ -1,6 +1,6 @@
 name: flutter_qjs
 description: This plugin is a simple js engine for flutter using the `quickjs` project. Plugin currently supports all the platforms except web!
-version: 0.1.0
+version: 0.1.1
 homepage: https://github.com/ekibun/flutter_qjs
 
 environment:

test/flutter_qjs_test.dart

@@ -3,15 +3,19 @@
  * @Author: ekibun
  * @Date: 2020-09-06 13:02:46
  * @LastEditors: ekibun
- * @LastEditTime: 2020-09-24 22:55:33
+ * @LastEditTime: 2020-10-02 17:27:52
  */
 import 'dart:convert';
 import 'dart:io';
 
-import 'package:flutter_qjs/ffi.dart';
+import 'package:flutter_qjs/isolate.dart';
-import 'package:flutter_qjs/flutter_qjs.dart';
 import 'package:flutter_test/flutter_test.dart';
 
+dynamic myMethodHandler(method, args) {
+  print([method, args]);
+  return args;
+}
+
 void main() async {
   test('make.windows', () async {
     final utf8Encoding = Encoding.getByName('utf-8');
@@ -51,28 +55,21 @@ void main() async {
     expect(result.exitCode, 0);
   }, testOn: 'mac-os');
   test('jsToDart', () async {
-    final qjs = FlutterQjs();
+    final qjs = IsolateQjs(myMethodHandler);
-    qjs.setMethodHandler((method, args) {
+    qjs.setModuleHandler((name) async {
-      print([method, args]);
-      return args;
-    });
-    qjs.setModuleHandler((name) {
       print(name);
       return "export default '${new DateTime.now()}'";
     });
-    qjs.evaluate("""
+    var value = await qjs.evaluate("""
       const a = {};
       a.a = a;
       import("test").then((module) => channel('channel', [
-          (...a)=>`hello \${a}`,
+          (...args)=>`hello \${args}!`, a,
           0.1, true, false, 1, "world", module
         ]));
-      """, "<eval>").then((value) {
+      """, "<eval>");
-      print(value);
+    print(value);
-    });
+    print(await value[0]('world'));
-    Future.delayed(Duration(seconds: 5)).then((v) {
+    qjs.close();
-      qjs.close();
-    });
-    await qjs.dispatch();
   });
 }