Wednesday, July 22, 2015

Rusthon and WebWorkers


Rusthon is the fork of PythonJS that I currently maintain and use on all my personal and work projects. It has an improved JavaScript transpiler, with a powerful syntax that mixes Python and Golang. Recently I added special syntax for timed processing and webworkers, see these posts: [1], [2]

WebWorkers

The WebWorker standard is many years old, but often web developers do not have time to break apart their code in order to use it. Code inside a webworker has no access to the DOM, and passing objects back and forth requires packing them into a message protocol. You also need to worry about loading your shared JavaScript with importScripts, or have some logic to create a URL Blob at runtime, and splitting off the worker code into its own file. In other words, refactoring your JavaScript to leverage webworkers, can be a big pain in the ass.

Rusthon has simple syntax inspired by Golang for using webworkers, see the wiki here. The transpiler and runtime take care of all the details of passing messages, reconstructing objects, splitting apart your code, and generating a URL blob at runtime. Example source code.

transpiler input

This example passes instances of SharedClass back and forth from webworker to the main thread. Python3 style function annotations are used to type the input arguments and return type, this allows the transpiler to inject code to restore the objects __proto__ class|prototype property, this is required when you intend to call methods on objects that cross from webworker to main, or main to webworker.

#backend:javascript
from runtime import *

def show(txt):
 document.getElementById('CONTAINER').appendChild(
  document.createTextNode(txt + '\n')
 )

class SharedClass:
 def __init__(self, x,y,z):
  self.x = x
  self.y = y
  self.z = z

 def foobar(self):
  return self.x + self.y + self.z


with webworker:
 def somefunction() -> SharedClass:
  s = SharedClass(10,20,30)
  return s

 class MyWorker:
  def send(self, obj:SharedClass ) -> SharedClass:
   print obj
   print obj.foobar()
   obj.x = 10
   return obj

  def somemethod(self) -> SharedClass:
   s = SharedClass(100,200,300)
   return s



def main():
 global WORKER
 show('spawn worker...')
 WORKER = spawn( MyWorker() )

 show('creating SharedClass')
 a = SharedClass(1,2,3)
 print(a)
 show(a.foobar())

 show('sending data to worker')
 WORKER <- a

 show('getting data from worker')
 b = <- WORKER
 show(b.foobar())

 c = <- somefunction()
 show(c.foobar())

 d = <- WORKER.somemethod()
 show(d.foobar())

javascript output - webworker

note below: somefunction.returns = "SharedClass"; is the result of this input from above def somefunction() -> SharedClass: that sets the return type of the function. The WebWorker runtime manager checks the returns property of functions it calls, and sets the __proto__ of the result in the main thread.

var somefunction = function()
{
 var s;
 s =  new SharedClass(10, 20, 30);
 return s;
}
somefunction.returns = "SharedClass";

var MyWorker = function(){}

MyWorker.prototype.send = function(obj)
{
 var obj;
 obj.__proto__ = SharedClass.prototype;
 console.log(obj);
 console.log(obj.foobar());
 obj.x = 10;
 return obj;
}
MyWorker.prototype.send.returns = "SharedClass";

MyWorker.prototype.somemethod = function()
{
 var s;
 s =  new SharedClass(100, 200, 300);
 return s;
}
MyWorker.prototype.somemethod.returns = "SharedClass";

javascript output - main

var show = function(txt)
{
 document.getElementById("CONTAINER").appendChild(document.createTextNode((txt + "\n")));
}

var SharedClass = function(x, y, z)
{
 this.__init__(x, y, z);
}

SharedClass.prototype.__init__ = function(x, y, z)
{
 this.x = x;
 this.y = y;
 this.z = z;
}

SharedClass.prototype.foobar = function()
{
 return ((this.x + this.y) + this.z);
}

var main = function()
{
 var a,c,b,d;
 show("spawn worker...");
 WORKER = __workerpool__.spawn({new:"MyWorker", args:[]});
 show("creating SharedClass");
 a =  new SharedClass(1, 2, 3);
 console.log(a);
 show(a.foobar());
 show("sending data to worker");
 __workerpool__.send({message:a,id:WORKER})
 show("getting data from worker");
  __workerpool__.recv( WORKER, function (b) {
  show(b.foobar());
   __workerpool__.call( "somefunction", [],  function (c) {
   show(c.foobar());
    __workerpool__.callmeth( WORKER, "somemethod", [],  function (d) {
    show(d.foobar());
   });
  });
 });
}

Tuesday, May 12, 2015

Rapydscript Kicks Ass


Rapydscript by Alexander Tsepkov is a Python to JavaScript translator that really kicks ass. It is stable, the transpiler can be run fully client side, produces very fast JavaScript, and is well maintained by Alex. Rapydscript syntax is very similar to PythonJS, and both can be used together in the same markdown file compiled by Rusthon.

Rapydscript has the right mix of Pythonic-style and features to make it inter-operate well with JavaScript. Alex made all the right choices on what parts of Python to throw away, and adapting the syntax to the requirements imposed by generating fast JavaScript. Check out this example for how to use Rapydscript in Rusthon.

Sunday, September 7, 2014

Python to Go


Golang is a fully typed language with simple and clean syntax. The Go compiler will throw an error if you made a typo or called a method on the wrong object. This means you need to worry less about runtime errors, and writting unit tests. Go programs are compiled to native executables, or can be compiled to JavaScript using GopherJS.

PythonJS has a new backend in development that translates to Go. See the documentation here. The Go backend supports translation of Python classes and list comprehensions to Go. The PythonJS syntax has also been extended to support Go features like: go-routines and channels.

Go/Python Meetup

The Barcelona Python users group is hosting a free meetup on Oct 13th, see the details here. I will be going and giving a talk about using PythonJS to translate to Go, and using GopherJS to translate it back to JavaScript.

Tuesday, July 22, 2014

new pythonjs syntax


PythonJS is being extended with optional static typing and new syntax. The translator pre-processes input source code and transforms it into an intermediate form that can be parsed by Python's ast module. This simplifies adding new syntax to PythonJS, see this commit that implements Exception Expressions.

a = d[ 'somekey' ] except KeyError: 'mydefault'

inline anonymous functions

I am now researching new syntax that can be added to PythonJS. After looking at RapydScript, I see that inline anonymous functions are a must have, and recently implemented them in PythonJS. Inline functions can be used in function calls as a named keyword argument, or in a dict literal. This solves the problem that Lambda functions can only be a single statement, which is often not that useful.


a.func(
 callback1=def (x,y,z):
  x += y
  return x - z,
 callback2= def (x, y):
  return x * y
)

b = {
 'cb1' : def (x):
  return x,
 'cb2' : def (y):
  return y
}

switch


switch a==b:
  case True:
    pass
  case False:
    pass
  default:
    pass

Go channels and select

def send_data( A:chan int, B:chan int, X:int, Y:int):
 while True:
  print('sending data..')
  A <- X
  B <- Y

def select_loop(A:chan int, B:chan int, W:chan int) -> int:
 print('starting select loop')
 y = 0
 while True:
  print('select loop:',y)
  select:
   case x = <- A:
    y += x
    W <- y
   case x = <- B:
    y += x
    W <- y
 print('end select loop', y)
 return y

def main():
 a = go.channel(int)
 b = go.channel(int)
 w = go.channel(int)

 go(
  select_loop(a,b, w)
 )


 go(
  send_data(a,b, 5, 10)
 )

 z = 0
 while z < 100:
  z = <- w
  print('main loop', z)

 print('end test')

Thursday, July 17, 2014

WebGL CSS3D Hybrid part2


Despite WebGL having been around since 2011, you probably do not run into many websites with 3D. And when you do, its likely a fullscreen game or demo, not a typical website. Traditional web technologies do not mix well with WebGL. Libraries like Voodoo.js try to bridge this gap by allowing you to insert 3D objects inside a normal webpage. Voodoo.js graphics are not anti-aliased when rendering with WebGL, the contrast with the rest of the smooth text and graphics on a webpage then becomes very harsh and downright ugly.

Threepy is an experimental library built on THREE.js that allows some HTML and CSS to be mixed together with WebGL rendering and post-processing FX. The library wraps many standard HTML element types with special hacks like: callbacks, catching events, and rendering to proxy clones. Not every combination of HTML and CSS is compatible with this method. The video below tests the basic UI elements: checkboxes, text input, select drop down lists, number input, color selector, tab menus, slider controls, images and videos. Works with GoogleChrome and NodeWebkit.

Monday, July 7, 2014

WebGL CSS3D Hybrid


Jerome Etienne has an interesting post where he combines CSS3D and WebGL, see his post here. However, his method is not compatible with Three.js post-processing shaders that is required to blur and perform other 2D effects on the 3D rendering.

The videos below show integration of WebGL and CSS3D with post-processing in THREE.js. The main hack do to this requires double rendering the DOM in two CSS3D layers. The first layer is rendered under the WebGL layer, and is a clone of the top CSS layer. The top CSS layer is rendered transparent over the WebGL layer and catches keyboard and mouse events. The WebGL layer is rendered with alpha blending. See the source code here.

dnd, iframes, video, collada

Update july10th, testing new hacks to integrate: iframes, html5 video, and collada 3D files by drag and drop. Starting new module dddom.py with simple API for creating 3D widgets mixing WebGL and CSS3D.

Tuesday, June 24, 2014

GLSL Classes


GLSL is not object oriented, there are no classes or method calls. GLSL4.0 has subroutine variables that can provide an object oriented solution, but this is not compatible with GLSL in WebGL.

PythonJS GPU Class

In the example below an array of MyObject is created and uploaded to the GPU, where it is iterated over. The method call s.mymethod(1.1, 2.2) translates to MyObject_mymethod(s, 1.1, 2.2) in the GLSL shader.

@gpu.object
class MyObject:
 @gpu.method
 float def subroutine(self, x,y):
  float x
  float y
  return x + y * self.attr2

 @gpu.method
 float def mymethod(self, x,y):
  float x
  float y
  if self.index == 0:
   return -20.5
  elif self.index == 0:
   return 0.6
  else:
   return self.subroutine(x,y) * self.attr1

 def __init__(self, a, b, i):
  self.attr1 = a
  self.attr2 = b
  self.index = int16(i)


class myclass:
 def run(self, w):
  self.array = [MyObject(1.1,1.2,x) for x in range(w)]

  @returns( array=64 )
  @gpu.main
  def gpufunc():
   struct* A = self.array
   float b = 0.0

   for s in iter(A):
    b += s.mymethod(1.1, 2.2)

   return b

  return gpufunc()