Toshok Electric Light & Manufacturing

Why I'm Ditching CoffeeScript

So, I’m going to try and write this all down as emotionless as possible. So first I’ll tell you exactly what the emotions are:

I’m extremely frustrated about the situation, and am not sure when/if I’ll take up ejs+coffeekit work again at this point. Given that this particular side project was the one thing keeping me feeling sane and productive, I’m also a little down. Don’t really feel like coding at all the moment.

Okay, now that that’s out of the way, here’s some background, followed by the current situation.

(warning: this is a long post. my blog, my dime.)

Background

2011 came with layoffs from Novell/Attachmate. I reverse engineered monotouch, and in my spare time I was working on a couple of games that I wanted to be both browser-based and ios-based. For ios in particular I had some fun ideas for the dual screen ipad-to-appletv support for playing board games (like the tv would show the current state of the world, and the ipad would represent the current user, and give them a view that also contained all their private info). You know, try to bring some of the fun of turn-based online gaming back into a place where people play together. There’d also still be a browser version so people could play along remotely, and of course the ios app would permit remote play as well.

Given all the various places I was going to have to write code, I settled on JavaScript pretty much immediately - It’s a no brainer in the browser, it’s supported enough (I don’t want to get into scaling issues) on the server with node.js, and there are apis and sdk’s (especially now with Parse) that make cloud storage/retrieval of data insanely easy.

The place where JS falls down, in my opinion, is when you start talking about ios. There are a few products out there, but for some reason they seem to revolve around bringing ALL web technology to ios. I don’t want to generate mobile UI’s using HTML and CSS. I also don’t want to involve UIWebView. I don’t want to rely on anything in a browser at all.

I got some initial game logic done but never started on the mobile side of things. I had ideas, but no code.

At this point Xamarin started being an actual thing. For me that ended 5ish months later with broken wreckage where a few friendships used to be. And many fewer frequent drinking partners. And vastly more drinking.

At this point, I started working on the mobile side of things almost excusively.

CoffeeKit

This is where CoffeeKit comes in. It’s essentially what other cross platform mobile propellerheads (i.e. Xamarin) are doing, except it’s JS, not C#. Right from the start I picked coffeescript for the binding language, as it is incredibly terse, especially with the DSL I ended up with. seriously, go look at the bindings They’re beautiful. And they all get compiled into executable JS. The plan was at some point to make that an ahead of time pass so framework registration wouldn’t be something that impacts startup time, but everything was so much in flux, having everything executable yielded HUGE dividends in debugging.

I got coffeekit to a place where I had some nice demos running. It had canvas+webgl support (mapped to a EAGLView along with OpenGLES), it supported XmlHttpWebRequest (built on top of ios’s networking stack), it supported the Web Workers api (using NSThreads), it had typed arrays. It was pretty nice.

I downloaded some J3D demos, some mozilla webgl demos, and put up testflight builds so people could install them on their ipads and watch gold reflective monkey heads rotate, all powered by javascript.

At the time everything but the bindings was closed source. The bindings can be found here along with the sample app here

I switched JS engines partway through, from JavaScriptCore to Spidermonkey, since Spidermonkey’s interpreter was much speedier than JSC for these demos (remember, no JIT on ios), but the use of the interpreter always bugged me. I wanted compiled code. I figured all I had to do was beat the interpreters and it was a win regardless of the engineering outlay (which of course would end up being enormous and ongoing).

Ejs

This is where EJS comes in. EJS is a javascript-to-native (via llvm) compiler written in coffeescript. You put .js files in, you get .js.exe files out that will run on osx or ios depending on the flags you give the compiler.

EJS was also a departure for me from thinking of keeping everything but the bindings closed source. I may at some point in the future try to turn this thing into a product, but any revenue will not be based on the availability of the code. I also chose a liberal license (MPL, although not all files are actually decorated with it), because the GPL pisses me off something fierce (and it should piss you off too, but that’s another discussion.) So I’ve been cannabalizing the original coffeekit code piecemeal and putting it into the open.

Just a week ago I got enough of ejs’s compiler, runtime and binding bugs fixed that I could bring up the original coffeekit demo in the simulator. I’ve never had a motivation problem when it comes to ejs/coffeekit, but this kicked it into pretty high gear. I brought over the webgl code from coffeekit and started massaging things into place, but there was one bug that I kept running into. I would periodically crash in the declaration of a subclass, and even if i didn’t crash, I’d see that getter code was being executed in places it shouldn’t be. I mean this would crash/produce bad output:

1
2
class MyViewController extends uikit.UIViewController
  ...

not when constructing it. not in the new MyViewController(). in the extends

This was due to the fact that in order to present an API that mirrored objective-c as much as possible, I added ES5 properties, both as static properties (on the constructor function in JS), and instance properties (on the prototype). Prototypal properties are fine. Static properties were the cause of the probem.

The above snippet of coffeescript ends up generating a bunch of JS that’s pretty standard, along with this function:

1
 __extends = function(child, parent) { for (var key in parent) child[key] = parent[key]; function ctor() { this.constructor = child; } } ctor.prototype = parent.prototype; child.prototype = new ctor(); child.__super__ = parent.prototype; return child; },

This gets invoked (in this case) as: __extends(MyViewController, _super) where _super is uikit.UIViewController.

Keen-eyed observers will realize that child[key] = parent[key] will work just fine for normal fields (“value properties” in ES5 parlance), but not for properties with getters (“accessor properties”).

The Pull Request

Okay, so we’re back to mostly the present. Two nights ago I made an initial fix and put up a pull request, which was almost immediately rejected due to the fact that coffee-script generated code must run on browsers as far back as IE6. While I would have preferred an initial response a bit more welcoming than a closed pull request, eventually we got things hammered out, the pull request was reopened, and I updated the code with fallbacks so IE6 would continue to use the child[key] = parent[key] code, and newer browsers would copy things in a more accessor property-safe way.

Then it was closed again, presumably with some finality since it came from jashkenas himself.

I have enormous problems with all of the points which were raised as a reason to keep this patch out, and honestly I let myself go a bit with the comments in the pull request. It had been a long day, and having a pull request closed twice in one day after stealing spare moments to work on it throughout the day to address what I still consider to be the greatest technical concern… yeah I’d love for everyone else to say they’d remain totally calm :)

I’ll try and be a bit more calm here. Let’s go over their points one by one:

  • Coffeescript doesn’t support the definition of getters and setters

I love the Coffeescript language (I hope that’s evidenced by my investment in it.) I’m certainly not trying to steer Coffeescript’s syntax. Also, nowhere in the pull request did I ask for this. I agree that they add complexity to the grammar, and they’re also perfectly easy to use without direct language support (which is how CoffeeKit does it).

  • Property accessors in javascript are a very bad addition to JavaScript

People can find ways to write terrible code using any construct you give them. Disallowing accessors to be defined in coffeescript is your call. Refusing to interop with them when they’ve been written in other code is another matter entirely.

  • CoffeeScript makes assumptions about field access being idempotent that is broken by property accessors with side effects.

This is the only technical problem I can see here. The implication in the pull request was that it is one which nobody is interested in fixing, and one for which a fix will not be entertained. It’s a feature, not a bug. These are not the droids you are looking for.

The problem with this attitude is that property accessors are not the only way to introduce side effects which break coffeescript, or at least yield code that doesn’t do what the user expressly wanted. Take:

1
2
3
4
arr = [1,2,3,4,5]
for i in arr
  console.log i
  arr.push 6 if i is 3

That’s another instance of a place where caching arr.length gives what the user might consider invalid output. The relative quality of the code is not something I’m interested in discussing. The point is, it’s valid code. Just doesn’t necessarily do what the user wanted.

But that’s a minor concern - you can write code that fails in the same way in JS (although JS does allow you to fix this by explicitly controlling the caching yourself), and I don’t want to win this argument by pointing out other problems with coffeescript.

This issue is hopefully unique, but is a pretty big reason why CS can’t (and won’t) reliably interop with external code, be it vanilla JS or any other transpiled languages - if you pass to a CoffeeScript function an object whose fields are accessor properties, you might not get your accessors called when you expect, or as many times as you expect.

As a concrete example:

Say you wanted to write a layer over GL in JavaScript and wanted to map everything that started with glGet to properties on the gl object. So instead of glGetError() you’d have gl.Error.

(for those who don’t know, glGetError() returns the current error and also clears it. Direct all outcry to the Khronos Group.)

You’ve just made it impossible to rely on the semantics of the gl.Error getter when used from CoffeeScript. They’re free to remove all but one fetch of the property value, and cache it wherever they want, such that it doesn’t break behavior under the assumption .Error is a value property.

For CoffeeKit specifically, the getters for all accessor properties are totally without side effects. They are getters because they have to wrap access through objective-c selectors. I can’t make them fields. I also can’t in good conscience make them all functions, requiring people to write:

1
uikit.UIScreen.getMainScreen().getBounds()

instead of:

1
uikit.UIScreen.mainScreen.bounds

yuck.

  • They just don’t want interop

This is probably the most frustrating part of the whole thing because in a more inviting atmosphere, I would have done all the remaining work for them. The explicit statement that changes required to interop correctly will not be accepted.

My response

  • CoffeeScript does not exist in a vaccuum.

I challenge everyone to find a single website that uses CoffeeScript for everything in the frontend, without a single additional web framework written in some other language. I’m sure it’s possible. But nobody in their right mind would do it. You use jQuery. You use ember. You use underscore. You use the 50 other frameworks I haven’t heard of. Many of these frameworks support ES5. So you’ll at some point get or create objects with accessor properties. And you want to pass those to your coffeescript module…

Interoperability is important when you’re compiling to a base language that’s shared with other frameworks.

CoffeeScript’s target browser is IE6, which people think is laudable (I did. ultra portable generated code!) But they take it one step further, and require that any code that interacts with coffeescript also targets IE6. Welcome to 2001, wrapped up in different syntax.

  • Property breakage is inconsistent

Turns out the __extends issue is only a problem for properties on constructor functions. Properties on prototypes? No problem! CoffeeScript passes those right along, since it doesn’t manually copy the prototype’s fields.

Nothing the coffeescript maintainers can do up to and including plugging their ears and yelling is going to stop the entire planet from collectively moving to newer JS features. And “explicitly ignoring” properties is doing exactly that.

  • Forks are not the way

Out of all the “you have got to be kidding me” moments I had yesterday, the biggest one came when a fork was suggested. I can understand forks for interesting syntax changes (adding pattern matching, actually adding property setter/getter support, etc), or for new compiler passes. Forks of languages are for language hackers, who have the intention of either modifying the language to create something new or to experiment with “risky” changes that might or might not get merged back into the mainline.

I’m already busy writing something I want people to use. Sure, I can fork CoffeeScript, add my tiny patch, and it’ll work for me - it’ll generate the proper javascript for my bindings and for my test programs and samples.

But then every person who wants to use coffeescript with ejs+coffeekit has to use my fork. I’m suddenly in the language implementation maintainer camp. Sorry, I have way too much else to do. And besides, that isn’t what ejs+coffeekit is about. The whole point of this project is that everything is javascript - you can use any language you want to access the bindings. Now it’s you can use any language except coffeescript. For that you need my fork.

And worse, if they choose to write coffeescript, it will be perfectly valid coffeescript code. There is absolutely no indication at compile time that they’ve done anything wrong when they accidentally use coffee instead of toshokscript. Syntactically they’re identical, it’s just one will break at runtime in baffling ways.

Fragmenting language implementations based on interoperability concerns has to be the worst things I’ve ever heard of.

Where I am now

I’m sitting on a pile of 10,000 lines of coffeescript, encompassing the bindings for coffeekit and the Ejs compiler.

The compiler I’m not worried about. Once I fix self hosting for the umpteenth time people will have a binary they run. They won’t even need node or coffeescript on their system to compile their JS. I will not be (in the near term) moving the compiler away from coffeescript.

The bindings, on the other hand, aren’t workable as coffeescript until/unless this is fixed. And given the overwhelming response, I’m expecting to have to rewrite all of them. Sweet.js is a little young, but maybe it’s time (for all of us) to start contributing to it so I can end up with a DSL that does the right thing and is still compact enough. It’s either that or I write my own binding generator. Neither task makes me smile, but that’s where I am. (before anyone suggests it: Rewriting the bindings directly in JS would be rather painful, because even with the improvements in ES6, the information density of the bindings would drop dramatically and the boilerplate would go way, way up.)

Oh, and one more thing I have to do: Add “CoffeeScript” as the inaugural member of my “List of languages that compile to JavaScript that you can’t use with CoffeeKit”

(I should probably switch names from CoffeeKit too, eh?)

okay, you can now all say “I told you so.”

Comments