The Design of Greasemonkey 1.0

This is a long, deep, technical post.  The goal is to share Greasemonkey's history, to help explain how the decisions made in Greasemonkey 1.0 were reached.  Hopefully if you have any interest in user scripting, and especially if you are a script author, you'll find it enlightening.

Greasemonkey is at least seven and a half years old (version 0.2.5 was released on March 28, 2005; version 0.2 seems to date from January 14th, version 0.1 from way back on December 6th, 2004).  Greasemonkey in particular and user scripting in general, not to mention the browser platforms they rely upon, have come a long way in the past eight years or so.

If you examine the version history of Greasemonkey, you'll see that core APIs like GM_xmlhttpRequest and GM_getValue/GM_setValue were added very early on.  They've helped define a lot of what user scripting is: a quick and simple way to fix/enhance existing web sites with (generally) short and simple javascript programs.  These APIs were simple by design, yet more powerful than what standard javascript (in a web page) can do.

One of the early great scripts was Book Burro, which allowed users to compare book prices at Amazon, Barnes & Noble, and others, right on the existing web page.  This was possible only because GM_xmlhttpRequest allowed the user script to break the same origin policy that would be applied to a normal XMLHTTPRequest call made by a web page.  The ability to store persistent values and use them at next run also enabled things like adding a search history to GMail.

These features aren't provided to normal scripts on web pages, because they can be dangerous.  They were provided to scripts, because the user has much more control over which scripts they choose to install.  Links clicked online can point to any web site, with any potentially malicious goal, so they're much harder to control ahead of time.  It was in July of 2005 that Mark Pilgrim pointed out the giant unfortunate security hole in the way that Greasemonkey gave user scripts access to these more powerful APIs, which made it possible for malicious pages to access and use them.

This is when Greasemonkey's security sandbox came into being.  Greasemonkey version 0.3 was public at the time that this security problem became evident.  Version 0.4 was in development, but was scrapped to build 0.5 which contained the security fix, later in the same month.  It was right at this time that Mozilla was implementing XPCNativeWrappers into Firefox.  This formed the core of the execution model that Greasemonkey has used ever since, until the release of Greasemonkey 1.0, a span of over seven years.  An entire article, at least this long, could be written just on the repercussions of this change.  In short, as the linked page says, "XPCNativeWrapper is a way to wrap up an object so that it's safe to access from privileged code."  In this case the privileged code is the user script (with the APIs that Greasemonkey grants it), and the wrapper makes it "safe to access" the untrusted web page with potentially unsafe javascript code.

The Greasemonkey Hacks O'Reilly book which was in progress at the time needed to be completely rewritten, as many of the scripts it contained would no longer function properly.  As part of this rewrite, the Avoid Common Pitfalls section was added, based on the author's insight from reading and re-writing dozens of scripts to deal with the issues this security focused change caused.  (And the pitfalls commonly tripped up script authors.)

This was the only intentional backwards-incompatible change (anything else was a bug, and fixed) in Greasemonkey's eight year history, until version 1.0.  I bring this up primarily to stress the point that we really do care about maintaining compatibility.  We broke it again in version 1.0, and not lightly, but rather because after years of experience (and changes around us), we decided it was the right thing to do.

But why?  Partially because we think it's a very small break.  Few scripts should be affected.  But also, go back and read that common pitfalls article.  It's long and complicated.  It relates to a number of things that may be completely natural, for experienced javascript programmers, and especially for new ones just trying to follow tutorials.  Time and time again, script authors would come to us with problems, saying that the script works fine in a web page (or the Firebug console, or similar), but fails in Greasemonkey.  And fails mysteriously.  Usually no error is given, sometimes something cryptic, but the answer is never obvious unless you know it already.  This makes developing scripts hard, which is not awesome.

On top of the broken features, the security restrictions mean that interacting with javascript on the page is harder.  The security restrictions intentionally isolate the script in the page (content scope) from the user script (script scope).  But when the script tries to alter the content scope, it can't.  There are plenty examples of authors trying this, and failing.  Then there's the location hack and reading content globals workarounds.  And the setTimeout access violation workaround for when content callbacks trigger script functions.  And the pitfalls article.  But you need to know both that these problems, and their workarounds, exist in order to use them!

Let's not forget unsafeWindow.  This beast was created back when the legacy security sandbox was added.  Some times, you just need to poke at the content scope.  With the legacy sandbox, sometimes unsafeWindow is the only way to do this.  But that name wasn't picked lightly.  The unsafeWindow object is unsafe!  Back in 2008, I figured out a way around the security sandbox, for any script that used unsafeWindow for any purpose.  That particular hole has been patched, and we're not aware of any others.  But that doesn't mean they don't exist.

Unfortunately, usage of unsafeWindow is extremely widespread.  My API usage survey showed it to be the most commonly used Greasemonkey feature, where using none at all was the only more common case.  Nearly one of five scripts use it.  And there's always the chance that it could cause real security or privacy issues, through some avenue which we simply aren't aware of yet.  If so any site that a script accesses via unsafeWindow could gain all of the Greasemonkey APIs.  Which means GM_xmlhttpRequest, and the ability to interact with any site you're logged into, whether it's Facebook or your bank.

The whole problem boils down to the security restrictions that keep the content scope and script scope both separated and different from each other.  The big idea in Greasemonkey 1.0 was to change this model.  I discussed something like this in 2009 and Johan brought it up again in 2011 with some distinct details.  In short, web browsers and javascript have come a long way since 2004.  There are features like DOM Storage and postMessage, and all the things popularly labeled as "HTML5".  Scripts can use these features, rather than the Greasemonkey specific ones.  Even more importantly, though, a majority of scripts don't use any special Greasemonkey APIs.  Yet they bear all the compatibility burdens that the security sandbox imparts while protecting these unused APIs.

So the goal of Greasemonkey 1.0 was to make it possible for the most common class of scripts, with no need for special APIs, to work just like a content script, with all the new features that the browser supports, with all the standard methods javascript programmers are used to (including jQuery), without breaking anything.  And none of the mysterious failure modes.

The way we chose to do this is to make the APIs an opt-in feature, rather than on by default for everybody like they always were before.  This is what the @grant metadata does.  It says "I really need it, so I want this special API for my script" and, as a result, I'm willing to deal with the limitations of the security sandbox it will be put into.

But for the past eight years, this hasn't existed, so nobody has used it.  What about all the scripts written in that time?  In Greasemonkey 1.0, simple content sniffing is performed on the script.  If it looks like it calls an API function, then Greasemonkey acts like the appropriate @grant line was given, even if it wasn't.  If you want the new execution mode where everything (besides the privileged APIs) works for sure, you simply specify @grant none in your metadata.

This mode also puts you much closer to the content scope.  From the script scope, you can read any content scope variable simply by using its name.  No reading content globals workaround.  And you can even write to the content scope, simply by addressing it as "window.thing = value".  No location hack necessary.  (In more javascript-y terms, window is the content global scope; in the script this is the script global scope.  And this has window as its prototype.)

Examining every existing script hosted at userscripts.org shows that not a single one does this (assigning a value to a property of window) today (so they won't get different broken semantics).  Unfortunately, jQuery does, so it needs to be fixed.  But the fix is the standard and documented jQuery feature for such a case.  On the other hand, if the page you run on already includes jQuery, then you can just use it!  You have direct read access to the global content scope!

Eventually, @grant none will become the default.  At that point, we will stop sniffing and faking grants that were not specified.  And at that point you will be required to specify the grant for any API that you do still actually want to use.  Only time will tell when we're comfortable enough making that change.

Finally, don't forget about the grant-none-shim.user.js, which provides an emulation layer for most of the Greasemonkey APIs on top of standard browser features.  If you (for example) use getValue/setValue or xmlhttpRequest, but only on a single origin, you can switch to this and get the thin sandbox goodness of a grant none style script, and everything will generally work as it always did, at little to no cost.  Just note that existing stored values will no longer be readable.