The four missing megabytes of Firefox
Hacking Firefox with Experimental APIs.
In our previous article, we explained why the Cliqz Browser is based on Mozilla Firefox. There we also briefly mentioned that our desktop browser is a Firefox fork with the Cliqz Extension on top of it, which carries most of the browser’s features. In this post, we will talk more about this extension, its history and one particular technology which gives it capabilities beyond what is allowed by the usual APIs exposed by browsers.
Don’t change, extend
To some of you, the idea of having almost all the functionality implemented in the extension, rather than directly in the browser, may sound odd. Let us lay out some of the benefits of this architecture as compared to a more traditional browser-based approach:
Extensions are lightweight and can be updated separately from the browser, which means that we can deliver updates and critical fixes much faster. At the time of writing this post, Cliqz Extension weighs about 4 megabytes, which is 20 times less than the the Cliqz Browser. It also simplifies the development process as testing changes requires only an extension update. That allows us to ship beta builds to all testers multiple times a day. This, in turn, allows developers to get the feedback from testers and product owners quicker.
Extensions updates require no browser restart. Users get new features without being interrupted in their workflow.
It is much easier to onboard new contributors. Compare writing the code in Javascript using well-documented (e.g. WebExtension) and standardized (e.g. DOM) APIs to making changes in the several-gigabytes big codebase of pretty diverse technostack (C++/Rust/Javascript/you name it), where the only documentation available is often from source code comments. Even with our complicated toolchain it only takes seconds to rebuild the extension and reload it live in the browser, while compiling the whole browser would take minutes, and would require a relaunch to see the changes made.
Finally, it minimizes changes to the browser’s code, allowing us to merge with Firefox upstream with more ease[1]. This would otherwise be a constant source of struggle for a small team like ours.
It would not be accurate, however, to say that four years ago, when we started the Cliqz Browser we really considered all these points and made a conscious decision based on that. It was more like we discovered them post factum. The real reasons behind this decision, as it usually goes, were historical.
A brief history of the Cliqz Extension
As we mentioned in one of our recent posts, the main feature of our browser, the search-as-you-type dropdown, was initially implemented as a Firefox Addon. Back then it made total sense: why would we spend resources (which we did not have) on creating our browser, while we could test our idea in a much simpler way? Firefox at the time was the best, and, in fact, the only suitable platform for such an experiment, because:
- Most of its UI and business logic was—and still is—built on Web technologies like Javascript, HTML, and CSS to the point that you can even use its Firefox Web Developer Tools to debug the browser itself.
- Firefox Addons had direct access to the browser’s privileged context and its internal modules.
Over time, our extension grew, encompassing more and more privacy-related features: forget tab, adblocker, anti-tracking, anti-phishing, and in this form managed to gather a stable userbase. Later, we decided to take more control over the platform we run our extension on, so that we could experiment more, and deliver updates faster. That is how the Cliqz Browser was born.
For several years, we have developed and supported both the Cliqz Browser (for our most dedicated fans) and the Cliqz Firefox Addon (for those who were not ready to move yet) until everything changed at the beginning of 2017, when Mozilla announced the deprecation of Legacy Addons in favor of WebExtension.
This meant that, if we wanted to keep the same architecture (minimally modified browser + lightweight extension), we had to migrate. We managed to successfully port a big chunk of our code to the new APIs, but it was obvious that our main feature—the dropdown—would be impossible to implement that way. The APIs we would have needed simply did not exist. We could still implement these APIs on the browser’s side, but that would kill all the upsides of having this functionality separated, which we described above. Luckily we found an alternative way to achieve that: Experimental APIs.
What is an “Experimental API” anyway?
WebExtension Experiments[3] or Experimental APIs is a Mozilla technology, which allows developers to implement their own WebExtension APIs for Firefox, play with them, share with other developers, and—potentially—promote them to real APIs.
Similarly to legacy addons, the code implementing such APIs has full access to Firefox’s privileged context and internal modules. Practically speaking, Experimental APIs allow you to do the same as the now-defunct Legacy Addons, while formally staying within the WebExtension framework. Do you want your extension to be able to move browser tabs to the left? You can do it. Does it need access to the file system? You can have it!
At this point, you are probably wondering: if it is so powerful, why is nobody using it? Well, because there is a catch. Extensions using experimental APIs are not allowed in stable or beta versions of Firefox, unless they are approved and signed by Mozilla. Moreover, you will not be able to even install them as a temporary addon for development or debugging purposes. They only work under certain conditions in the Firefox Developer Edition, Firefox Nightly[4], and unbranded or custom Firefox builds like our browser[5].
And this is not the only drawback of using Experimental APIs. Having worked with this technology for some time already, we can list some other major downsides one should consider before using it:
- It is extremely easy to break the browser. As it usually goes, a great power comes packaged with great responsibility. WebExtension APIs, due to their limited capabilities, are relatively safe to use. You can still break some things with your extension, but all the potential breakage is reversible and usually easy to fix. Experimental APIs are the opposite of that. Crashes, freezes, memory leaks, destroyed user profiles, browser UI broken beyond repair—are all too easy to achieve with improper use of Experimental APIs.
- If you do not break your browser, Mozilla will likely do that for you. With the death of Legacy Addons in November 2017, Mozilla stopped caring about those who may still use Firefox’s internals. This allowed them to move faster, finally refactor and even rewrite large pieces of their code, and ultimately make Firefox faster and more stable than it has ever been. Unfortunately, sometimes this works against those who use Experimental APIs. If you rely on some internal module, XUL element, or global object in the implementation of your API, do not be surprised if all that completely changes in the next version. We have to constantly check our code in Firefox Beta and Nightly in order to keep our APIs compatible.
- There is no documentation. With WebExtension API, it is possible, by reading documentation, to build an exhaustive list of what you can or cannot do with them. With Experimental APIs, you can do almost everything, but it is not at all obvious how exactly to do it. That is why most of the time spent on writing Experimental APIs goes to reading the Firefox code. And it goes without saying, that it is very unlikely for you to find answers on Stack Overflow or Medium.
While these drawbacks could make this technology a “no go” for most developers, it turned out to be the exact piece missing from our puzzle. Our current “legacy” code naturally transitioned to Experimental APIs with minimal changes. The fact that these APIs can be bundled and distributed together with the code which uses them, allowed us to keep changes to the browser core to a minimum.
Conclusions
In the beginning, building an addon was the obvious choice for us, thanks to flexibility and customizability of Firefox. It is important to mention that Firefox is perhaps the only truly open web browser in the sense outlined by the Extensible Web Manifesto[6]. The way Mozilla implemented its core value of openness enables developer to contribute to the way the future of the Web is shaped and we at Cliqz utilize this power to its full potential. Thank you Mozilla!
When we added the Cliqz browser to our product portfolio—to become more independent and to permit for faster release cycles—our choice paid off tremendously: it allowed us to simply take the existing addon code and add it on top of our fork of Firefox and the Cliqz browser was ready to ship. Over time, our code evolved as we added new features and moved to WebExtensions. Experimental APIs—despite their downsides—proved to be extremely valuable to us: they enabled us to keep most of our code base in one place without having to change the browser itself.
We are thrilled to see that what had started out as a browser addon to offer better search and privacy protection to Firefox users has, over time, grown into an integral part of many great Cliqz products across multiple platforms.
