GULYÁS, Gábor György, Ph.D.

# Blog

## Measuring how browsers implement Content Security Policy

2017-04-20 | Gabor

Back to the blog

Related to our recent project I started studing what is Content Security Policy (CSP) and how it works. In a nutshell, CSP is designed to protect website visitors against malicious activities. For example, if someone injects external scripts into a forum for some mal-intended reasons (like XSS or tracking), this could be prevented with CSP easily. The forum website only needs to declare that it does not accept external scripts. This need to be sent to the web browser as a header (before sending other content), which is something like this:

Content-Security-Policy: script-src www.my-forum.com;


Then if your visitors use a browser that implements CSP properly, malicious scripts or images (trackers) will be refused to load. CSP can also be used to protect content against unwanted embedding from www.my-forum.com (X-Frame-Options): third party websites will not be able to include scripts or to embed the whole site in an iframe.

While working on the project, first I had a naive implementation done on CSP, I observed strange (and rare) bugs. It seemed that the CSP reports – that we used for measuring the presence of web logins – sometimes didn’t arrive at all. Soon I started to wonder if I can still rely on my assumption that CSP implementations in different browsers works the same way.

Therefore I made a small tool to evaluate different implementations, and found interesting differences between implementations in differend browser brands. Furthermore, I also discovered a strange bug in the Chrome browser: sometimes it forgets to deliver CSP reports.

### The CSP testing tool

In order to clarify the situation, I wrote a CSP test tool to measure what happens. It is available in this GitHub repository. This is a very simple page that tries to load an image, while it logs all related events into the csp.log file. Of course loading the image is against a CSP constraint: external images are prohibites here. Following events are logged with a timestamp: when the image is loaded (never) or if there is an error, when the CSP violation is reported from the browser, and finally a timeout for reference. The CSP violation reporting can be requested in a similar fashion as the CSP example above:

Content-Security-Policy: script-src www.my-forum.com; report-uri x.php?params


The layout scheme below summarizes how this tool works.

Scheme of the CSP testing tool.

### But how CSP reporting is implemented in different browsers?

My initial (and seemingly naive) assumption was that when the image load fails because of the CSP constraint, all browsers will fire three events in a single atomic step:

1. An onerror JavaScript event is called (if there are any callback functions registered).
2. A notification is pushed to the browser console to notify the user.
3. The CSP violation report is sent to the report-uri.

To the contrary, I found that these events are not encapsulated in an atomic event, significant delays can occur in between them. In our experiment, the timing is important, because we are measuring if a given CSP violation occurs in time or not. However, if it doesn’t happen, how can we know if we can move on? Well, the onerror could be such an indicator, or in the worst case, if a timeout happens.

Playing with the CSP test tool, I found the following:

• Firefox has the cleanest CSP implementation, works just as expected. However, the load error and CSP violation events are fired concurrently, demonstrated by the following example:

This also shows that the onerror and CSP violation events are not encapsulated in an atomic transaction, as we can observe ~1s difference!

• Chrome does something similar to Firefox, but very strange bugs emerged during my experientation – I’ll discuss these a later.
• As Opera uses the Blink layout engine from Chrome, it has the same issues.
• Safari doesn’t fire the onerror event if the image cannot be loaded due to a CSP constraint violation.
• Microsoft’s browsers make the lives of web developers easier: CSP is neither supported by IE, nor Edge, as far as I know.

### Further findings and strange bugs: Chrome

In Chrome, I observed strange inconsistent behavior regarding how the violation event is fired. This appeared in multiple flavors, but rarely, making systematic reproduction difficult.

Probably the most serious bug in Chrome I encountered was when the browser just stopped sending violation reports. As I tend to avoid restarting my laptop for 1-2 weeks, usually I observed such phenomena after a longer period of time with varying delays.

Here you can see an example when Chrome forgets to send the violation report, while Firefox works nicely:

To get the Chrome browser working again, I had to restart it. Sometimes I also restarted the system, thus I can’t surely state that the browser restart was always enough.

If it didn’t stop sending violation reports, Chrome just produced varying delays. For example, the following patterns occured in Chrome when I was using a shared, slow a network at a workshop:

The networks was unpredictable: it most cases, pages and content loaded, but sometimes you had to reload hanging pages. Probably same issues made the violation arrive later than the timeout (left). In other cases there were simply large delays (right). However, I could not reproduce this phenomena in the office by simulating heavy traffic, e.g., opening a lot of tabs in parallel or downloading stuff.

I also discovered something which is more like to be a feature than a bug: if you make a lot of async requests from JavaScript, this blocks the way for violation reports. Probably the browser has a “pipeline” managing these requests, which would actually make sense.

Finally, there were another bug to be discovered: even if I close the test tab or reload it, the violation report is still sent (probably queued). This is why scenarios could appear with two violation reports as shown here: