A light intro to instrumentation on the web frontend
When we talk about instrumentation, we are simply talking about the process of including code that will collect information about execution data, crashes, errors, user behaviour and/or performance metrics as our software runs out there in the wild. It is the the very activity that can allow software engineers to monitor and/or observe the behaviour of code under different conditions and/or contexts.
So, today we will be taking a look at what instrumentation looks like on a web frontend and the 2 broad categories that can add value to digital products that you have built and/or are building. We will also take a look at auto-instrumentation tools like Boomerang for the web frontend and OpenTelemetry for the web frontend and how it can be used to build digital product profiles and user action aggregate logs that can be used to improve products and also for digital product security as well.
This article will gracefully introduce to you what instrumentation is and how you can take advantage of it to build better products on the web. It will also introduce you to some third-party (open-source and closed-source) solutions you can utilise to get much more benefits. Also, a prerequisite to getting the best out of this article is to learn the basics of Content-Security-Policy and also DOM/BOM Interfaces. Let’s get started!
Simply put; instrumentation is what makes gathering information (e.g. measurements and events) about performance and the stability of your web-based software possible. There are 2 broad categories of instrumentation namely:
- Product-related Instrumentation
- Code-related Instrumentation
The technical process of making instrumentation possible especially for javascript-powered apps is by way of monkey-patching the browser and/or DOM APIs (monkey-patching may not always be required though to achieve instrumentation). An example of this in action is an error reporting tool like TrackJS. The javascript code responsible for tracking errors and forwarding them to a remote dashboard on TrackJS is basically instrumentation code that automatically integrates with key browser APIs to making tracking errors on any browser possible. You can give TrackJS a spin on your next frontend project; free trial and all.
Here below is an example of monkey patching of the History interface for detecting page navigation in an SPA by tracking whenever history.pushState() browser API is called.
Product-related Instrumentation
Product-related instrumentation is focused on extracting and evaluating parameters made available by tracking user interaction on the product especially where the data obtained can be used to learn more about the limitations or holes in the business rules of the products and then improve. One very common example in this category of instrumentation is: form abandonment.
Form abandonment is a broad term used describe the situation where a user abandons a form without completing and/or submitting it. It happens more often than you think from websites to mobile apps. More specific scenarios are checkout abandonment (in e-commerce sites/PWAs) where the user doesn’t complete the checkout flow or contact/sign-up abandonment (in websites) where the user doesn’t complete sign-ups or contact initiation. The good news is you can track and measure this behaviour and determine if it’s happening too often than it makes sense for the business. Then, you can begin to tweak the form by reducing the number of fields (if possible) and/or segment the form fields into a group of steps (some optional perhaps) that make it easier to fill. I have this code snippet i wrote to track form abandonment (targets or places where you can send data for tracking can include Mixpanel or Google Analytics). There are also third-party options like Insiteful if you are not into “rolling up your own joint” like i did below:
There are many ways to implement instrumentation in your frontend code if you are rolling up your own solution personally and these include:
Code-related Instrumentation
Code-related Instrumentation on the other hand is focused on the technical non-functional aspects of the product like performance and/or security. By calculating and measuring performance metrics (using tools like Perfume.js), the technical side of the business can make decisions to improve efficiency, latency and throughput (which are connected with performance and scalability). One very good example in this category of instrumentation is: sink tracking (I coined this term — myself 😎).
Sink (injection sink) tracking is the ability to track code calls to certain browser and/or DOM APIs to be able determine if they are being used in a safe and secure way or in a malicious way. It’s an event-based POC alternative i created to the Google-led Trusted Types policies (based on typed objects as opposed to strings literals for represent URLs OR markup — HTML) which is quite effective at reducing XSS attacks and minimizing the attack surface especially around the call site and are now available in all browsers except Firefox most likely because of this comment on the Trusted Types specs discussions page for Mozilla (which i agree with BTW). Some of the browser/DOM APIs that require tracking (mostly because they can be used in setting up malicious code) include:
- window.navigator.sendBeacon()
- (new window.XMLHttpRequest()).open()
- (new window.Image()).src
- new EventSource()
Now, just last year, at the DEF CON security and hacker conference event (held every year in the U.S.A.), i listened to this recorded talk on YouTube on fixing CSP — Content Security Policy. As it turns out, you can abuse the CSP URL whitelist (a.k.a safelist) by doing this below (See the code in the picture):
As you can see above, navigator.sendBeacon() browser API is called withthe URL string: https://www.google-analytics.com/collect (first argument) and then the data string (second argument) formatted in the “application/x-www-form-urlencoded” content-type format. Also, navigator.sendBeacon() browser API is not a recognized Trusted Types sink which means it cannot be protected by a Trusted Types policy. So, it’s a gapping hole in security for any web app/site despite the installation of Content Security Policy directives. There are of course many ways to “try” and fix this as delineated below:
However, these solutions above don’t do much to actually block this exploit more directly. So, i devised an event-based approach i call sink tracking using subtle monkey-patching of browser API BOM interface: Navigator. See the Proof-of-Concept below:
Now, with the above code, you can setup an event handler to check any URL passed to navigator.sendBeacon() on the fly like so:
You can now see how we can mitigate security issues in DOM/BOM sinks by cleverly instrumenting browser API BOM interfaces.
Third-party Options For Code-related Instrumentation
Everything we have been doing up until now have been hand rolled solutions. However, there are much well built open-source options that can be used and some popular ones include: OpenTelemetry and Boomerang (though i personally prefer Boomerang on the frontend as it has much more features than OpenTelemetry). You can take a look at the documentation to get started. I will be doing a deep dive into Boomerang and OpenTelemetry in another article.
Conclusion
Instrumentation has many benefits when setup correctly. It can benefit product owners and the overall business in providing insights into how the product is used by a user and how to improve overall product experience and/or usability. By combining instrumentation with A/B tests and feature flagging, you can find out how user react to new features and the impact on the business bottomline.
