2022.10 Vol.2

Browser State and Why Ad Tech Sucks

At Mash we have been working on a feature where are product lives embedded in other websites. I wasn’t very familiar with browser tech before Mash and this feature gave me a crash course in some recent developments. We released the “embed” feature a little over a year ago. Over the year we saw more and more bugs even though the relevant code wasn’t changing. Turns out, the browsers were changing.

Our embed feature seemed simple on paper and we implemented it with “old” (20+ years) browser tech. We sandboxed our web application inside of another with an iframe tag. Iframe stands for “inner frame”. It has a src attribute where it pulls its content from and this can be a completely different origin than the parent website. Appears to fit our use case perfectly!

We started seeing issues though where users had to re-login to our webapp all the time. A painful amount of times. Debugging the issue was also painful because it appeared to depend on different combinations of device type, browser app, and browser settings. It was easy to see that increased privacy settings generally made the issue worse, but I didn’t know why. A deep dive was required.

Starting at the beginning, what does it mean for a user to be “logged in” to our webapp? From a technical perspective, a user is logged in when the client can pass a token (a string) to our server that the server can use to know which user made the request. The token can be validated by the server with some practical cryptography. So “logged in” technically means “client has a valid token”.

The process for a client to first get a token, or mint a token, usually involves some handshake with the server requiring user interaction. Like an email/password prompt. Our product would prefer a user experience where a user doesn’t have to enter their email/password all the time, but rather has the client remember the token for them. In our scenario, the client is a web browser. So with our embed feature, why were some browsers choosing to not remember the token while others did?

The crux of the issue is a thing called “third party” requests. When browsers, and the internet for that matter, were first designed they generally didn’t consider a adversarial environment. They were focused on just making this crazy thing work and figured everyone would be too busy doing that to purposely mess things up. When a browser loads data from a origin, the page itself can contain links to other data like images. This data can live on other servers than the one serving the main page. This is called a “third party” request. If this sounds familiar, its because its just like our Mash embed feature. iframe tags which have a src attribute that points to a different server than the parent page are considered third party requests.

This all sounds fine, but over the years big Ad Tech realized this was a great way to track people. Let’s say a users loads Site A which contains a third party request to Tracker A’s server. The user then loads Site B which also contains a third party request to Tracker A. The Tracker A server now has two request logs, but how does it connect them? Enter browser state. On the first request, the Tracker A server asks the browser to store some state for it, a simple key value: userID=1. It also asks the browser to send that state whenever it makes requests to this server. Now Tracker A can look at its logs and see that the user used Site A and then Site B. And thus, a terrible industry was born.

This example state flow is pretty much how browser cookies work. It is one of a handful of ways state is stored in the browser. Since cookies are not very performant, having to be sent on every request, some follow up methods were established over the years. These fall under the “Web Storage API”: Local Storage, Session Storage, IndexedDB. These improved upon the performance issues with cookies, but interestingly, follow the same security patterns as cookies. They follow them so closely that most modern browser cookie security settings also control the Web Storage settings.

Historically, a web application (e.g. a JS script running in the browser) could access the browser state, be it a cookie or Web Storage, which matched its origin. Basically, a script running from Site A could access data in the browser which belonged to Site A. This makes sense. We took advantage of these default settings with our embed feature. These rules are also how browsers are now tightening things up, quickly, in order to crush the ad tracking patterns. This is why our issue got worse over the year as different browsers started enforcing things.

The first step many browsers are taking is to just straight up ban third party requests from accessing browser state. This means only scripts from sites which match the site in the browser URL, first party, can access browser state. This explains why our users have to keep logging in. Our third party context webapp is not able to persist and re-use tokens in browser state. This might seem like a drastic first step for browsers, but in their defense, ad tech does fuckin suck.

In the medium-term, browsers are slowly adopting a partitioned state strategy. This means that a third party request won’t have access to its domain, but instead, have access to state that falls under the first party origin partitioned by the third party origin. Something like site-a.com^site-b.com. This blocks the ad tracking use case while still giving some state ability to legitimate use cases. Sadly it still wouldn’t work for our embed use case.

The proposed Storage Access API will help legitimate use cases like ours (and others like Single Sign On services). The API allows webapps to ask users to “promote” the app’s third party request to access first party domain state instead of the partitioned state. This would work for our Mash embed case! But doesn’t have great browser support yet. Seems like the ad tech practices were obnoxious enough that browsers disabled 100% of use cases before having a path for legit ones. Stupid ad tech.