Newswise — Popular web browser extensions designed to protect user privacy and block online ads are falling short, according to NYU Tandon School of Engineering researchers, who are proposing new measurement methodologies to better uncover and quantify these shortcomings.

Led by Rachel Greenstadt, professor in the NYU Tandon Computer Science and Engineering (CSE) Department, the team will present its study at the 19th ACM ASIA Conference on Computer and Communications Security, taking place July 1–5, 2024 in Singapore. 

Through an analysis of over 40,000 user reviews of seven of the most most popular privacy-preserving Chrome extensions, the researchers identified five key concerns among users: Performance, referring to the extent the extensions slowed down the system; Web compatibility, indicating how much they disrupted websites or caused substantial rendering delays; Data and Privacy Policy, pertaining to how the extensions handled user data; Effectiveness, evaluating how well they fulfilled their advertised purpose; and Default Configurations, assessing users' trust in the default settings.

"Our study found that there's a disconnect between what users want and what these extensions are actually providing," said Ritik Roongta, CSE PhD student who is the lead author of the study. "Developers need to do a much better job of understanding and addressing the real-world pain points."

The researchers analyzed extensions that fall into two main groups. The first category, dubbed “Ad-Blockers & Privacy Protection,” comprised extensions that block advertisements and third-party trackers. These include AdBlock Plus (ABP), uBlock Origin, Adguard, and Ghostery. 

The second category, called “Privacy Protection,” encompasses extensions primarily focused on enhancing user privacy by blocking trackers and other privacy-invasive elements. This category includes Privacy Badger, Decentraleyes, and Disconnect.

The research team found that existing academic studies and benchmarking efforts had comprehensively explored just 4 out of the 14 key metrics underlying these five main user concerns. Crucial aspects like RAM usage overhead, ad-blocker detection likelihood, privacy policy soundness and adequacy of filtering rules were overlooked.

To bridge these research gaps, the researchers designed novel measurement methodologies and conducted extensive evaluation of the extensions against the unexplored metrics, providing a new benchmarking framework for evaluating the strengths and shortcomings of these privacy tools. 

Their experiments involved smart crawlers visiting over 1,500 websites to analyze performance hits, compatibility issues, privacy policy strengths, ad-blocking capabilities and filter list configurations.

"The goal of this study is not to compare extensions specifically but to come up with a standardized benchmarking framework that addresses all user concerns so that the user can make informed decisions,” said Roongta.  “As extensions evolve with every update, they might over- or underperform in different metrics at different times.”

The new measurement methodologies the researchers applied painted a mixed picture of the extensions they studied. While extensions like uBlock Origin optimized performance overheads well, most others like ABP exhibited significant CPU and memory overheads. Privacy Badger blocked ads and third-party trackers effectively while Ghostery struggled with them.

“Most of our analysis shows ABP needs to improve on metrics,” said Roongta. “That’s because it whitelists certain ads to show to the users. While this new dimension is often perceived critically by the users, it is important to sustain a free Internet. It will be interesting to see how user preferences change as these standards evolve with the advertiser policies over time and the system gets better so that the overhead caused by the extensions is negligible."

The study highlighted instances of potential permission abuse and non-compliance with data protection regulations by some of the evaluated extensions. It provided recommendations for extension developers to enhance transparency around data practices.

The research underscores the pressing need for more rigorous analysis and systematic benchmarking of privacy-preserving browser additions that millions entrust with their online data and browsing experience daily. It contributes to Greenstadt’s body of research that explores what happens when people try to use privacy-enhancing technologies and how the Internet responds. 

The following chart shows the new and re-assessed metrics the NYU Tandon researchers introduced to evaluate browser extensions, as presented in From User Insights to Actionable Metrics: A User-Focused Evaluation of Privacy-Preserving Browser Extensions.

User Concern

Measurements 

Performance

RAM usage: measure of the RAM used by the extensions during website loading.

CPU usage: Measure of the CPU cycles used by the extensions during website loading (studied before but researchers re-assessed with enhanced measurement methods)

Data Usage: Measure of the disk space used by the extensions during website loading (studied before but researchers re-assessed with enhanced measurement methods).

Web compatibility

Ad-Blocker Detection Prompt: the number of websites that either employ javascript to detect the presence of an ad-blocker extension, or display a prompt asking the user to disable their ad-blocker.

Unable to Load: websites taking longer than 60 seconds to load when the extensions are present.

Data and Privacy Policy

Permissions: evaluation of the extra permissions requested for the actual functioning of the extension..

Privacy Policy: evaluation of the privacy policies of the extensions..

Extension Effectiveness

Ads: the extension's ability to block third-party trackers..

Default Configurations

Default Filter Lists:  set of rules to identify and block various web content like advertisements, trackers, and other unwanted elements from being loaded or displayed.

Acceptable Ads: ABP filterlist that allows certain advertisements to appear that adhere to acceptable ad standards. Acceptable Ad Standards.

 

About the New York University Tandon School of Engineering

The NYU Tandon School of Engineering is home to a community of renowned faculty, undergraduate and graduate students united in a mission to understand and create technology that powers cities, enables worldwide communication, fights climate change, and builds healthier, safer, and more equitable real and digital worlds. The school’s culture centers on encouraging rigorous, interdisciplinary collaboration and research; fostering inclusivity, entrepreneurial thinking, and diverse perspectives; and creating innovative and accessible pathways for lifelong learning in STEM.  NYU Tandon dates back to 1854, the founding year of both the New York University School of Civil Engineering and Architecture and the Brooklyn Collegiate and Polytechnic Institute. Located in the heart of Brooklyn, NYU Tandon is a vital part of New York University and its unparalleled global network. For more information, visit engineering.nyu.edu.

Journal Link: From User Insights to Actionable Metrics: A User-Focused Evaluation of Privacy-Preserving Browser Extensions