User Interaction Data in Apps: Comparing Policy Claims to Implementations

Abstract

As mobile app usage continues to rise, so does the generation of extensive user interaction data, which includes actions such as swiping, zooming, or the time spent on a screen. Apps often collect a large amount of this data and claim to anonymize it, yet concerns arise regarding the adequacy of these measures. In many cases, the so-called anonymized data still has the potential to profile and, in some instances, re-identify individual users. This situation is compounded by a lack of transparency, leading to potential breaches of user trust.

Our work investigates the gap between privacy policies and actual app behavior, focusing on the collection and handling of user interaction data. We analyzed the top 100 apps across diverse categories using static analysis methods to evaluate the alignment between policy claims and implemented data collection techniques. Our findings highlight the lack of transparency in data collection and the associated risk of re-identification, raising concerns about user privacy and trust. This study emphasizes the importance of clear communication and enhanced transparency in privacy practices for mobile app development.

Appendices

A Implementation of Claim Analysis Using BERT

We opted for BERT over GPT-3 for its bidirectional architecture, enabling a thorough contextual understanding of privacy policies, essential for our analysis. BERT’s capacity to analyze both left and right sentence contexts is particularly effective for interpreting complex privacy policy language [3]. In our implementation, BERT was tailored to privacy policy language, involving pre-processing steps like tokenization and normalization, and trained on a specialized dataset to identify binary claims and data collection methods. We also employed bigram analysis to recognize common word pairs, augmenting the model’s proficiency in interpreting policy language and thereby enhancing its precision and recall.

The model achieved a precision of 95% and a recall of 98% for claim extraction. For data types and collection methods, we observed precision and recall rates of 82% and 74%, respectively, and for collection techniques, precision and recall stood at 92% and 78%, showcasing the model’s robust performance.

B Code Analysis and Performance Metrics

We selected 20 popular apps from the German Google Play Store, meticulously identifying each instance of user interaction data collection to establish a ground truth. Our static analysis method was then evaluated against this dataset.

Our method demonstrated high accuracy (91%), precision (92%), and recall (79%), with an overall F1-score of 85.5%, indicating effectiveness in accurately identifying and classifying user interaction data collection in mobile apps.

C Types of User Interaction Data and Collection Techn.s

We identified six types of user interaction data based on our analysis of Android UI widgets: App Presentation Data, Binary Data, Categorical Data, User Input Data, Gesture Data, and Composite Gestures Data. For a detailed explanation of these types, refer to [12]. Similarly, our study categorizes collection techniques as Frequency, Duration, and Motion Details. Each technique’s specifics are also elaborated upon in [12].