Friends don’t let friends feed their data to AI agents

Your friends can now feed your private conversations and personal data to their AI assistants without you knowing.

Meta's new AI assistant, Muse, encourages users to grant broad access to personal data, including contacts, chat logs, and financial information. The app, which has seen millions of downloads and reached the top of Apple's App Store, prompts users to share not only their own data but also that of their friends and contacts. Meredith Whittaker of the Signal Foundation highlighted that these agents act on behalf of users and extrapolate information from their contacts, potentially exposing sensitive details of individuals the user interacts with. Meta spokesperson Daniel Roberts stated that Muse gathers data from public information and user-shared content to provide context and personalization, citing examples like identifying a plumber or remembering a spouse's flower preferences. Despite Meta's claims of opt-in processes and user control over privacy settings, incidents have emerged where users reported unexpected data access, such as extensive text message collection and the disclosure of personal addresses. Meta has disputed some of these claims, attributing them to misconfigured permissions, but cybersecurity experts like Eva Galperin of the EFF argue that the app's interface is designed to encourage users to share more data than they realize, using dark patterns.

The core issue is not just user privacy but the privacy of everyone an app user interacts with, extending mass surveillance to an unprecedented scale. AI agents like Muse blur the lines of consent by making it difficult for users to understand the scope of data they are sharing, thereby compromising the information of third parties who have not consented. This creates a scenario where digital contact with someone using such an agent can lead to involuntary data exposure.

This development signifies a significant shift in how personal data is commodified, moving beyond individual consent to networked consent, or the lack thereof. The convenience offered by AI assistants is presented as a Faustian bargain, where the pervasive collection of sensitive information about individuals and their social circles fuels the surveillance advertising model. The technical capacity to process vast amounts of data at speed enables this new level of invasive data collection, posing risks to individuals and potentially circumventing encryption for activists and others needing privacy.