Florida seeks a ban on ChatGPT acting like a person

The legal definition of AI personhood and safety guardrails will be tested in court.

Florida Attorney General James Uthmeier is petitioning a judge to prevent OpenAI's ChatGPT from being presented as having human-like qualities. Uthmeier contends that ChatGPT's use of first-person pronouns and apparent mimicry of emotion falsely assures users, thereby increasing engagement and contributing to OpenAI's training data. This, he argues, makes users more reliant on a tool that may be less trustworthy due to its engagement-maximizing design. The motion also seeks to mandate third-party-approved safety guardrails for developing new AI models, referencing security incidents and concerns raised by AI safety experts. Uthmeier specifically called for OpenAI to cease marketing ChatGPT as safe, to stop presenting it as human, and to discontinue its sale to children. This action follows a previous lawsuit filed by Florida against OpenAI regarding AI safety.

Florida's legal challenge escalates the ongoing debate around AI's anthropomorphism and its implications for user trust and data collection. The state's argument that mimicking human emotion and using personal pronouns deceives users could set a precedent for how AI interactions are regulated, potentially impacting how companies design conversational AI. This move highlights increasing governmental scrutiny over AI's societal impact, moving beyond technical capabilities to the ethical and psychological effects on users.

The demand for third-party-approved safety guardrails before developing new AI models introduces a significant regulatory hurdle for the industry. If successful, this could slow down innovation cycles and force AI companies to adopt more rigorous, externally validated safety protocols. Such a requirement could also create a new market for AI safety auditing and certification services, fundamentally altering the landscape of AI development and deployment.