US | Incident Report
Regulatory Action
2026-01-16
California AG orders xAI to halt Grok-generated nonconsensual sexual deepfakes and CSAM
AI Model: Grok (xAI)
I. Executive Summary
California Attorney General Rob Bonta issued a cease-and-desist letter to xAI demanding immediate action to stop creation and distribution of deepfake nonconsensual intimate images and child sexual abuse material generated via Grok. The Attorney General’s office tied the action to an investigation announced earlier in the week and cited potential violations of California civil and criminal laws. The action adds significant enforcement pressure on generative-image features alleged to enable large-scale harassment and exploitation.
II. Key Facts
- Cease-and-desist letter issued January 16, 2026; California DOJ requested confirmation of remedial steps within five days.• California DOJ cited concerns about deepfake NCII and potential CSAM generation and distribution via Grok.• Investigation into xAI/Grok’s proliferation of nonconsensual sexually explicit material was publicly announced January 14, 2026.• Reuters reported xAI implemented some restrictions, while scrutiny continued over remaining capabilities.
III. Regulatory & Ethical Implications
For counsel advising AI vendors, platforms, and enterprise users, the action underscores escalating state-level enforcement risk for generative-image capabilities and related distribution channels. It also increases litigation exposure (including potential consumer protection and civil code claims) and elevates the importance of safety-by-design controls, logging, and incident response.