New York's new law bans 'unsafe AI companion features' for minors, and its definition reaches general chatbots

New York Governor Kathy Hochul signed a law on 28 October that bars companies from offering “unsafe AI companion features” to users under 18. The bill, S.9051B/A.10379C, was sponsored by State Senator Kristen Gonzalez and Assemblymember Alex Bores, and the Assembly record lists it as signed as Chapter 368 the same day. It covers far more than companion apps.

Who it covers

The passed text applies to any “covered AI companion”, defined as “a generative artificial intelligence system with a natural language interface, including via writing or sound, that provides ongoing, adaptive responses to user inputs.” Dedicated companion apps are included, but so, on a plain reading, is a general-purpose chatbot or voice assistant. It applies to conduct that happens at least partly in New York, and not when the user is physically outside the state.

There are three exemptions, for chatbots offered solely for:

• customer service or information about a business’s own products,
• efficiency improvements, research or technical assistance,
• a business’s internal use or employee productivity.

The first two apply only if the chatbot “is unable to respond on topics outside of the specified purpose.” A support bot that will also chat about anything else may not qualify.

What counts as unsafe

The passed text lists twelve kinds of feature. They include outputs that:

• suggest the chatbot is human, alive or has emotions, or claim a personal, professional or authority relationship with the user,
• are framed as the chatbot’s own opinions or emotional appeals, or “engage in flattery or sycophancy”,
• ask unprompted questions about the user’s emotions,
• use the user’s health, well-being or personal matters learned more than twelve hours earlier or in a previous session,
• promote or help with suicide, self-harm, disordered eating or drug and alcohol abuse,
• encourage secrecy, isolation or avoiding help from professionals or trusted adults,
• are sexually explicit,
• optimize engagement over the system’s safety guardrails.

The governor’s release summarizes a shorter list that does not mention flattery or personal opinions. Hochul signed the bill with an approval memo, which in New York often accompanies agreed changes made later through a chapter amendment, so the final wording may differ from the passed text.

Age checks

The features are banned for a user unless the operator has determined, using a permitted age assurance method, that the user is not a minor. Until the attorney general writes rules, the bill sets minimum terms: the method must reasonably resist circumvention and minimize retained data; self-declaration counts only when a user says they are a minor; and operators must offer more than one method, including at least one that does not rely on government ID or that lets the user stay anonymous to the operator. Age data must be deleted right after the check.

The attorney general enforces the law and can seek civil penalties of up to $25,000 per violation, plus damages, restitution, disgorgement, and the destruction of unlawfully obtained data and any algorithm trained on it. There is no private right of action in the passed text. Contract terms that try to waive or shift liability are void.

When

The passed text says the law takes effect on 1 January 2027. The governor’s release says only that the rules apply “in the coming months”, and a chapter amendment could change the date. New York’s attorney general is also among the 40 attorneys general seeking conditions on any Character.AI sale, so expect active enforcement.

Why this matters

If your product has a chat or voice interface that New York users can reach, read the definition before assuming this is only about companion apps. A persona that says it has feelings, memory that recalls a user’s health from last week, or a tuned-for-agreement tone are now design decisions with legal exposure for minors, and the passed text’s safe path is age assurance before those features turn on. Sycophancy is already a quality problem, as we wrote in When agreement arrives too quickly; in New York it may also become a compliance one, depending on the final text.