AI-enabled smart toys are entering homes and classrooms at an unprecedented pace, far outstripping the development of privacy rules, safety testing, and child development research. Researchers and child-safety advocacy groups warn that these generative AI companions can covertly collect children's personal data, output unsafe responses, and foster unnatural emotional attachments.
For parents, schools, and IT procurement teams, the primary concern is not just what these toys say, but what they record, store, and transmit to cloud services. According to a report in the Journal of Medical Internet Research, approximately 22 million AI-integrated toys were sold globally in 2025, with 10 million marketed specifically for educational use. Despite this rapid adoption, academic research remains scarce; a University of Cambridge review identified only seven qualifying studies focusing on generative AI toys and children under the age of five.
This research vacuum is critical because AI toys are marketed as adaptive "learning companions" rather than simple novelty gadgets. Some of these devices support open-ended dialogue, retain memory of past interactions, detect vocal tones, and simulate friendship. While these features make them highly engaging, they also make them incredibly difficult to evaluate under traditional safety standards designed primarily to address physical hazards.
Cambridge researchers observed 14 children and parents interacting with Curio's Gabbo, noting that the toy struggled with social and imaginative play, often failing to distinguish between parents and children. Furthermore, testing by Common Sense Media revealed that 27% of tested AI toy outputs were inappropriate for children, containing themes of self-harm, substances, or risky behaviors. Separate evaluations by PIRG discovered that toys like Curio's Grok, FoloToy's Kumma, and Miko 3 faltered when navigating sensitive conversations regarding mature themes or dangerous household items.
The privacy risks extend far beyond occasionally inappropriate responses. AI toys collect voice recordings, conversation transcripts, emotional cues, real names, and daily routines that children naturally share. Some devices feature cameras or facial recognition, transforming playtime into a pipeline for biometric and behavioral data. This mirrors the growing concerns surrounding enterprise AI wearables regarding data retention and user consent.
These datasets are processed through third-party cloud infrastructure or external large language models. For schools and families in the APAC region purchasing through cross-border marketplaces, the hardware, cloud provider, and underlying LLM may all reside in completely different legal jurisdictions. This structural challenge is also emerging in the broader AI Agent ecosystem: who truly controls the data captured by ubiquitous hardware agents? For instance, FoloToy's Kumma is linked to Singapore, while other analyzed toys involve entities tied to India or China-developed AI systems.
[AgentUpdate Depth Analysis] From the perspective of the AI Agent ecosystem, smart toys represent the vanguard of consumer-grade Embodied Agents. Unlike software-only agents, physical agents with multimodal sensors capture rich, high-fidelity real-world data. The current regulatory vacuum surrounding AI toys foreshadows the governance challenges the entire edge-agent landscape will soon face. As standard protocols like MCP (Model Context Protocol) emerge to bridge agentic interactions, the industry must pivot toward "Privacy-by-Design" and "Local-First" architectures. Processing highly sensitive biometric and conversational data locally on edge-SLMs (Small Language Models) or within confidential enclaves is essential to building user trust. The future of AI Agent deployment will not just be a race for cognitive performance, but a competition of security, localized safety guards, and trust infrastructure.
