Does this apply to me?
Most studios discover their safety obligations in the worst possible way: a storefront questionnaire, a publisher's compliance annex, or a regulator's information request. The trigger is almost never "we made a game for kids." It is "we added a way for players to reach each other."
Ask three questions about your title:
| Question | If yes |
|---|---|
| Can players send text, voice, or images to each other — including via lobbies, clans, friend requests, or emotes with free text? | You operate a communication service, not just a game. |
| Can players create, upload or share content others see — levels, skins, names, avatars, custom lobbies? | You host user-generated content. |
| Is your game, realistically, accessible to under-18s? | Youth-safety duties attach — regardless of your intended audience. |
This is the single most misunderstood point. A studio making a stylised shooter, a survival crafting game, or a racing title with voice chat is squarely in scope if minors realistically play it. Regulators look at your actual audience, your marketing, your art style, your storefront ratings and your telemetry. "We didn't target kids" is not a defence when your player base is 30% teenagers.
What changes the moment chat exists
Before chat, your risk surface is content you shipped. After chat, your risk surface is what your players do to each other, in real time, at a volume no small team can read. That shift brings four new obligations in most regimes:
- Risk assessment — a documented analysis of how minors could be harmed on your service.
- Proportionate measures — controls that actually address the risks you identified.
- Reporting routes — a way for players to report harm, and for you to act on it.
- Record-keeping — evidence that you did the above, kept current.
Studios routinely respond by disabling chat or geo-blocking whole countries. That trades away retention, community and revenue to avoid an engineering problem that has an engineering answer. Removing social features is a legitimate choice, but it should be a decision you make on the merits — not a panic response to a compliance email.
The vocabulary that trips everyone up
Half the confusion in this field is four terms used interchangeably when they mean different things with different costs, different accuracy and different legal weight.
| Term | What it does | Typical cost | Trade-off |
|---|---|---|---|
| Age verification | Confirms age against an authoritative source — a government ID, a credit reference, a bank signal. | ~$0.80–$1.90 per check | Strongest assurance, highest friction, and it creates a store of identity data you now have to protect. |
| Age estimation | Infers an age band from a signal — most commonly a facial scan — without establishing identity. | ~$0.10–$0.35 per check | Low friction, no document retained, but it is a probabilistic band with error margins that widen near the 18 boundary. |
| Age inference | Guesses age from behaviour, account age, purchase history, device signals. | Near zero | Weakest. Useful as a risk signal that triggers a real check — not as a gate on its own. |
| Parental / guardian authorisation | An adult verifiably consents to a minor's account and settings. | Varies | The right model for under-13s in several regimes. Requires proving the adult is an adult — a different problem from checking the child. |
Age assurance is the umbrella covering all of the above. When a regulation or a publisher says "age assurance," it is usually asking for a method proportionate to the risk — not automatically the strongest one.
You are generally not required to apply the heaviest check to every user for every action. The expectation is that the strength of assurance matches the risk of the thing being gated. Opening a chat channel with strangers and buying a cosmetic are not the same risk, and should not necessarily carry the same friction.
The retention paradox nobody warns you about
Several regimes require you to check age. Several other rules — data minimisation under GDPR, and some US state age-verification statutes explicitly — restrict or forbid retaining the identifying data you used to do it. Studios get caught between "prove you checked" and "don't keep what you checked with."
The resolution is architectural: keep the proof that a check occurred and discard the material used to perform it. A signed attestation that says "an adult was verified at this timestamp, for this account, under this policy" is durable evidence. A photograph of a driving licence in your S3 bucket is a liability with a half-life.
The regulatory map, briefly
This is orientation, not legal advice, and this field moves fast — verify current requirements with qualified counsel for your specific markets before relying on anything here.
| Regime | What it broadly expects of a game with chat/UGC |
|---|---|
| UK — Online Safety Act | Services likely accessed by children complete a children's risk assessment, apply proportionate safety measures, keep records, and review them. Age assurance is expected where risk warrants it. Enforcement has emphasised accuracy of what you tell the regulator. |
| EU — DSA + GDPR | Risk-based protection of minors, with age assurance among the measures a service may need. GDPR adds data-minimisation and children's-data duties that constrain how you may verify. |
| USA — state age-verification laws | A growing patchwork, several with private rights of action. Many restrict retention of verification data. Requirements vary meaningfully state to state. |
| USA — COPPA | Verifiable parental consent before collecting personal information from under-13s, with specific accepted consent methods. |
| Australia | Age-assurance obligations with an emphasis on effectiveness and privacy of the method used. |
A pattern worth internalising: penalties in this space have repeatedly attached to misdescribing a control and to failing to answer a regulator's information request — not solely to lacking a control. One operator was fined over a misstated configuration parameter. The practical lesson for a small studio is blunt: never describe a safety control to a regulator, a publisher or a storefront more strongly than your code actually implements, and be able to produce records on demand. Accuracy is cheaper than ambition.
Where the age signal belongs
The most common architectural mistake is treating age as a property of the account, checked once at signup, then never consulted again. Age is not a signup problem. It is an authorisation signal that gates capabilities.
Model it the way you model entitlements. An age signal should be:
- Established once, at the point the first gated capability is requested — not necessarily at account creation.
- Stored as a claim, not as evidence — an assurance level and a timestamp, not the document.
- Consulted at every gate, server-side, on every request that opens a risk surface.
- Re-checkable, so an assurance can expire or be stepped up when the risk changes.
Gate capabilities, not the front door
Gating your entire game behind an age check maximises friction and abandonment while doing little for safety. Gating capabilities puts the friction exactly where the risk is:
| Capability | Typical assurance needed |
|---|---|
| Play single-player / offline content | None |
| Matchmake with strangers, no comms | None or inference |
| Text chat with strangers | Age band (estimation) + adult/minor separation |
| Voice chat with strangers, DMs, friend requests from strangers | Stronger assurance; the classic grooming vector |
| Upload or share user-generated content publicly | Age band + content screening |
| Spend money / trade items | Age band + parental controls where minor |
Let everyone play. Establish an age band before stranger communication is enabled. Apply parental authorisation for under-13s. Keep the strongest checks for the smallest number of highest-risk actions. This is both cheaper and more defensible than a wall at the login screen.
The highest-harm surface: adults and minors
If you do only one thing from this document, do this one. The single most consequential control in a social game is not a content filter. It is governing whether, and how, adults can reach minors.
Once you have an age signal, you can implement controls that no amount of message scanning replaces:
- Default separation — adults and minors do not get open one-to-one channels by default.
- Asymmetric friend requests — an adult cannot cold-request a minor; the minor initiates or a guardian approves.
- Constrained comms for cross-band contact — where adults and minors must interact (a co-op match), restrict to canned phrases, emotes, or team-scoped chat rather than free-text DMs.
- No private channels across the band — the grooming pattern depends on moving a child somewhere unobserved. Deny the affordance.
- Off-platform migration signals — treat attempts to move a minor to another platform as a high-severity signal in its own right.
Detection is probabilistic and always arrives after the harmful message. A structural control is deterministic and arrives before it. A small studio with limited detection capability can still be genuinely safer than a large one, simply by making the dangerous interaction impossible rather than trying to notice it.
This is also the control most likely to be asked about in a publisher's or storefront's safety review, and the easiest to evidence: it is a configuration and a code path, not a model.
Detecting predatory behaviour
Studios reach for a word list. Word lists do not work for this harm, for a specific and important reason.
Why keyword filters fail at grooming
Grooming is not a message. It is a process, typically spanning days or weeks, built almost entirely from ordinary, affectionate, supportive language: attention, flattery, sympathy, small gifts, shared secrets, gradual isolation. The individual messages are not merely benign-looking — most of them are benign in isolation. The harm lives in the trajectory, not the vocabulary.
Meanwhile the words that do appear on a filter list appear constantly in normal teenage conversation, so a word list produces a flood of false positives while missing the actual pattern. You get the worst of both: noisy alerts and a false sense of coverage.
What actually signals risk
| Signal class | What it looks at |
|---|---|
| Trajectory | How a relationship develops across sessions — escalating intimacy, secrecy requests, isolation from peers, testing boundaries. |
| Asymmetry | An account that consistently initiates contact with much younger accounts, at scale. |
| Network shape | Who an account engages: high fan-out to young accounts with low interconnection between them is a targeting pattern, and it depends on neither the text nor the profile. |
| Behavioural | Session timing aligned to school hours, rapid account cycling, new accounts contacting minors immediately. |
| Linguistic | Grooming-consistent language in context — as one weighted signal, never the sole trigger. |
| Migration | Pressure to move to another platform, or to voice, or to disappearing messages. |
Under-detection: a system weighted mostly on text will miss a careful adult using entirely ordinary language. Over-detection: a system that flags teenagers for being teenagers destroys trust and buries real signals in noise. Both are safety failures. The way out is multiple independent signal classes, weighted, with a human at the end.
A practical note on scope: detecting sexual imagery shown to a minor is ordinary adult-content detection, flagged because of the recipient. It does not require handling material depicting minors. Known-CSAM detection is a separate, tightly controlled path — for that, you use hash-matching under a proper programme, you do not build a classifier, and in most architectures you route it to a specialist service rather than touching the material yourself.
What to do when you detect something
Detection without a considered response path is worse than no detection, because it creates a record that you knew and did nothing coherent.
An automated signal should never, by itself, produce a punitive outcome. It should produce friction, protection, or a review. Punitive action — suspension, ban, report — stays human-gated. This protects wrongly-flagged players, and it protects you from acting on a model's mistake at scale.
A graduated ladder
- Monitor — raise internal signal, change nothing for the user. Most detections should end here.
- Friction — a lightweight challenge, a rate limit, a cool-down on stranger contact. A genuine user passes with a shrug; an automated fleet pays a real cost.
- Protective constraint — restrict the specific capability at risk (stranger DMs) rather than banning the account. Protect the potential victim first; that does not require punishing anyone yet.
- Human review — a person looks, with the evidence assembled for them.
- Action — suspension, removal, or a report, decided by a human, recorded with its reasoning.
Two things belong at the top of that ladder, not the bottom: care for the minor (safety messaging, reporting routes, resources — not just enforcement against the adult), and evidence preservation at the moment of decision, before anything is deleted.
Obligations to report child sexual abuse material to the relevant national body are legal duties with specific procedures. They are also not something to automate casually — filing is typically a deliberate human action, and preserving the evidence correctly matters as much as the report. Know which body you report to in each jurisdiction you serve, and get the process reviewed before you need it, not during your first incident.
Records, evidence and retention
You will eventually have to answer one of three questions, possibly all three:
- A regulator: "Describe your controls and show us they operated."
- A platform, publisher, or acquirer in diligence: "Prove your safety claims."
- Law enforcement or a court: "Produce what you have on this account and this incident."
All three are answered by the same thing: a durable record of what you decided, when, and why.
What a safety decision record should contain
| Field | Why it matters |
|---|---|
| What was decided | The outcome — allowed, constrained, escalated, actioned. |
| Policy version in force | Lets you reconstruct the decision under the rules that applied then, not the rules you have now. Frequently forgotten; extremely hard to backfill. |
| Signals that contributed | Reason codes, not raw content where avoidable. |
| Timestamp and actor | Automated or human, and which human. |
| Human review status | Whether a person confirmed it — the difference between a machine output and a considered decision. |
| Integrity marker | Something that lets a third party verify the record has not been altered since. |
The apparent conflict between "preserve evidence" and "delete personal data on request" dissolves once you separate the protected content from the tamper-evident history. You can redact or destroy the readable material while retaining a verifiable record that an event occurred, what was decided, and that the destruction itself was lawful and deliberate rather than an unexplained gap. Design for the record to outlive the content.
Retention periods vary by jurisdiction and by data type, and some obligations pull in opposite directions. Decide your retention policy deliberately, write it down, implement it in code, and — critically — make sure your public description of it matches what the code does.
Build or buy?
An honest breakdown for a team of two to twenty.
| Component | Verdict | Why |
|---|---|---|
| Adult/minor interaction controls | BUILD | It is your game's own social graph and permission model. Nobody can bolt this on for you, and it is the highest-value control. This is config and code paths, not ML. |
| Reporting & block flows | BUILD | Needs to live in your UI, in your language, in your player flow. Straightforward and expected. |
| Risk assessment & policy docs | BUILD | Must describe your game. A template helps; an outsourced document that doesn't match your product is a liability. |
| Age assurance | BUY | Specialist, regulated, and the accuracy and privacy engineering is genuinely hard. Building it yourself also means holding identity data you don't want. |
| Predatory-behaviour detection | BUY | Needs labelled data and adversarial testing you will not have. A naive in-house classifier gives false comfort. |
| Known-CSAM hash matching | BUY / route | Never build. Operates under controlled programmes; route to a specialist service and keep the material out of your systems. |
| Evidence preservation & audit trail | BUY | Easy to build badly. Tamper-evidence, retention, and lawful erasure interact in ways that are unpleasant to discover during a legal request. |
| Content moderation (text/image) | EITHER | Commodity APIs are cheap and decent, including free tiers. Buy the baseline; spend your own effort on the youth-specific risks above. |
Build what is specific to your game and cheap to get right. Buy what is specialist, regulated, adversarial, or dangerous to hold. A two-person studio can implement world-class adult/minor separation in a sprint. The same studio should not be training a grooming classifier.
Twelve questions to ask any safety vendor
Including us. If a vendor deflects on more than two of these, keep looking.
- What data do you retain about my players, and for how long?The best answer is "as little as possible, and here is exactly what." Vague answers here become your breach.
- Do you store identity documents when performing age assurance? Can you do it without?Retention is the liability. Privacy-preserving methods exist; ask why they aren't using one.
- Who holds the encryption keys to my evidence — you or me?If the vendor can read your evidence, so can anyone who breaches the vendor.
- Has this been independently red-teamed? Can I see the findings?Ask for the report, not the adjective. "Enterprise-grade security" is not a finding.
- What does your detection miss?Every system has a false-negative profile. A vendor who won't describe theirs either doesn't know it or won't tell you.
- What is your false-positive rate on normal teenage conversation?The number that determines whether your community hates the system.
- Can an automated flag ban a player without a human?The right answer is no.
- What happens when your service is down or your model provider is unavailable?Fail-open silently exposes players; fail-closed blocks legitimate ones. You need to know which, and to be able to choose.
- Can I export my own evidence, and verify it independently of you?If your audit trail only validates inside their console, you don't own it.
- What exactly do you mean by "compliant"?No vendor can make you compliant. A vendor provides controls; compliance is a determination about your service. Be wary of anyone who blurs this.
- How do you handle a legal request or a data-deletion request that conflicts with retention?Tests whether they have thought about the hard case.
- What happens to my data and my evidence if you go out of business?Ask for the exit path in writing. Very few vendors have a real answer.
A reference architecture
The minimum coherent shape, vendor-agnostic. Four stages, in this order.
PLAYER ACTION
│
▼
┌─────────────────────────────────────────────┐
│ 1. WHO IS THIS? │
│ age band + guardian authorisation │
│ → stored as a CLAIM, not a document │
└─────────────────────────────────────────────┘
│ age signal available to every gate below
▼
┌─────────────────────────────────────────────┐
│ 2. WHAT ARE THEY ALLOWED TO DO? │
│ capability gates, server-side │
│ adult↔minor separation enforced HERE │
│ (deterministic — no model involved) │
└─────────────────────────────────────────────┘
│ only permitted interactions proceed
▼
┌─────────────────────────────────────────────┐
│ 3. IS SOMETHING WRONG? │
│ cheap rules → owned classifier │
│ → expensive model (rare) │
│ fail toward protection on outage │
└─────────────────────────────────────────────┘
│ signals, never automatic punishment
▼
┌─────────────────────────────────────────────┐
│ 4. CAN WE PROVE WHAT WE DID? │
│ decision + policy version + reason codes │
│ tamper-evident, retention-aware │
└─────────────────────────────────────────────┘
│
▼
HUMAN REVIEW ──► action (the only path to punishment)
Three properties worth copying
- Order is meaningful. Identity before capability, capability before detection, detection before evidence. Each stage narrows what the next has to handle. Putting detection first means scanning traffic that structural controls should have prevented.
- Cascade your detection by cost. Cheap deterministic rules handle the bulk; an owned classifier handles most of the rest; an expensive model sees only genuinely ambiguous cases. This is what makes real-time safety affordable at player-base scale — and it means an outage at one provider degrades you rather than stopping you.
- Decide the failure posture per stage, deliberately. "Fail closed" is right for a gate on minor contact. "Fail open with a label" may be right for a credibility annotation. The wrong default in either direction is a safety incident. Write down the posture for each stage.
A 90-day plan for a small team
Ordered so that each phase produces something defensible on its own, in case you run out of time.
Days 1–30 — Know and document
- Write the children's risk assessment. Name every way a minor could be harmed on your service. This is the document everyone will ask for first.
- Inventory your comms surfaces — every path by which one player can reach another, including the ones you forgot (lobby names, custom emotes, profile bios).
- Determine your actual age demographics from telemetry, not assumption.
- Write down your current controls honestly, including the gaps. This becomes your baseline and your credibility.
Days 31–60 — Build the controls only you can build
- Implement adult/minor interaction separation. Highest value per engineering hour of anything on this list.
- Ship reporting and blocking flows that a 13-year-old can find and use in under ten seconds.
- Add capability gates server-side, so age governs actions rather than the login screen.
- Start logging safety decisions with policy version and reason codes — even before you have a detection system, because the labels you don't capture are gone.
Days 61–90 — Bring in what you should not build
- Integrate age assurance at the capability gates you identified.
- Add behavioural detection for the adult/minor surface.
- Stand up evidence preservation with a retention policy that matches your written policy.
- Run an adversarial test against your own controls — try to reach a minor as an adult on your own service, and write down what worked.
- Review the whole thing with counsel for your actual markets.
1. Write the risk assessment. 2. Separate adults from minors in your comms model. 3. Log your safety decisions with a policy version. Those three cost little, need no vendor, and put you ahead of most studios of your size.
Why we wrote this
We build safety infrastructure, and we also operate two live platforms — one of them for teenagers. Everything in this document is a lesson from running that youth platform, not a theory about someone else's.
We wrote it free and without a form wall because the studios most likely to get this wrong are the ones least able to pay for advice, and because a small studio that implements Part Five well is genuinely safer than a large one that bought a dashboard.
If you'd like the parts you shouldn't build yourself
The Catalyst Collection is our answer to Parts Four, Six, Seven and Eight: privacy-preserving age assurance and guardian consent (Ceremony), predatory-behaviour detection (Assurance), an auditable decision trail (Ledger), and tamper-evident evidence with lawful erasure (Evidence Preservation) — one integration, independently red-teamed, retaining as little as possible.
We are selecting 25 founding studios to run it in production, with hands-on onboarding and protected pricing. And if you'd rather just use this document and build it yourself — genuinely, good. That was the point.
Run the free Safety Readiness Assessment →