Performance Calibration Without a Room: Running Reviews Across Time Zones

Calibration — the process where managers compare ratings across their teams to keep "exceeds expectations" meaning the same thing regardless of who's rating — was designed around a room. Everyone sits down together, argues through the borderline cases live, and lands on a consistent bar by the end of the meeting. That format assumes managers can all be in the same room, or at least the same working hours, at the same time. A lot of distributed companies don't have that, and running the traditional format anyway produces exactly the outcome calibration exists to prevent: whoever's awake and vocal at 3pm UTC shapes the bar, and managers nine time zones away either sit through it at an unreasonable hour or don't meaningfully participate at all.

What calibration is actually trying to solve

The underlying problem calibration exists to fix isn't a scheduling one — it's that individual managers rate against their own internal bar, which drifts between people even when everyone's acting in good faith. A manager who's generous by nature and a manager who's harsh by nature can give the same performance different ratings without either one being dishonest. Calibration exists to surface and correct for that drift. A live room is one way to do that, but it's not the only way, and it's a particularly bad fit for a team where "everyone in one room" means someone is joining at 11pm.

A written-first calibration process

1. Every manager writes ratings with evidence first, independently.Not just a number — a short written justification per person, tied to specific concrete outcomes, not general impressions. This is where most of the actual rating-quality work happens, and it doesn't need to be synchronous with anyone.
2. Ratings and justifications go into one shared doc, visible to all calibrating managers, with a deadline to review. Every manager reads every other manager's justifications for comparable levels before any discussion happens — this is the same deadline-to- object structure from a decision doc that survives a time zone gap applied to ratings instead of proposals.
3. Managers flag disagreements as async comments, tied to specific cases."I think this is rated a level too generous compared to [comparable case] — see the note" is concrete enough to resolve without a live conversation most of the time.
4. Only genuinely contested cases go to a live session. If the written round surfaces disagreement on 3 out of 40 ratings, that's a 30-minute call, not a two-hour meeting rehashing all 40 — and it's short enough that rotating the inconvenient hour fairly, per the fair rotation formula, is a much smaller ask.

Why the written round is often higher quality, not just more convenient

Live calibration rooms have a known failure mode independent of time zones: whoever argues most persistently or has the most organizational standing tends to move outcomes more than the actual evidence justifies, and quieter managers' well-reasoned cases get steamrolled by the clock running out. A written-first round forces every case to be argued in text, evaluated on its own before group dynamics enter the picture, which reduces — though doesn't eliminate — that specific bias. The live session that follows is then genuinely about resolving disagreement on the hard cases, not about establishing who's more comfortable pushing back in a meeting.

What still needs to be synchronous

Fully eliminating live discussion isn't the goal — some disagreements genuinely resolve faster in real conversation than in a comment thread, particularly ones involving nuance that's hard to fully capture in writing, like weighing two managers' different read on the same ambiguous outcome. The point of the written-first process is to shrink the live session to only the cases that actually need it, rather than making every manager sit through every case regardless of whether their input is needed.

Setting this up for a cycle

• Set a firm deadline for written ratings and justifications — treat it the same as any other async deadline, with real consequences for missing it, not a soft suggestion.
• Give every manager a fixed window (48–72 hours works well) to review peers' ratings and flag disagreements before scheduling anything live.
• Keep the live session tightly scoped to contested cases only, with a hard time limit, and rotate whose time zone absorbs the inconvenient hour across cycles.
• After the cycle, note which cases needed live discussion and why — patterns here often reveal where rating guidelines themselves are ambiguous, which is worth fixing before the next cycle rather than re-litigating the same ambiguity every time.