WEEK 10 · INTER-ANNOTATOR AGREEMENT AND THE FIRST REAL ANALYSIS
Do one thing. Then you are ahead.
Two referees watching the same match will not call every foul the same way. If they disagree constantly, the rules are unclear, not the players. Cohen's kappa measures how much two annotators agree beyond what chance alone would give, and it is the number that tells reviewers whether your gold set is a standard or an opinion. This week you compute it per field, sit down with the disagreements, adjudicate them, and f…
Shields et al. (2021), "Bayesian reaction optimization as a tool for chemical synthesis"
Saturday sandbox
Shields et al. (2021), "Bayesian reaction optimization as a tool for chemical synthesis", Nature 590:89-96 — BO beating expert chemists, the self-driving-lab argument