Modern Statistics for Modern Biology ch. 6 "Testing"
Modern Statistics for Modern Biology ch. 6 "Testing" (multiple testing, FDR) (60 min)
In: LABLOOP-W14 Notebook
WEEK 14 · Mon 21 Dec → Sun 27 Dec 2026
UNDERSTAND · PRACTICE · KEEP EVIDENCE
Read the week in plain language, refresh the foundations, follow each challenge step by step, and check what evidence to keep. Includes worked examples, answer keys and prompts for visual study resources.
Use this within your existing weekly blocks. Your learning completion and research evidence remain in your current tracker and week files.
START HERE · 2 MINUTES
STATUS
| Deliverable | Acceptance | State |
|---|---|---|
paper-1-introduction.mdBuild create + |
Write introduction and related work from verified sources, stating novelty or extension accurately. Acceptance: Each contribution corresponds to an implemented method or measured result. | |
paper-1-methods.mdBuild create + |
Write reproducible corpus, schema, annotation, extraction and evaluation methods with availability statement. Acceptance: A reader can identify data versions, splits, prompts, annotators and scoring rules. | |
paper-1-results.mdBuild create + |
Write results and limitations, apply the planned multiple-testing correction and evaluate any fuzzy-field judge against humans. Acceptance: Effect sizes and denominators accompany comparisons; judge output is not ground truth. | |
study-notes.mdLearning create + |
Selected exact free resources; bounded lesson/exercise; separate solution; quiz and grading rubric; Peter's answers and gaps when executed | |
founder-paper.mdFounder papers create + |
Verified source and P11 digest; authentic question and connection sentence; access limitations | |
branding-pack.mdBranding create + |
Wednesday note, Sunday build log, X adaptation, figure/caption/alt text and the week's special asset; each claim linked to real proof | |
opportunity-actions.mdOpportunities create + |
Qualified target/role, source/date, ladder stage, next action, message/interview/proposal draft and actual status | |
evidence.mdEvidence & review open & edit ↗ |
Actual paths/URLs, versions, checks/metrics, preparation versus execution state and external receipts when applicable | |
review.mdEvidence & review open & edit ↗ |
Checklist, learning assessment, gate status, shortfalls, next actions and carryover |
These filenames are outputs to prepare during this week's execution; this scaffold does not claim they already exist. Drafts and measured results have different states. Technical code/data stay in the canonical project repository and are referenced from these coordinating files.
During preparation, future measured tables get column definitions, input requirements and run instructions with PENDING status. During execution, replace pending values only with measured outputs. Do not create realistic-looking fake evidence.
600 MINUTES MAXIMUM
Target. Write introduction and related work from verified sources, stating novelty or extension accurately. Stop when: Each contribution corresponds to an implemented method or measured result.
No material for Monday — this is a pure build day. Need something to study instead? →
Target. MSMB ch. 6 (testing, multiple testing) Produce: Multiple-testing correction applied to a FermLit comparison
Do these in this order — the plan's own sequence:
Target. Write reproducible corpus, schema, annotation, extraction and evaluation methods with availability statement. Stop when: A reader can identify data versions, splits, prompts, annotators and scoring rules. Reserve the last 10 minutes for the optional Wednesday note.
No material for Wednesday — this is a pure build day. Need something to study instead? →
Target. Stanford CS336 lecture 12 (evaluation) — cs336.stanford.edu; CS224N benchmarking lecture (judge biases) Select material/exercise that fits 60 minutes; use the last 30 for the founder paper.
Do these in this order — the plan's own sequence:
Target. Write results and limitations, apply the planned multiple-testing correction and evaluate any fuzzy-field judge against humans. Stop when: Effect sizes and denominators accompany comparisons; judge output is not ground truth.
No material for Friday — this is a pure build day. Need something to study instead? →
Target. Resolve the earliest critical spillover or dependency. Prepare any long-form publication draft from existing figures here if needed. Sandbox only after commitments are met.
Optional, only if the day went well:
Target. 20 min build-log draft/publication when authorized; 5 min evidence/checklist; 20 min source-grounded gap review; 15 min next week's six physical actions.
No material for Sunday — this is a pure build day. Need something to study instead? →
Spare pockets, repo exercises and anything the plan lists without a day. Take them in this order when a day finishes early.
The model must turn these into exact physical actions using the observed files and available inputs. Each 90-minute block includes start/stop rituals. All calls, posts, debugging, reading and exercises consume this same capacity.
The source lists more material than necessarily fits. Select exact excerpts for the learning objective and park the remainder; no full-course completion requirement.
WHAT TO WATCH, READ AND RUN
Writers have a saying: you do not write a book, you write a page a day and one day it is a book. This week you write three sections in three build blocks, and the order matters. Monday: Introduction and Related work, the "why should anyone care, and what exists already". Wednesday: Methods, the recipe precise enough that another cook produces the same dish. Friday: Results and Limitations, the plates on the table an…
0 of 12 finished · give me one to start now → · open in the deck →
In the plan's running order: each card says which block it belongs to and what has to come before it. The seven-day plan lists the same cards under the day that owns them.
Modern Statistics for Modern Biology ch. 6 "Testing" (multiple testing, FDR) (60 min)
In: LABLOOP-W14 Notebook
Tuesday · after Modern Statistics for Modern Biology ch. 6 "Testing"
StatQuest "False Discovery Rates, FDR, clearly explained" and "p-hacking: What it is and how to avoid it!" (YouTube, 20 min)
In: LABLOOP-W14 Notebook
Stanford CS336 lecture on evaluation (cs336.stanford.edu, videos on Stanford Online YouTube) (60 min); your week 7 notes on Zheng et al. 2023 judge biases (15 min)
In: LABLOOP-W14 Notebook
Saturday sandbox
Gopen & Swan (1990), "The Science of Scientific Writing", American Scientist 78:550-558. The best 9 pages ever written on why readers get lost
In: LABLOOP-W14 Notebook
Saturday sandbox · after Gopen & Swan (1990), "The Science of Scientific Writing"
Whitesides (2004), "Whitesides' Group: Writing a Paper", Advanced Materials 16(15):1375-1377. Outline-first writing from a legendary chemist
In: LABLOOP-W14 Notebook
Saturday sandbox · after Whitesides (2004), "Whitesides' Group: Writing a Paper"
Benjamini & Hochberg (1995), "Controlling the false discovery rate", JRSS B 57(1):289-300 (abstract + procedure)
In: LABLOOP-W14 Notebook
Saturday sandbox · after Benjamini & Hochberg (1995), "Controlling the false discovery rate"
Scientific Data "Data Descriptor" article guidelines (nature.com/sdata) — preview of the week 36 venue
In: LABLOOP-W14 Notebook
Writing · Coursera Writing in the Sciences (Stanford), modules 1-2 (cutting clutter, active voice) — watch at 1.5x during Saturday buffer if needed
In: LABLOOP-W14 Notebook
statsmodels.stats.multitest.multipletests with fdr_bh on the journal comparison; report adjusted p-values
In: LABLOOP-W14 Notebook
Implement the fuzzy-field judge as a scorer; log judge vs human agreement on 50 pairs
In: LABLOOP-W14 Notebook
Read how tasks define metrics and few-shot prompts; borrow the discipline, not the tool
In: LABLOOP-W14 Notebook
Point it at your RELATED.md PDFs and ask "which of these papers report inter-annotator agreement?" to double-check your related-work claims (verify every answer by opening the paper)
In: LABLOOP-W14 Notebook
READ FIRST
This week delivers the three concrete artifacts below and the supporting learning/branding/opportunity pack. The scientific or commercial outcome follows the preserved source brief, while the acceptance checks make its evidence testable. The career objective is to turn this work into something a scientist, hiring manager or buyer can inspect.
READ FIRST
Read v6 · Week 14 · Write Paper 1 before Monday. It carries this week's v5 brief unchanged and adds the Welcome analogy, Learn it (▶ core and + deeper sources), Run it (repos with a 20-40 minute exercise), the NotebookLM kit and the mentor check below. Standing v6 rules: resource rules, NotebookLM protocol, chapter anatomy. v6 adds depth only; it does not change a goal, gate, output or the 600-minute cap, so this specification still governs execution.
Act IV · Threads: B, M, FL · Next gate: 16
Where this week sits. - ← From: week 13 skeleton and reviewer list; weeks 5-11 artifacts. - → Feeds: week 15 (review and human read), week 16 (submission), week 31 (Paper 3 uses the same writing process), week 36 (journal version). - ⟳ Returns in: every future paper: your Methods style becomes your signature.
Goals. - 🎯 This week: full draft of Paper 1 (Intro, Related work, Methods, Data availability, Results, Limitations); multiple-testing correction applied; calibrated judge for fuzzy fields. - 🏁 Next milestone: Gate 16 (bioRxiv DOI). - 🌟 Final: L3 preprinted, the rung that makes founders answer.
Mentor check. "My Methods section lets a stranger rebuild FermLit, and I can explain why I corrected for multiple testing."
OPEN WHEN NEEDED
READ FIRST
Source unit: B14 · Statistics for biologists II
Material: MSMB ch. 6 (testing, multiple testing)
Produce: Multiple-testing correction applied to a FermLit comparison
Understanding check: You can defend an effect in Paper 1
During preparation, write one worked example connected to the week's actual task, one exercise with a separate answer key and five questions. Peter supplies answers; score explanation quality and flag misconceptions in QUESTIONS. Numeric/scientific conclusions require actual data or explicitly synthetic teaching values.
Source unit: M14 · Evaluation and judges
Material: Stanford CS336 lecture 12 (evaluation) — cs336.stanford.edu; CS224N benchmarking lecture (judge biases)
Produce: Calibrated judge for FermLit fields
Understanding check: You can explain judge-human agreement
Use the prompt pack: P1 explanation ladder, P2 concept-to-project bridge and P3 quiz. Coding practice belongs within the allocated study/build block. Where the source asks for a favorable scientific result, substitute an honest measured comparison as explained below.
Select one verified founder/scientific-lead paper relevant to this week's domain. Use P11; save the question, method, numerical result with context, data workflow, three author questions and one connection sentence in founder-paper.md. Keep abstract-only access limitations explicit. On source weeks with no study, prioritize the commercial/submission task and log any skipped digest.
OPEN WHEN NEEDED
Branding: Draft a methods-section learning note and a plain-language explanation of one supported result.
Opportunity action: Schedule the human scientific review now, with scope and return date before week 16.
Use career asset contracts. Publish at most the agreed cadence; one figure and one supported number or admitted failure can support multiple drafts. In long-form weeks, prepare a 1,500–2,500-word article from existing evidence within the build/buffer allocation. In release/demo weeks, prepare a 60-second script: problem, input, command/action, result, limitation and proof link.
Verify individuals, jobs and affiliations at execution time. The GridX list and v4 company map are research seeds, not current facts. Do not reveal named company weaknesses or employer/client confidential information. Keep drafted/sent/replied/booked/held statuses separate. The planning request alone does not authorize external messages or publication; honor explicit authorization in the execution session.
OPEN WHEN NEEDED
Do not use an LLM judge to erase strain-level mistakes or support a reporting-completeness claim without manual validation. Unsupported hypotheses stay in limitations or future work.
Apply the root scientific evidence contract: provenance; meaningful independent splits; no future-data leakage; baseline comparison; actual denominators/uncertainty; failed cases; human review where required. State whether data are simulated, public experimental, synthetic or partner-provided.
Keep source gates intact, and log scope/time conflicts in review. Missing people, money, access or results must not be replaced with invented evidence. Reduce optional polish first. A specific essential external dependency stays pending while independent preparation continues.
READ FIRST
This is an enabling week. Its deliverables feed the next scheduled gate: 16. Complete the acceptance checklist below; do not promote reputation from a draft.
READ FIRST
Source-preserving minimum: Methods section.
Operational first step if stalled: open paper-1-introduction.md, write the required inputs and produce the smallest checkable example in ten minutes. If a more urgent missing gate dependency controls the week, use the source minimum above.
If the full week does not fit, use P14: retain one gate-critical deliverable and one directly supporting study exercise; record what is skipped versus moved. Do not mark the original full gate passed merely because the minimum was completed. After a missed week, reforecast from actual evidence and avoid a second unexamined lost week.
OPEN WHEN NEEDED
In review, state exactly what exists, what was verified, what Peter can now explain, which external events remain pending, where the canonical code/data live, and the next file/function or draft paragraph to touch. Link each fact to evidence. Carry scientific assumptions, split definitions and rights constraints forward explicitly.
OPEN WHEN NEEDED
Prepare Week 14 of this LABLOOP execution plan.
Read execution-plan/specs.md, execution-plan/DECISIONS.md,
execution-plan/GATES.md and
execution-plan/weeks/week-14-paper-1-full-draft/specs.md, commitment.md, evidence.md and review.md.
Read prior prerequisite reviews and inspect actual repository/data state.
Use v5 as the governing sequence, the v6 Week 14 chapter
(LABLOOP-Master-Plan-v6.md) as the deeper walkthrough, and v4 only for
retained details.
Produce the full local preparation pack and the exact week-specific files named
in the artifact table. Give me a bounded lesson with exercises/answer key,
daily physical actions, technical runbook or draft, branding drafts and
career/commercial actions. Keep the total to 600 minutes.
Separate prepared artifacts from measured results and my learning completion.
Verify sources and do not invent metrics, citations, papers, people or outcomes.
Do not execute external communications, publication or spending unless the
current session already authorizes the specific action.
Finish by updating evidence/review and TODAY with the next physical action.
OPEN WHEN NEEDED
v6 reproduces this brief unchanged inside v6 · Week 14 · Write Paper 1. The following is source planning text, not a claim of completion. Apply the explicit evidence corrections in this specification and DECISIONS.md.
RELATED.md, every citation verified via Semantic Scholar). Wed: Methods (corpus, schema, guidelines, agents, evaluation, kappa) and Data availability. Fri: Results (completeness finding, per-field F1, descriptive tables) and Limitations (honest: extraction errors, OA bias, English-only).WEEK FILES
State: UNCONFIRMED. Owner: Peter Ciancaglini. Specification
Start date: not set. End date: not set. Available hours: confirm (planned ceiling 10). Primary outcome: paper 1 full draft. Minimum that keeps the next gate alive: Methods section.
paper-1-introduction.md: Each contribution corresponds to an implemented method or measured result.paper-1-methods.md: A reader can identify data versions, splits, prompts, annotators and scoring rules.paper-1-results.md: Effect sizes and denominators accompany comparisons; judge output is not ground truth.| Day | Minutes | Physical action / outcome | Actual slot | Done |
|---|---|---|---|---|
| Mon | 90 | Write introduction and related work from verified sources, stating novelty or extension accurately. | Not set | ☐ |
| Tue | 90 | Multiple-testing correction applied to a FermLit comparison | Not set | ☐ |
| Wed | 90 | Write reproducible corpus, schema, annotation, extraction and evaluation methods with availability statement. Last ten minutes: optional note. | Not set | ☐ |
| Thu | 90 | Calibrated judge for FermLit fields Include relevant founder-paper work within this block. | Not set | ☐ |
| Fri | 90 | Write results and limitations, apply the planned multiple-testing correction and evaluate any fuzzy-field judge against humans. | Not set | ☐ |
| Sat | 90 | Resolve critical spillover; long-form draft if due; sandbox only if done. | Not set | ☐ |
| Sun | 60 | Build log, evidence, gap review, next actions. | Not set | ☐ |
Peter's commitment / adjustments: pending. Reviewing model and date: pending.
Execution state: NOT_STARTED. Specification This file is a blank register with planned artifact names; none of the rows prove completion.
| Item | Path / URL / version | State | Verification / actual result | Verified by / date |
|---|---|---|---|---|
| paper-1-introduction.md | Not created | NOT_STARTED | Pending: Each contribution corresponds to an implemented method or measured result. | — |
| paper-1-methods.md | Not created | NOT_STARTED | Pending: A reader can identify data versions, splits, prompts, annotators and scoring rules. | — |
| paper-1-results.md | Not created | NOT_STARTED | Pending: Effect sizes and denominators accompany comparisons; judge output is not ground truth. | — |
| Learning exercise / Peter's quiz | Not created | NOT_STARTED | Peter's actual answers required | — |
| Founder paper digest | Not created | NOT_STARTED | Source/content and access checked | — |
| Branding pack | Not created | NOT_STARTED | Claims audited; drafted/public status distinct | — |
| Opportunity actions | Not created | NOT_STARTED | Drafted/sent/replied/booked/held distinct | — |
| Question / metric | Dataset, N and unit | Split / seed / config | Command or verification method | Output path and measured value | Uncertainty / limitation |
|---|---|---|---|---|---|
| Date | Action / counterpart | Receipt or private reference | State | Next action / date |
|---|---|---|---|---|
Peter's teach-back / quiz score: not assessed. Scientific assumptions and deviations: none recorded yet. Budget allocated / spent: not assessed / no spend recorded. Missing evidence and responsible owner: pending inspection.
Do not put API keys, source full texts with restricted rights, client data or invoice personal details in a public version of this register. Keep private evidence references private.
Review status: NOT_REVIEWED. Specification · Evidence
What Peter can explain or reproduce now: not assessed. Quiz / teach-back gaps: not assessed. Measured result and limitations: pending. Best evidence link for a scientist/recruiter/buyer: pending. What failed or changed, and why: pending.
Gate requirement: No standalone gate; check this week's acceptance criteria and next gate dependencies. Gate status: NOT_STARTED. Original target week: See next gate in specs. Actual completion date / forecast: not set. Missing external evidence: pending inspection. Blocks completed / 7: not recorded. Days with commits / 7: not recorded. Sunday build log / optional Wednesday note: not recorded. Founder conversations cumulative: not recorded. Earned evidence level and proof: not assessed. Consecutive lost weeks: not assessed.
Week-specific risk to revisit: Do not use an LLM judge to erase strain-level mistakes or support a reporting-completeness claim without manual validation. Unsupported hypotheses stay in limitations or future work.
Continue / recover / blocked: not decided. Retain as the minimum: Methods section. Skipped permanently with reason: none recorded. Carryover with owner, dependency and target week: none recorded. Canonical repository paths and versions: pending. Data permissions, split definitions or assumptions to preserve: pending. External action authorization that still applies: not assessed. Next physical action, exact file/function/paragraph: pending. Updates to root PROGRESS / LOG / TODAY completed: ☐
YOUR SPACE