# Week 22 — protein validation and baselines

[**Full learning notebook: explanations, step-by-step challenges, stashes and evidence**](./learning-notebook.md) · [NotebookLM export](./notebooklm.txt)

Status: NOT_STARTED. Calendar dates: not assigned. Owner: Peter Ciancaglini. Planning authority: v5 Part 7 with retained v4 details. Deeper walkthrough: [v6 · Week 22 · Baselines beaten honestly](../../../LABLOOP-Master-Plan-v6.md#week-22--baselines-beaten-honestly). Active project/repository: enzymelens.

[Programme](../../README.md) · [Model instructions](../../specs.md) · [Decisions](../../DECISIONS.md) · [Resources](../../SOURCES.md) · [**v6 walkthrough**](../../../LABLOOP-Master-Plan-v6.md#week-22--baselines-beaten-honestly) · [Previous week review](../week-21-protein-study-and-partner-scope/review.md) · [Next week](../week-23-extremophile-candidate-ranking/specs.md)

## Outcome and relevance

This week delivers the three concrete artifacts below and the supporting learning/branding/opportunity pack. The scientific or commercial outcome follows the preserved source brief, while the acceptance checks make its evidence testable. The career objective is to turn this work into something a scientist, hiring manager or buyer can inspect.

- Scientific/build priority: Define family/sequence-cluster grouped splits and keep homologous proteins out of independent partitions; document thresholds and counts.
- Public/profile priority: Prepare a baseline comparison post including comparability caveats and one failure case.
- Career/relationship/commercial priority: Use partner feedback to refine assay context; prepare technical interview answers on leakage and generalization.

## Deeper walkthrough (v6)

Read [v6 · Week 22 · Baselines beaten honestly](../../../LABLOOP-Master-Plan-v6.md#week-22--baselines-beaten-honestly) before Monday. It expands this week's one-row v5 brief into daily build steps and adds the **Welcome** analogy, **Learn it** (▶ core and ＋ deeper sources), **Run it** (repos with a 20-40 minute exercise), the **NotebookLM kit** and the mentor check below. Standing v6 rules: [resource rules](../../../LABLOOP-Master-Plan-v6.md#prologue-3--the-resource-rules-so-you-never-waste-an-evening-on-a-bad-source), [NotebookLM protocol](../../../LABLOOP-Master-Plan-v6.md#prologue-4--the-notebooklm-protocol-one-notebook-per-week-zero-decisions), [chapter anatomy](../../../LABLOOP-Master-Plan-v6.md#prologue-5--how-to-read-a-chapter-the-anatomy-identical-for-all-41). v6 adds depth only; it does not change a goal, gate, output or the 600-minute cap, so this specification still governs execution.

**Act V · Threads: M, EL, R · Next gate: 24**

**Where this week sits.**
- ← **From:** week 21 baselines, week 16 conformal prediction, week 4 grouped CV.
- → **Feeds:** week 23 (candidate ranking with uncertainty), week 24 (pre-registration defines success against these baselines), week 30 (validation analysis), week 31 (Paper 3 Methods).
- ⟳ **Returns in:** Paper 3 Table 1.

**Goals.**
- 🎯 **This week:** family-split CV; conformal intervals; comparison to FLIP published baselines; L-Sun with the table.
- 🏁 **Next milestone:** Gate 24.
- 🌟 **Final:** a Paper 3 no reviewer can call "leaky".

**The plan (from v5 Part 7, expanded; gate and output unchanged).**
> *v5 row:* 22 | Baselines beaten honestly | Brady Neal causal inference lecture 1 (for confounders in Paper 3's analysis) | Cross-validation by protein family (no leakage); conformal intervals; compare to FLIP published baselines | L-Sun with the table
- **Goal:** a baseline table a sceptical reviewer accepts.
- **Tue (study):** Brady Neal, *Introduction to Causal Inference*, lecture 1 + book ch. 1 (bradyneal.com/causal-inference-course).
- **Build:** [ ] Mon: cluster sequences by identity (e.g., MMseqs2 at 30-50% identity) and build grouped folds. Done when no cluster appears in two folds. [ ] Wed: re-run the three heads on grouped folds; conformal intervals; coverage. Done when the table shows random-split vs family-split side by side. [ ] Fri: FLIP published numbers in the same table, same metric; write the three-line honest interpretation. Done when `make baselines` regenerates it.
- **Sun:** L-Sun "Why my protein model got worse when I stopped letting it cheat" with the table.
- **Only one thing:** the family-split table.

**Mentor check.** *"I can explain why my score dropped with family splits, and why the lower number is the one I publish."*

## Required inputs and dependency check

- [Week 21](../week-21-protein-study-and-partner-scope/specs.md): inspect its actual evidence and review; source/code artifacts must exist before the dependent task begins.
- Read [commitment](./commitment.md), [evidence](./evidence.md), [review](./review.md) and the current root TODAY/PROGRESS/QUESTIONS files.
- Read this week's chapter, [v6 · Week 22 · Baselines beaten honestly](../../../LABLOOP-Master-Plan-v6.md#week-22--baselines-beaten-honestly), then the standing sections: [Part 0 · how to use the document](../../../LABLOOP-Master-Plan-v6.md#part-0--how-to-use-this-document-read-this-every-monday-for-the-first-month), [Part 3 · the scientific thesis](../../../LABLOOP-Master-Plan-v6.md#part-3--the-scientific-thesis-explained-twice-like-a-kid-then-like-a-reviewer), [Part 4 · curriculum](../../../LABLOOP-Master-Plan-v6.md#part-4--curriculum-from-zero-four-tracks-all-free-exact-units), [Part 8 · the prompt pack](../../../LABLOOP-Master-Plan-v6.md#part-8--the-prompt-pack-copy-paste-fill-the-brackets) and [Part 9 · branding and endorsement](../../../LABLOOP-Master-Plan-v6.md#part-9--branding-connection-ladder-endorsement). The same sections exist in the [v5 master](../../../LABLOOP-Master-Plan-v5.md), which remains the governing sequence; v6 reproduces them and adds depth. Retained commercial/instrument details come from [v4](../../../AI-Upskilling-Plan-Biotech-v4.md).
- Inspect the actual canonical repository state, available data, existing figures and required human/external inputs. Never assume a previous week succeeded from its folder name.
- Verify current resource links, versions, dataset rights and any live company/venue information before relying on them. Use [source catalog](../../SOURCES.md); explicitly record inaccessible resources.

## Seven-day plan: 600 minutes maximum

The model must turn these into exact physical actions using the observed files and available inputs. Each 90-minute block includes start/stop rituals. All calls, posts, debugging, reading and exercises consume this same capacity.

| Day | Budget | Work and stopping condition |
|---|---:|---|
| Monday | 90 min | Define family/sequence-cluster grouped splits and keep homologous proteins out of independent partitions; document thresholds and counts. Stop when: Produce protein-split-audit.md and one held-out baseline score with no family leakage; an unfavorable score is still evidence. |
| Tuesday | 90 min | Brady Neal causal inference lecture 1: confounding; draw a diagram of organism/family, assay context and observed stability. Produce: One concise applied note, a worked decision/example and unresolved questions. |
| Wednesday | 90 min | Evaluate heads against simple sequence/composition and published baselines where targets and splits are comparable. Stop when: MAE/RMSE/rank metrics, N and group-aware uncertainty accompany results. Reserve the last 10 minutes for the optional Wednesday note. |
| Thursday | 90 min | Apply grouped validation and interval calibration to this week's protein model; explain why homologs crossing splits inflate accuracy. Last 30 minutes: founder paper. Apply the Half 2 one-formal-study-block rule. |
| Friday | 90 min | Add calibrated intervals where assumptions are appropriate and record confounding and out-of-distribution failure cases. Stop when: A held-out table exists whether or not the stronger model wins. |
| Saturday | 90 min | Resolve the earliest critical spillover or dependency. Prepare any long-form publication draft from existing figures here if needed. Sandbox only after commitments are met. |
| Sunday | 60 min | 20 min build-log draft/publication when authorized; 5 min evidence/checklist; 20 min source-grounded gap review; 15 min next week's six physical actions. |

Half 2: Tuesday is the formal study block unless this is a selling/submission/decision week. Thursday supports applied review, partner work or the market task; do not add another full course.

## Learning specification

### Biology/domain unit

Material: Brady Neal causal inference lecture 1: confounding; draw a diagram of organism/family, assay context and observed stability.

Produce: One concise applied note, a worked decision/example and unresolved questions.

Understanding check: The note directly supports this week's experiment, collaboration, delivery or career action.

During preparation, write one worked example connected to the week's actual task, one exercise with a separate answer key and five questions. Peter supplies answers; score explanation quality and flag misconceptions in QUESTIONS. Numeric/scientific conclusions require actual data or explicitly synthetic teaching values.

### AI/scientific-method unit

Material: Apply grouped validation and interval calibration to this week's protein model; explain why homologs crossing splits inflate accuracy. Last 30 minutes: founder paper.

Produce: One concise applied note, a worked decision/example and unresolved questions.

Understanding check: The note directly supports this week's experiment, collaboration, delivery or career action.

Use the [prompt pack](../../../LABLOOP-Master-Plan-v6.md#part-8--the-prompt-pack-copy-paste-fill-the-brackets): P1 explanation ladder, P2 concept-to-project bridge and P3 quiz. Coding practice belongs within the allocated study/build block. Where the source asks for a favorable scientific result, substitute an honest measured comparison as explained below.

### Founder/scientist paper

Select one verified founder/scientific-lead paper relevant to this week's domain. Use [P11](../../../LABLOOP-Master-Plan-v6.md#part-8--the-prompt-pack-copy-paste-fill-the-brackets); save the question, method, numerical result with context, data workflow, three author questions and one connection sentence in founder-paper.md. Keep abstract-only access limitations explicit. On source weeks with no study, prioritize the commercial/submission task and log any skipped digest.

## Artifact contracts

These filenames are outputs to prepare during this week's execution; this scaffold does not claim they already exist. Drafts and measured results have different states. Technical code/data stay in the canonical project repository and are referenced from these coordinating files.

| Artifact | Required content / acceptance | Primary responsibility |
|---|---|---|
| `protein-split-audit.md` | Define family/sequence-cluster grouped splits and keep homologous proteins out of independent partitions; document thresholds and counts. Acceptance: Produce protein-split-audit.md and one held-out baseline score with no family leakage; an unfavorable score is still evidence. | Model assists; Peter checks and owns the outcome |
| `protein-comparison.csv` | Evaluate heads against simple sequence/composition and published baselines where targets and splits are comparable. Acceptance: MAE/RMSE/rank metrics, N and group-aware uncertainty accompany results. | Model assists; Peter checks and owns the outcome |
| `protein-uncertainty.md` | Add calibrated intervals where assumptions are appropriate and record confounding and out-of-distribution failure cases. Acceptance: A held-out table exists whether or not the stronger model wins. | Model assists; Peter checks and owns the outcome |
| `study-notes.md` | Selected exact free resources; bounded lesson/exercise; separate solution; quiz and grading rubric; Peter's answers and gaps when executed | Model prepares; Peter learns |
| `founder-paper.md` | Verified source and P11 digest; authentic question and connection sentence; access limitations | Model assists; Peter checks |
| `branding-pack.md` | Wednesday note, Sunday build log, X adaptation, figure/caption/alt text and the week's special asset; each claim linked to real proof | Model drafts; Peter owns voice/public claims |
| `opportunity-actions.md` | Qualified target/role, source/date, ladder stage, next action, message/interview/proposal draft and actual status | Model prepares; Peter owns external relationships |
| `evidence.md` | Actual paths/URLs, versions, checks/metrics, preparation versus execution state and external receipts when applicable | Updated after each deliverable |
| `review.md` | Checklist, learning assessment, gate status, shortfalls, next actions and carryover | Peter and reviewing model |

During preparation, future measured tables get column definitions, input requirements and run instructions with PENDING status. During execution, replace pending values only with measured outputs. Do not create realistic-looking fake evidence.

## Branding and opportunity execution brief

Branding: Prepare a baseline comparison post including comparability caveats and one failure case.

Opportunity action: Use partner feedback to refine assay context; prepare technical interview answers on leakage and generalization.

Use [career asset contracts](../../CAREER-ASSETS.md). Publish at most the agreed cadence; one figure and one supported number or admitted failure can support multiple drafts. In long-form weeks, prepare a 1,500–2,500-word article from existing evidence within the build/buffer allocation. In release/demo weeks, prepare a 60-second script: problem, input, command/action, result, limitation and proof link.

Verify individuals, jobs and affiliations at execution time. The [GridX list](../../../GridX%20Startups.csv) and v4 company map are research seeds, not current facts. Do not reveal named company weaknesses or employer/client confidential information. Keep drafted/sent/replied/booked/held statuses separate. The planning request alone does not authorize external messages or publication; honor explicit authorization in the execution session.

## Quality risks and week-specific constraints

Published FLIP and custom family-split scores cannot be ranked as apples-to-apples without reproducing the same task. No guaranteed conformal coverage under distribution shift.

Apply the root scientific evidence contract: provenance; meaningful independent splits; no future-data leakage; baseline comparison; actual denominators/uncertainty; failed cases; human review where required. State whether data are simulated, public experimental, synthetic or partner-provided.

Keep source gates intact, and log scope/time conflicts in review. Missing people, money, access or results must not be replaced with invented evidence. Reduce optional polish first. A specific essential external dependency stays pending while independent preparation continues.

## Completion and gate review

This is an enabling week. Its deliverables feed the next scheduled gate: 24. Complete the acceptance checklist below; do not promote reputation from a draft.

- [ ] Monday artifact satisfies: Produce protein-split-audit.md and one held-out baseline score with no family leakage; an unfavorable score is still evidence.
- [ ] Wednesday artifact satisfies: MAE/RMSE/rank metrics, N and group-aware uncertainty accompany results.
- [ ] Friday artifact satisfies: A held-out table exists whether or not the stronger model wins.
- [ ] Study pack is prepared; Peter's actual learning/quiz result is recorded separately.
- [ ] Branding and opportunity drafts use supported claims and the correct external-action state.
- [ ] Reproduction or document checks are recorded with file/version references.
- [ ] This week's special gate, if any, has real evidence or remains explicitly pending.
- [ ] Review, root progress/log and the next physical action are updated.

## Minimum viable week and recovery

Expanded minimum derived from the source goal: Produce protein-split-audit.md and one held-out baseline score with no family leakage; an unfavorable score is still evidence.

Operational first step if stalled: open `protein-split-audit.md`, write the required inputs and produce the smallest checkable example in ten minutes. If a more urgent missing gate dependency controls the week, use the source minimum above.

If the full week does not fit, use [P14](../../../LABLOOP-Master-Plan-v6.md#part-8--the-prompt-pack-copy-paste-fill-the-brackets): retain one gate-critical deliverable and one directly supporting study exercise; record what is skipped versus moved. Do not mark the original full gate passed merely because the minimum was completed. After a missed week, reforecast from actual evidence and avoid a second unexamined lost week.

## Handoff to the next model

In review, state exactly what exists, what was verified, what Peter can now explain, which external events remain pending, where the canonical code/data live, and the next file/function or draft paragraph to touch. Link each fact to evidence. Carry scientific assumptions, split definitions and rights constraints forward explicitly.

## Ready-to-copy model prompt

```text
Prepare Week 22 of this LABLOOP execution plan.
Read execution-plan/specs.md, execution-plan/DECISIONS.md,
execution-plan/GATES.md and
execution-plan/weeks/week-22-protein-validation-and-baselines/specs.md, commitment.md, evidence.md and review.md.
Read prior prerequisite reviews and inspect actual repository/data state.
Use v5 as the governing sequence, the v6 Week 22 chapter
(LABLOOP-Master-Plan-v6.md) as the deeper walkthrough, and v4 only for
retained details.
Produce the full local preparation pack and the exact week-specific files named
in the artifact table. Give me a bounded lesson with exercises/answer key,
daily physical actions, technical runbook or draft, branding drafts and
career/commercial actions. Keep the total to 600 minutes.
Separate prepared artifacts from measured results and my learning completion.
Verify sources and do not invent metrics, citations, papers, people or outcomes.
Do not execute external communications, publication or spending unless the
current session already authorizes the specific action.
Finish by updating evidence/review and TODAY with the next physical action.
```

## Preserved original v5 weekly brief

v6 expands this row into a full brief, with the gate and output unchanged, inside [v6 · Week 22 · Baselines beaten honestly](../../../LABLOOP-Master-Plan-v6.md#week-22--baselines-beaten-honestly).
The following is source planning text, not a claim of completion. Apply the explicit evidence corrections in this specification and DECISIONS.md.

| Week | Goal | Study (Tue) | Build (Mon/Wed/Fri) | Output |
|---|---|---|---|---|
| 22 | Baselines beaten honestly | Brady Neal causal inference lecture 1 (for confounders in Paper 3's analysis) | Cross-validation by protein family (no leakage); conformal intervals; compare to FLIP published baselines | L-Sun with the table |