parkinson, Parkinson's research notebook

The tasks

Replication. A prediction reached p < 0.05 in its discovery half of the donors. Will it hold in the other half? The analysts see only the discovery-half donors. The split is rebuilt from the seed written down when the prediction was registered, so the answer can't leak in.

Follow-up. The swarm proposed a new prediction after a failure. Will it point the same way, at p < 0.05, in the Kamath et al. 2022 cohort? The analysts see only the first atlas.

Each answer is a probability, scored with the Brier score: the average squared distance between the probability and what happened (0 is perfect; always answering the base rate is the bar to beat). AUC says how well the probabilities rank the hits above the misses (0.5 is chance).

power        chance the effect clears the next test, given its size and the donors
confounder   share of the effect left after age, sex and post-mortem interval
composition  share of the effect left after the cell group's share of nuclei
cell type    how well the gene is measured in this group
axis         whether the gene moves the expected way on the other pathology measure
skeptic      the worst p after dropping one donor; the effect without the most severe

Replication

125 tasks: 87 train genes, 38 dev genes. 2 held (2%). Base rate on train: 0.011. Scores below are on the dev genes only; lower Brier is better, AUC 0.5 is chance.

model	Brier	AUC	dev tasks
always the base rate	0.026	0.500	38
six analysts, equal weight	0.448	0.216	38
power alone	0.499	0.108	38
confounder alone	0.848	0.730	38
composition alone	0.748	0.162	38
cell type alone	0.719	0.703	38
axis alone	0.026	0.500	38
skeptic alone	0.500	0.041	38
tuned on train genes	0.026	0.135	38

fitted weights (train genes)
  intercept    -4.64
  power        +0.08
  confounder   +0.04
  composition  +0.09
  cell type    +0.00
  axis         +0.00
  skeptic      +0.02

Follow-ups

251 tasks: 172 train genes, 79 dev genes. 53 held (21%). Base rate on train: 0.186. Scores below are on the dev genes only; lower Brier is better, AUC 0.5 is chance.

model	Brier	AUC	dev tasks
always the base rate	0.202	0.500	79
six analysts, equal weight	0.433	0.445	79
power alone	0.559	0.469	79
confounder alone	0.725	0.490	79
composition alone	0.710	0.580	79
cell type alone	0.496	0.492	79
axis alone	0.275	0.491	79
skeptic alone	0.563	0.466	79
tuned on train genes	0.191	0.548	79

fitted weights (train genes)
  intercept    -0.89
  power        -0.36
  confounder   -0.14
  composition  -0.08
  cell type    +0.33
  axis         +0.08
  skeptic      -0.44

Where the tuned analysts were most wrong

STX12 in astrocytes, Lewy stagesaid 17% it would holdit held
GPR37L1 in microglia, Lewy stagesaid 17% it would holdit held
UBIAD1 in oligodendrocytes, Lewy stagesaid 17% it would holdit held
LAMTOR3 in oligodendrocytes, Lewy stagesaid 17% it would holdit held
KCNG2 in OPCs, Lewy stagesaid 17% it would holdit held
SLC30A7 in oligodendrocytes, Lewy stagesaid 18% it would holdit held

Sealed, then scored on the future

A score on held-out genes can still flatter a method that was chosen after looking. So the tuned weights are frozen, hashed, and scored only on predictions registered after that moment. The agent makes new predictions every few minutes and the swarm keeps proposing follow-ups, so this test set grows on its own.

sealed      2026-10-23T13:45:11Z
hash        dbbc59373b0341489c8aa4fcb8d3e32b88f905302587afb433ff2a3399b78324
trained on  101 replication tasks, base rate 0.050

after the seal   replication: 125 tasks, Brier 0.017 (base rate 0.017), AUC 0.528
                 follow-up:   95 tasks, Brier 0.116, AUC 0.459

Rules