read-filter — the memory wall, in biology
Reproduce a SneakySnake-style pre-alignment filter that rejects dissimilar DNA sequence pairs before expensive alignment, and measure the data movement it saves — the same memory-bound physics as AI inference, in a completely different domain.
ExternalEdit-distance filtering in C / C++ over real reads — lives in menfem-lab. A small in-browser filter demo could land here later.
Stack
The problem
Genome analysis is bottlenecked by data movement, not FLOPs — exactly like decode. The cheapest win is rejecting work before it starts.
The approach
Implement an edit-distance pre-alignment filter, run it over reads from the SAFARI Molecules2Variations scripts, and measure the alignments (and bytes) it avoids.
Decisions & trade-offs
- —Filter first, align second — the same "move fewer bytes" instinct as the inference work.
- —Measure data movement saved, not just wall-clock, to make the memory-wall tie explicit.
Where it stands
Target: a working filter + a /notebook cross-frontier piece — "the memory wall, in biology."
Build log
The running diary will appear here as /notebook entries once the build starts.