Optical music recognition that turns a scan of printed sheet music into MusicXML. The organizing idea is the opposite of end-to-end: every stage emits a typed intermediate representation and a debug artifact you can look at, so when the output is wrong you can see which stage was wrong.
Status is pre-alpha, and the numbers below say so honestly.
Where it actually stands
Measured across 18 pieces and 1,041 ground-truth notes:
| metric | value |
|---|---|
| Pitch accuracy | 84.8% |
| Duration accuracy | 69.4% |
| Note accuracy (pitch and duration) | 61.2% |
| Token error rate | 18.3% |
Reported separately on purpose. A single “61% accurate” headline hides the thing that matters: pitch is largely solved and duration is not. The error breakdown makes that unambiguous.
| error type | count |
|---|---|
| Duration | 307 |
| Pitch | 146 |
| Insertion | 32 |
| Deletion | 12 |
Duration accounts for more errors than everything else combined. The spread across pieces is
wide — suzuki_book_1_piece_1 reaches 97.9% note accuracy, while feuillard_exercise_1, a
dense beamed étude, collapses to 6.2% duration accuracy with pitch still at 64.6%. Reading
which note is a different problem from reading how long it lasts, and only the second one
depends on parsing beam groups, dots and ties correctly.
How it works
Notehead segmentation. A U-Net segments noteheads from the page, with a trained dynamic threshold policy rather than a fixed cutoff — printed scores vary enough in scan quality that one global threshold either drops faint heads or merges adjacent ones. Split-component merging recombines noteheads broken by a crossing stem or beam.
Pitch from geometry, not from a classifier. Staff and system geometry is recovered explicitly, so a notehead’s vertical position resolves to a staff degree and then a pitch through the key signature. This is why pitch accuracy sits 15 points above duration accuracy: it is a geometric measurement, not an inference.
Duration as sequence selection. Durations come from a supervised selector over candidate sequences with deterministic evidence terms, and measures are checked against their time signature — a measure that under- or over-fills is a detectable defect rather than a silent one.
Artifacts at every stage. The overlays on this page are not marketing renders; they are the pipeline’s own debug output, produced on every run. The error report that generated the tables above is a committed artifact with per-piece and per-error detail.
Why MusicXML-first
Symbolic output that a notation editor can open is worth far more than a bounding-box dump. It also makes evaluation tractable: two MusicXML files can be diffed symbolically, note against note, which is what produces the per-error-type counts rather than a pixel-level score that tells you nothing about what to fix.
Unverified / open items
- The U-Net recognition pipeline lives on an unpushed branch (
feat/unet-omr-pipeline). The public repository’smainis still the deterministic scaffold described in its README, so the numbers on this page are not reproducible from a fresh clone yet. - The 18-piece corpus is cello-repertoire-heavy (Suzuki Book 1, Feuillard) and small. These are fixture measurements, not a benchmark result against a standard OMR dataset.
- No comparison against an established baseline (Audiveris, oemer) has been run, so “how good is 61%” is genuinely open.