What Is OMR? How AI Reads Sheet Music

What OMR is

OMR stands for Optical Music Recognition. It recognizes the staves, notes, and symbols in an image of sheet music and converts them into music data: which note to play, when, and for how long. Once the score is data, you can play it back, change the tempo, transpose it, or use it in a practice app. The point of OMR is to turn a paper score from something you read into data you can use. Researchers have been working on the problem since the 1960s. If you want the field's own definition of itself, the standard reference is the tutorial survey "Understanding Optical Music Recognition" by Calvo-Zaragoza, Hajič jr. and Pacha (ACM Computing Surveys, 2020).

How it differs from text OCR — why sheet music is harder

OMR is considered harder than text OCR. The reason is the structure of the information:

• Text is a one-dimensional sequence read left to right. Sheet music is two-dimensional: pitch runs vertically, time runs horizontally.
• Chords stack several notes vertically — one position carries multiple pieces of information.
• A single note's meaning is not self-contained. It depends on the clef, key signature, accidentals, and octave marks around it.
• Notes, rests, ties, slurs, and dynamics are drawn on top of the staff lines, which makes the symbols hard to separate.

There is a second difference that matters more than it sounds. In text OCR, getting a character wrong costs you one character. In OMR, getting a clef or a key signature wrong shifts every note that follows it — a single misread symbol can corrupt an entire system of music.

The four stages inside an OMR system

A conventional OMR system works in four stages, each feeding the next. Understanding where a system failed usually means identifying which of these four stages broke.

1. Preprocessing — deskew the page, correct curvature and uneven lighting, binarize the image, and locate the staves. Staff lines are removed or explicitly modeled at this point, because they run through every symbol on the page.
2. Symbol detection — find and classify the graphical primitives: noteheads, stems, beams, flags, rests, accidentals, clefs, dynamics. Modern systems do this with object-detection networks rather than the hand-written rules of earlier decades.
3. Notation assembly — reconstruct the musical meaning from the detected symbols. This is where a notehead plus a stem plus a position on a staff becomes a specific pitch and duration, and where clefs, key signatures, accidentals, and octave marks are applied.
4. Encoding — write the result into a music format. What the system can express here is limited by the format chosen, which is the subject of the next section.

Two architectures — staged pipeline or end-to-end

The four stages above describe the staged approach. The alternative is an end-to-end neural model that reads an image of a staff and emits a symbol sequence directly, with no explicit detection or assembly step. End-to-end models work well on monophonic, single-staff music. The Camera-PrIMuS dataset was built specifically to test them on realistic, camera-distorted images of that kind. Polyphonic piano music — two staves, stacked voices, cross-staff beams — is substantially harder, and remains an active research problem rather than a solved one. This distinction matters when you evaluate a product. A system that reads a single melody line reliably is not evidence that it reads piano scores reliably.

The output format decides what survives

OMR output is usually MusicXML, MEI, or MIDI, and the choice throws away different things.

• MusicXML and MEI are notation formats. They preserve how the music is written: note spelling (F♯ versus G♭), beams, slurs, ties, articulation, and page layout. They are what you want if the score has to remain a score.
• MIDI is a performance format. It records pitch, timing, and velocity — but not spelling, not beams, not slurs, not layout. A MIDI export of a score can play back correctly and still be unable to reproduce the page it came from.

So "the OMR worked" is not one claim but two. It can be accurate as a performance and lossy as notation at the same time. When comparing tools, check which format they emit before comparing their accuracy numbers.

Why accuracy is never 100%

Even practical OMR systems make mistakes. The main causes:

• Faded or smudged printing — symbol shapes degrade.
• Capture conditions — shadows, page curvature, and skew get in the way.
• Dense notation — in fast passages, symbols overlap each other.
• Layout variety — engraving conventions differ across publishers, and handwritten scores are harder still.

This is why practical OMR must let a human review and correct the result. No sheet music recognition today can honestly claim to be fully automatic and perfect.

How the field measures accuracy — and why numbers don't compare

If two OMR products both advertise an accuracy figure, those figures are usually not comparable. The field has no single agreed benchmark, and different systems report against different corpora and different metrics.

• Symbol Error Rate (SER) is the traditional measure. Its strength is that it tracks roughly how much human correction effort a result will need.
• Its weakness is that it is a single number. It tells you a page needed fixing without telling you whether the system failed on clefs, noteheads, beams, accidentals, or key signatures.
• Recent work targets exactly this gap: the Sheet Music Benchmark (2025) proposes OMR-NED, a normalized edit distance that breaks errors down by notation element instead of collapsing them into one score.

Practical reading: treat any single advertised accuracy percentage — including ours — as unverifiable across systems. Test a tool on your own scores instead.

The datasets OMR systems are trained and tested on

OMR progress is bounded by its datasets, and there are fewer of them than in text recognition. The main public ones:

• MUSCIMA++ — handwritten notation, built on CVC-MUSCIMA, with 91,255 annotated symbols covering both primitives and higher-level objects. The reference dataset for handwritten OMR.
• DeepScoresV2 — typeset notation, 255,385 images and roughly 151 million symbol instances across 135 classes. Synthetic, which is how it reaches that scale.
• PrIMuS — 87,678 real-music incipits distributed as image, MIDI, and MEI, with both semantic and agnostic encodings. Camera-PrIMuS adds photographic distortion to the same material.
• DoReMi — around 6,432 images with close to a million annotated objects across 94 classes, published with MusicXML and pixel masks.

Two things follow from that list. Most large-scale training data is synthetic or monophonic, which is part of why photographed polyphonic piano scores remain the hard case. And a maintained index of these corpora exists, so you do not have to hunt them down one paper at a time.

What OMR makes possible

Once a score is data, you can do things paper can't:

• Hear it — find out instantly how an unfamiliar score sounds.
• Change the tempo — practice at a speed you can manage, whatever the original tempo.
• Split the parts — isolate the right or left hand.
• Use it in a practice app — drive key-by-key guidance and MIDI keyboard feedback.

Hanon AI — GrandScore's own in-house AI

GrandScore develops its own OMR engine, Hanon AI, for printed piano sheet music. It shows you the recognized result as notation before anything is saved, which is the human-review step this page argues every practical OMR system needs. The score you keep goes straight into GrandScore's practice tools: visual guidance, slow tempo, one-hand practice, and MIDI connection.