I built something that takes a recording, separates it into parts, finds the guitar and produces a draft tab you can read.
Then I measured it, which is the part most of these announcements skip.
Seven solos, transcribed by a human, 1,875 notes
The reference set was seven guitar solos that a professional player had transcribed himself. That comes to 1,875 reference notes.
Having a human transcription to measure against is the whole point. Without one you're left reporting that the output looks plausible, which isn't a measurement. The black-window diagnosis came out of the same habit - capture what the machine actually produced, then check it against something you didn't write.
About 90%, and 27%
Precision was about 90%. Nine in ten of the pitches it found were pitches the soloist played. That's the flattering figure and it's the one that would go in a launch post.
Against the same run, F1 on position was 27%. Same note, same place, scored in sequence. Precision and F1 aren't like for like - F1 folds in recall, so it's the harsher of the two by design, and that's why it's the one worth reporting.
Both figures describe the same run. Neither is wrong.
Finding the right note isn't the same as finding it in the right place
Precision asks whether the notes it produced were real. Position-scored agreement asks whether the sequence lines up with the human one, note for note, in order.
Roughly one note in ten is genuinely foreign - bass bleed from the separation, or the rhythm guitar underneath the solo. The rest are the right pitches, not always in exactly the right place.
If you're evaluating any model that produces a sequence, this gap is the thing to look for. A high set-level score and a low sequence-level score is a specific, diagnosable state, and a single headline percentage hides it completely.
Which number goes in front
The 27% sits next to the 90% wherever the accuracy is reported, with an explanation of why they differ.
The alternative is presenting a draft as a transcription, and letting someone find out the hard way in the middle of learning a solo.
So the tool reports the rest of the numbers too. String and fret agreement with the human transcription is 53%, which is why the fingering is described as a starting point rather than a result. Beat spacing is within 7 to 13 milliseconds and no bar fails to add up, which is why the rhythm can be trusted and the pitches can't.
There's also a feature that's built and switched off. Slide detection scored 3% precision and hammer-ons 13% against a 20% base rate, so it's disabled. A tab that says hammer-on where the player picked teaches the wrong hand, and a wrong articulation is worse than a missing one.
The part that keeps wanting to be left out is the error bar
I can only speak for the one thing I measured. But the pattern is easy to recognise once you've looked for it: capability is what gets announced, and the error bar is what gets omitted.
The number a builder chooses to publish tells you more about the work than the number itself does. It's the same discipline behind the speed comparison on this site, where both sites were measured the same day, on the same test, and both numbers went in.
Pages worth acting on
Related page
Technical profile
AI systems, automation and full-stack engineering work, set out properly.
View pageRelated page
Portfolio and case studies
Shipped work, with what got measured before and after.
View pageRelated page
WordPress performance optimisation
Where the measured-not-claimed habit earns its keep on a live site.
View pageRelated page
About Rod Fernandez Design
Who's doing the measuring, and why the error bar stays in.
View pageWhat to read next
A VST3 plugin opened as a black window, and the bug was in neither the plugin nor my code
One hosted plugin drew its editor. Another opened a window with nothing in it. The variable was the UI toolkit, and the fault was in the host library both of them sat behind.
Read articleCustom-coded vs WordPress: a real-world speed comparison
A rare like-for-like test. The same local business, the same services, one website replaced by another, both measured with Google PageSpeed Insights on the same day.
Read articleQuestions local business owners usually ask next
Built, and then checked
Anyone can show you something that works in a demo. The question worth asking is what happened when they measured it.