An artificial intelligence that can grade the skill of a pianist with near-human accuracy could be used in online music tutoring.
Brendan Morris at the University of Nevada, Las Vegas, and his colleagues selected almost 1000 short video clips of people playing piano from YouTube and got an expert pianist to manually grade each on a 10-point scale.
The researchers used half of these videos and their grades to train a neural network, a form of AI, creating a model that can assess piano playing in unseen videos. They then used the other half of the clips to test the model.
Advertisement
The team ran the test three times, first giving the AI access to just audio, then just video and finally to both together. The audio-only version was 65 per cent accurate when compared with the grades assigned by an expert, increasing to 74 per cent for video-only and 75 per cent for the combined version.
Morris says that although the software works well, it is unclear how the AI chooses grades. This is a common problem with neural networks.
“We don’t know specifically what it’s doing. As with a lot of AI, we can’t say exactly what’s happening,” says Morris.
The software could be doing anything from identifying a pianist’s ability to play two widely separated notes with one hand to their ability to quickly play notes at large intervals, he says. The AI has probably identified hundreds of small clues like these and taken them all into account in its assessment, he says.
Even without fully understanding how it works, Morris hopes the technique will eventually prove useful in tutoring, potentially reducing costs and lowering barriers to learning piano.
Reference: arXiv,
