Pitch darkness. Then a single spotlight on a piano in the middle of the stage. A figure walks on – head down, almost apologetic – and sits. The auditorium is silent. There is a gentle cascade of notes, fingers rippling the length of the keyboard again and again in the harmonic equivalent of rotating a kaleidoscope. The audience seem barely to breathe: hundreds of people all joined in intense, ecstatic concentration, cameraphones held reverentially aloft.

What music inspires such rapturous attention? On this occasion, it is the bearded Ukrainian pianist-composer Lubomyr Melnyk playing what he calls “continuous music”. Lots of arpeggios (yes, the ones you practised for hours if you ever had piano lessons, going up and down the notes that form a chord) and lots of sustaining pedal. There is not a huge amount else on the list of musical ingredients – but the euphoric absorption displayed by all those listeners outstrips just about anything I’ve ever witnessed at more mainstream classical performances.

Yet Melnyk is far from alone in generating such responses. Take Dutch pianist-composer Joep Beving. His music is subdued, softly intimate and pedal-heavy. Understatement comes as standard. He has described what he creates as “simple music for complex emotions”. He also clocks up more monthly listeners on Spotify than classical piano legends András Schiff, Mitsuko Uchida and Alfred Brendel. Combined.

Beving and Melnyk are two figures in a huge musical phenomenon unfolding on the fringes of classical music. It can be heard on film and TV soundtracks, adverts and call-centre hold music, and behind that trickledown lies an enormous number of albums, singles and playlists by pianist-composers now attracting more listeners than most big names of classical piano performance. Did you think Yuja Wang, the 37-year-old Chinese phenomenon, was one of the most famous pianists in the world? Think again.

Beving, Melnyk and similar artists work in a solo piano soundworld sometimes known as “ambient” or “neoclassical” or “postminimalist” – although categories aren’t really its thing. As one fan put it to me with a hint of impatience: “It’s all just stuff, you know?” And this particular stuff is about soulful simplicity. A stripped-back aesthetic. Quiet melancholy. It is a world of arpeggios and gentle undulations, of atmosphere rather than athleticism.

There’s nothing new in that, you might think if you’re classically inclined. And true, there are slow movements by Mozart that might fit the bill, not to mention meditative keyboard pieces by Bach or Couperin. Or Erik Satie, whose groundbreaking experiments in ambient composition saw him labelled an eccentric in the late 19th century. Yet none of these continuities can explain one of the most striking aspects of the 21st-century “sad-piano” phenomenon: the fact that the classical music establishment has largely ignored it.

Take Riopy, a French pianist-composer who spent his childhood in a cult, ultimately escaping it, and now records his signature brand of meditative keyboard-rippling for Warner Classics. He may not be a household name yet he has more than 725,000 monthly listeners on Spotify, easily exceeding Wang’s 645,000.

More obviously avant garde is Berlin-based composer and musician Nils Frahm, who was taught piano as a child by Nahum Brodsky, himself a pupil of composer-pianist Sergei Rachmaninov. “He made me practise Russian style,” Frahm tells me, “although without the weight on my fingers.” With a grin, he adds: “And no bleeding was involved.” Frahm’s music does attract critical attention – not to mention 1.4 million listeners a month on Spotify – but he is still treated as “pop”, not “classical”.

This may be strange considering that, in the course of our conversation, Frahm talks energetically about a vast array of musicians, from Thelonious Monk and Valentin Silvestrov to JS Bach and Heinrich Biber. His compositions often use electronics as well as the piano (his 2022 track Brainwash sounds like the cheerful progeny of US minimalist Steve Reich and an arcade game). But the mechanism-heavy aesthetic of Frahm’s solo piano pieces is instantly recognisable to those in the know – and it is widely imitated.

Beving, despite huge listener figures on Spotify (1.8m per month), also flies beneath the radar of classical music writers – this despite being signed to Deutsche Grammophon (DG), another explicitly “classical” record label. Like Frahm, Beving had some formal piano training as a child. He was accepted on a conservatoire course while studying at university, before repetitive strain injury forced him to drop out. “I had very, very little time to do my Czernys,” he confesses from his studio, referring to the studies that have been a foundation of piano training for almost two centuries. “And so yeah, my technique was really bad. Still is.”

That didn’t trouble Christian Badzura, vice president of DG’s A&R New Repertoire department, who signed Beving after hearing his self-released first album in a Berlin bar. When I ask via email whether Badzura defines Beving’s music as “classical”, he fires back: “Just because it is not a late Beethoven sonata or a complex atonal piece, why should it not be part of the classical music repertoire?” That surely depends on who you think polices the endlessly movable boundaries of classical music.

Which brings us to the Italian pianist-composer Ludovico Einaudi. His music does dominate the UK classical charts, and he boasts an astonishing 8.1m monthly listeners on Spotify – more than Bach, Beethoven or any classical performer, dead or alive. And you can’t say Einaudi is ignored by the critical establishment – in fact, critics of all stripes love to hate him. “Einaudi’s music speaks fluent cliche,” diagnosed a classical reviewer in the Guardian back in 2019. A few years earlier, a pop critic in the same paper detected “the balladry of Westlife but without their clarity of purpose”. More recently, the Times simply branded him “the middlebrow maestro”.

This isn’t purely a case of snobbery. For hundreds of years, classical music has valued musical “development”, meaning today’s composers are supposed to function as the next step in an onwards march of evolution. In that context, the perma-tonality and looping structures of music by Einaudi and others like him seem inexplicably static – much like the once mind-boggling repetitions of 1960s minimalism, but decades after the fact and with the philosophical daring excised.

Einaudi’s fans beg to differ, though. One tells me he has seen Einaudi live three times: “His music is so universal. You have to react. I get in touch with my joy and with my sadness.” Another says she used to play Einaudi’s music herself in school assemblies. “Some people would be really snobby. They’d be like, ‘Oh, that’s pretty basic.’ But I love it. I played it at my grandma’s funeral. It’s very moving music.”

And that’s the thing about this type of piano music: its creators are mostly uninterested in labels but they are very interested in “authenticity”. Biographical backstories are crucial. Beving started composing after a burnout. “I couldn’t really talk,” he tells me. “But the piano was giving me clarity and something to hold on to.” His compositional process is all about “allowing things to happen”. Improvisation is essential. And the characteristically barely-there softness of his music? “I couldn’t deal with the normal sound, so I had the study pedal [which dampens sound] on. There was a profoundness, a warmness and intimacy which allowed me to give more space between the notes.” It’s that sound, he reckons, that draws listeners in: “It’s almost like you’re on the piano stool, next to the guy playing.”

Ah, yes: it almost always is a guy playing this type of music – usually a white guy at that. There are female artists out there – Hania Rani, Sophie Hutchings, Poppy Ackroyd – but none match the listening figures of Frahm, Beving and their male colleagues. Frahm winces when I point this out. It may, he suggests, be connected to “the desirable image of the modern man, who should be very sensitive”. Speaking to me from her home in Australia, Sophie Hutchings is unsure why the skew remains so extreme, beyond still-entrenched expectations about the gender of “a composer”. But she admits: “Most female composers I’ve met in my world, which isn’t that many – they’re very determined. Because, yes, I would say it can be harder as a female.”

Like Beving and co, Hutchings writes through improvisation: “It’s a very subliminal process.” As we chat, she refers to the “therapy” that composing and listening entail. Also like Beving, Hutchings – a long-term insomniac – emphasises music’s capacity to heal and soothe. “We live in an overly saturated world of complexities,” she says. “And I think we are craving to strip back the layers, to feel things.” Which is where her own delicately circling solo pieces come in. Does she hope her music helps others to sleep, I wonder, or is that a rather backhanded compliment? “I take that absolutely as a compliment!” she says. “It’s not that it doesn’t tap into an emotion. It’s just not demanding your attention.”

And therein lies a clue as to why this music is so widely embraced by listeners and so little discussed by the establishment. What can you say about music that is intentionally so stripped back and evanescent? As Hutchings puts it serenely: “The music replaces the conversation. It’s this silent world of communication. It just hovers.”

• Nils Frahm is at the Jazz Cafe festival, London, 15 September. Ludovico Einaudi is at the London Palladium, 11 and 12 November