Musical theatre tends to divide public opinion as effectively as the controversial Park Hill estate in Sheffield. Both have admirers who argue for their underrated artistry and equally fervent critics who regard them as an affront to aesthetic sensibility; both have undergone significant reinventions in recent years. So the news that the Sheffield musician Richard Hawley is to create a new musical chronicling 50 years in the life of the estate has been greeted with eager anticipation in some quarters, and raised eyebrows in others.

Is the rise and fall and gentrification of a council estate the right material for the heightened and stylised world of a musical? Musical theatre is a strange beast; many people feel passionately averse to it without having seen very much. “I hate musicals,” a friend declared when I suggested going to one at the Edinburgh festival. “Which ones?” I asked. “All of them,” she said, with feeling.

For her, as for many musical-intolerant people, the very idea of characters expressing themselves in song, dance and rhyme is the enemy of plausible storytelling. It’s a particular idea of what musicals look like, epitomised by Damien Chazelle’s multi-Oscar-winning 2016 homage La La Land: effete, candy-coloured, heartstring-tugging confections far removed from realism.

But this is to miss the quiet revolution taking place in musical theatre in recent years, spearheaded by the extraordinary success of Lin-Manuel Miranda’s Hamilton. By blending hip-hop and show tunes, while examining who gets to write history and whose stories are told, Miranda revitalised and opened up the genre to new audiences in terms of age and background, making musical theatre part of the cultural conversation in ways it hadn’t been for some time. Successful productions on subjects that might once have been supposed “inappropriate” for a medium as light as musical theatre have followed.

Alison Bechdel’s Fun Home was originally a graphic novel that told the author’s story of growing up gay in a small-town funeral home with a closeted gay father who committed suicide – not obviously the stuff of song and dance but it was adapted into a Broadway hit.

The premise of a feelgood 9/11 musical sounds like “a self-spoofing concept”, as the New York Times wrote of Come From Away, but the Canadian production set in a small Newfoundland town that welcomed 7,000 passengers from stranded planes after the attacks in New York has been hugely popular and transferred to Broadway last year.

Closer to home, musical theatre has grappled with grittier political material. The National Theatre’s 2011 production London Road was a musical by Alecky Blythe and Adam Cork about the Ipswich community shaken by the serial murders of sex workers in 2006, later adapted for film by Rufus Norris. Songs were composed around verbatim transcripts of interviews with neighbours of the killer, offering the kind of voices whose stories are rarely told on stage, especially set to music. “I wanted to see if I could make a musical that doesn’t make me cringe the moment they break into song,” Blythe said, echoing the common dread of the form.

Since then we have had the “musical tragicomedy” Removal Men at London’s Yard theatre in 2016, set in a detention centre and inspired by the experience of its creator, MJ Harding, of being involved in direct action to prevent the deportation of a woman from Yarl’s Wood. In the same year, the announcement that Daniel York Loh had won the Perfect Pitch award, granting him £12,000 to develop a musical about the drowning of 23 Chinese migrant workers in Morecambe Bay, provoked controversy. The idea was condemned as insensitive and “an insult to the memories of the victims” by locals – although Loh’s intention was to tell the stories of an already marginalised group – and an online petition was set up to have the production banned.

Which all leads to the question: who are these musicals for? Norris, Blythe and Cork kept the real residents of London Road involved throughout the development process (some appear as extras in the film version), though they brought in security for the press previews in anticipation of protests, and the show has not had a run in Ipswich. No doubt Hawley’s proposed musical Standing at the Sky’s Edge will also run into objections from Sheffield residents, particularly given that Park Hill generates so much strong feeling in the city.

Even now the estate stands as a visual symbol of division, one half expensively redeveloped and no longer principally social housing, the other a brutalist ruin. It will help that the former Pulp guitarist has his roots in the city and that the Sheffield Crucible nurtures close connections with the local community, but it will be a testing exercise in diplomacy, ensuring that those whose stories are being told feel they have been fairly represented and not turned into entertainment for those who can afford the ticket price.

Musical theatre is often seen as elitist because of the prices in the West End and Broadway, where top seats can go for three-figures. It’s one thing to expand the range of voices heard on stage, but theatres need to ensure that those being represented are also able to access the experience, otherwise there’s a danger that these stories become voyeurism for the middle class. Musicals are ruinously expensive to stage and the larger venues often can’t afford to take risks on untried work, so it’s all the more important that development support is given to the smaller arts festivals and fringe venues experimenting with new and exciting ideas about what the genre can do, and who it speaks to. Now that is worth making a song and dance about.

• Stephanie Merritt is a novelist. Her latest book is While You Sleep