“You’re part of an experiment in connection,” Nick Cave says from the stage. His In Conversation show, currently touring Australia, centres on questions and song requests but, like Cave’s new Red Hand Files website, “there will be no moderator”. So first he must briefly mire himself in rock’n’roll’s nemesis: a bit of housekeeping. The rules of play.

While Cave is dressed impeccably at the Sydney Opera House, his collaborators tonight are no cool-as-fuck Bad Seeds. They’re in sensible shoes and fluoro vests, and are cheerfully waving those lightsaber batons used to direct planes on to the tarmac. “If you have a question signal to them,” says Cave. “And they’ll … whatever.” He burns out on explicating the detail. Fair enough. He’s got bigger fish to fry.

The first fish is slapped down by a woman who shares his love of Leonard Cohen. “But can you explain Avalanche?” she says. “It’s always seemed so dark and incomprehensible to me.” He pauses. “I like it because it’s dark and incomprehensible. The big problem with most songs is that the meaning is too clear. I can play it if you like?”

“Would you?” she says, her voice gone high and swoony.

“I’ll do a version where I’m singing to you,” he says. Then, at the piano: “And now I’m going to fuck it up because I’ve talked too much about it.”

Thus we ease into the evening’s rhythm. The lights fade, Cave does a single song at the piano. The lights brighten and he gestures with weary elegance at a flashing baton to answer a question. He spins in slow circles because not only do tables press in cabaret-style around his piano, but the stalls behind the stage are filled too.

The layout gives him a dancing-bear captive air – which suits his efforts to place himself at our mercy. And at the mercy of an 11-year-old girl who asks “What’s the hardest thing about what you do?” He replies: “With live performance the terror of standing in front of people ... is incredibly addictive. But I grew to have a certain control over the outcome … these conversation events have brought back the terror. Because I don’t really know what I’m doing. And here I am answering the questions of a child.”

Sometimes he refuses requests – “I can’t do the build on Jubilee Street” or “too many chords” – but mostly, he says yes. Some he sings very well and others he inhabits. Most of the latter are newer songs – Higgs Boson Blues, Mermaids and Skeleton Tree – with the exception of the best rendition of The Mercy Seat I’ve heard (or seen on YouTube), and an incendiary belting-out of God Is in the House.

The Weeping Song suffers from the tyranny of his distance to it. It’s all plonky playing and affect. Another night you might not notice. Tonight, it jars due to how real he feels; up on his feet again to do the TEDx walk and talk.

Our interactions fall into three categories. First, when he quips back: many are gently teased by Cave tonight. In these moments he’s “as funny as a circus” (that’s my dad, who I took, not Nick). “Maybe you don’t need to clap after I answer each question?” Cave suggests. “I start to analyse the volume of each clap and, well … ”

The second is when he latches on to a topic and answers with depth and honesty. It’s like a late-night D&M with a mate – only the mate is Nick Cave and we’re yarning with the lights up amid a cast of thousands. “We’ll all live two lives,” he says in response to a comment about David Bowie’s final years – when he was facing mortality and reaching out to connect with fans. “In his first life, he did what many of us do and put himself forward as an individual … then later he looked for something communal and collective. It’s what happens to us all … I think we’re united by suffering.”

The “great thrill” of the creative process, meanwhile, is when something “starts to emerge from meaninglessness even when you don’t see it at first. To be a creative person requires most of all, faith. Faith the moment will come. And when it does you can see it vibrating in front of your face”.

The third category of interaction is tidbits about his music and collaborators. “It’s really about loss, the Bad Seeds,” he says. “It’s hard for people to look at them and not see the absences, now, of some towering figures.” He names some who’ve died and some who left the band: Blixa Bargeld, Conway Savage, Barry Adamson and Kid Congo. He talks about Brett Whiteley, Shane MacGowan, Tony Cohen.

Rowland S Howard’s influence on guitar music globally has been “vast”, he says. “He was the first real game-changer for me and I owe him a huge amount for that.” Their artistic collaboration was fierce; “we forgot how to be friends”. He adds: “That’s how the Birthday Party collapsed as a band. We forgot how to be friends.”

Cave says his work with Warren Ellis has deepened over the years. “Something special happens these days with Warren and I, where nothing can stop us if we’re in a room together.” Later, at a loss to name any new music he listens to (“I’m writing all day so I can’t”), he mentions their album. “We have something pretty cool brewing, me and old Wazza.”

In no particular category falls the young fool who endears himself to Cave by asking to have his bare foot signed. “Do you have a pen?” he yells when Cave agrees.

Ultimately, it’s true: he doesn’t know what he’s doing. He’s an amateur again! Only this time we get to be there. A professional would be more energy efficient. They’d know how to approximate questions, bridge to easier answers and recycle anecdotes. Instead of these pro moves, Cave instead chooses to not answer a few.

He struggles particularly with a question on male sexual predators. “Who? I still don’t know who do you mean?” The man who asked tosses out Louis CK as an example. “I can’t comment on Louis CK,” Cave says.

Just as the question is washing out to sea however, Cave relents and tugs it back. “Look, I’m in no way cavalier about this,” he says. “I’m not aware of it somehow having stained the music industry … where I don’t recognise this way men are being painted as monsters. It seems to be an industry that respects women – in some way. Is that right?” There’s a rumble of protest but no eruption.

He’s wrong on this point of course – and his thoughts on #MeToo are far better expressed here – but tonight’s not about being right. It’s about being present, knowable, vulnerable and real. And funny. Always that.

• Conversations With Nick Cave continues through January, in Queensland, Victoria, South Australia, Western Australia and New Zealand