Most of us have an internal soundtrack - we may not always notice it, but it is there. It could be a song we heard on the radio that haunts us for the rest of the day, or it may be something that swells up unaccountably from memory, some favourite tune brought to the surface by a waft of perfume, or familiar place. Consciously or unconsciously, we assign motion picture scores to our lives – a first kiss, a break-up, a re-union – all can be brought back by the right melody.

This is something that Maxis, the developer behind the hugely successful Sims series, has always appreciated. Ever since the original title arrived in 2000, revolutionalising the whole concept of the life simulation, music has been an important part of the experience. On the screen, virtual humans go about their lives, falling in and out of love with each other, developing their careers, having families. And for players, all of that fascinating stuff, as well as building new houses and designing fresh sims character, is accompanied by music. Sims are even able to switch on the radio, often hearing amusing versions of real-life pop songs, rerecorded in "simlish", the game's gibberish language.

Several well-known composers have worked on the game's background scores, providing short stabs of music to accompany key onscreen events. That process is happening again with Sims 4, the latest addition to the series, due out this autumn. Except this time, changes in the gameplay have encouraged the studio to put even more emphasis on the impact of the soundtrack.

All about the feels

The big theme of Sims 4 is emotion. Using a new AI technology the company is calling "smart sim", your onscreen characters will react to the world and its inhabitants in new ways, becoming angry, falling in love, or breaking down in tears, in response to what happens. "Emotions are really a launching point into having really believable, very intelligent sims," says producer Graham Nardone. "You look at past sims games and we bring that out in artificial ways, but this is our opportunity to really capitalise on how lifelike they can be and let you tell new stories based on that. Emotions are now a really impactful part of gameplay that unlock new situations. Whether you're interacting and socialising with other sims or playing with the various objects that are out there - it all lets you do new cool things."

So to enhance this evolved area of the game, the team brought in film soundtrack composer Ilan Eshkeri, who has previously worked on movies like Young Victoria, Layer Cake and KickAss. He has composed the background music for the house and sim creation screens, but also wrote over 140 short musical stings which accompany key emotional moments in the gameplay – a fight, a kiss, a major realisation. All require a miniature musical moment.

"I grew up in that first generation of home computer users, " explains Eshkeri on his decision to take the project. "I had a Vic 20 and a C64 and I've played computer games throughout my life. But being the person that's consuming the entertainment is very different form being the person who's creating it, in any situation. I don't think any amount of game playing would have really prepared me for what is essentially a whole new concept."

"In film, there's a fixed narrative – and actually most computer games have some sort of a narrative within them. But The Sims doesn't have any fixed narrative whatsoever – mentally, that was the first hurdle to get over. I've got to write all this music, but it doesn't have any kind of framework - it's purely there to set an emotion, a tone; to give the player a feeling in a moment. Conceptually, that was quite different – and difficult. It was a big gear shift."

Collaboration and emotion

So he began working closely with Maxis, conversing with the studio's audio director, Robi Kauker, an experienced musician in his own right. And eventully Eshkeri developed a sort of systematic approach. "I really wanted to try and unify the style," he says. "I didn't want the stings to be these random pieces of music, I wanted them to relate back to the music you get when you build a character or home. I invented my own set of rules.

"So if something emotional happens, like doing a set of stings around getting married or having a baby, I'd try to relate all of those to a few notes or a riff or a chord sequence that appeared in one of the longer pieces of background music. For example, if character is doing something in the house or if something breaks in the house, I'd try to relate that to the music you heard when you were building the house. It doesn't always work - if you're getting married, for example, you want to hear the Wedding March or some version of that, so some bits of the stings have this very obvious musical impetus."

We're talking at the famed Abbey Road recording studios, where Eshkeri has been working with the London Metropolitan Orchestra and Maxis for several days. I sit in the control room for one session, as the composer leafs through a vast wad of papers, and calmly speaks directions to the assembled musicians on the other side of a glass divide. Around him sound engineers tap into computers and twiddle nobs on a vast mixing board. The piece they're recording is about six seconds long and has the working title "first kiss... shy". It's pretty clear what it will accompany in the finished game.

It is a lovely wilting piece of romantic music, but through the first few plays, Eshkeri thinks it sounds too strident. He experiments with removing certain instruments from the sound and tries again; then adds a momentary pause, perhaps a few beats, just before the final notes. Suddenly, there is an element of uncertaintly and self-consciousness to it. "I think that's it… next one," he says to the recording engineer. And outside in the vast studio, the musicians turn the pages of their score accordingly.

It feels like a pretty natural process for Eshkeri and the players. "I know the orchestra very well, I know the individual players very well – I have them in my head while I'm writing," he says. "I have a very clear sense that if I do this or write this, this is how it's going to come out in this room.

"But with some of the stings, they start playing them and I'm like, no, that could be more shy, how could I do that - I'm thinking of stuff in the moment. And if we need to be more shy, more angry or whatever, the musicians really engage with that as well - they're not just playing the notes. The reason why I love the London Metropolitan Orchestra is that they do really engage on a conceptual and emotional level with not just the notes on the page but what the purpose of us recording this music is."

Music and the message

Also present in the recording studio is Rachel Franklin, exececutive producer on The Sims. She sees a direct correlation between this emerging score and the addition of emotional AI and gameplay into the series. "Ilan is known for these theatrical sweeping, wonderful compositions," she says. "Its a way for the Sim to respond back to the player, explaining, 'hey, I'm feeling this'… You can really feel that in the audio.

"Combining that with animation technology and facial emotional overlays – all of these things work together in a really cool way to make you feel more related to your sim. Because ultimately you're caring for them – whether you torture them or not – you're still responsible for them.

"The music brings your relationship really to a height."

Music to change games by

Sims 4 has a lot of work to do to expand the series beyond its core base of invested fans. Too many add-ons and expansions have confused casual observers and some feel the concept has strayed too far away from creator Will Wright's original vision of a sandbox life simulation. But the AI and emergent emotion stuff is interesting, especially in how it promises to guide sim activities – if they're depressed for example, they may paint an amazing picture; if they're angry, they may put all that tension into an amazing work-out that boosts their fitness stats.

Apparently, sims will also be more naturally sociable. "Just having a group conversation, being able to interact realistically and allow new sims into a conversation, to acknowledge them and respond... it sounds like a small thing but it's never happened in the sims game," enthuses Nardone. "The sims were kind of robotic, so bringing in an AI where we can be very reactive in group situations, it changes fundamental aspects of the game - that's the coolest thing in there."

It'll be interesting to see if Maxis can work in the nostalgic and evocative elements of music. If a sim switches on the radio during a romantic encounter, will that song always trigger a certain response? It should do. Eshkeri and his orchestra have seemingly brought a sweeping sense of romance to the audio, now the sims have to run with it.

• Sims 4 is due out on PC and Mac in autumn or winter 2014.

• Sims 4 announced as single-player offline title

• Q&A: Will Wright, creator of the Sims