First a confession. I have never quite 'got' the music of Bob Dylan – so I was worried that Conor McPherson's sort of musical, sort of play, with music and lyrics by the literary Nobel Prize winner would leave me cold.

As it turns out the songs – smartly chosen and sung with force and feeling by an incredible cast – are the best thing about a very peculiar show indeed.

Its origins explain some of its oddity. McPherson, a skilled spinner of tales who won such acclaim with The Weir, was approached `by Dylan's management and asked if he would be interested in basing a show around the songs. After some hesitation, the idea took hold and he came up with a setting in Dylan's hometown of Duluth, Minnesota.

So here we are, in winter 1934, at the height of the Great Depression, in a boarding house owned by the bank but run by Nick (a powerful Ciaran Hinds) and his wife Elizabeth (Shirley Henderson) who is in the grip of dementia. Among the characters around them are their wastrel writer son, an adopted black daughter Marianne, a widow who loves Nick, a couple burdened by a manchild who has never grown up and may have killed someone, an evil preacher and a good boxer. Oh, and there's a wise doctor, played with easy authority by Ron Cook, who comments on the action.

Their stories, such as they are, are revealed in the intervals between songs, performed to the audience and with microphones, accompanied by an excellent, pared-back on stage band using period instruments (arranged and supervised by Simon Hale). You can see where it's heading, but it simply doesn't work.

For one thing, the songs compelling though they are in themselves, disrupt rather than progress the action. There's one thrilling moment early on when Sheila Atim's magnificent Marianne bursts into "Tight Connection to My Heart (Has Anyone Seen My Love)" and words and emotion are perfectly matched; there's one towards the end when Henderson (in a performance of characteristic but distracting eccentricity) appeals to Hinds with "Forever Young", and makes it seem relevant.

But when a boy comes back from the dead and sings "Duquesne Whistle" in a carefully choreographed routine (courtesy of Lucy Hind), it feels as if the song has been shoehorned in. And the need to move from the stylised performances to the actual narrative means constant and distracting shifting of furniture and scenery on Rae Smith's cluttered set.

The other problem is that the sheer number of characters means that no story is ever fully developed; when people sing of their love you need to know why you should care; when a child dies it should make you cry rather than simply being an excuse for another number. McPherson, who also directs, fails to weave his magic; it is as if his wild and wonderful imagination has been constrained by the structure he has imposed on himself.

On the plus side, Mark Henderson's starkly shadowed lighting makes the entire thing look like a beautiful tableau and the singing really is terrific. But, looking at the wonderful cast assembled, which includes fine moments from Bronagh Gallagher, Stanley Townsend, Debbie Kurup and Arinze Kene (who gets to sing "The Hurricane"), it's hard not to feel that their great talent is mostly wasted in such a half-baked affair.

Girl From the North Country runs at the Old Vic until 7 October.

Buy tickets to Girl From the North Country