"We've got magic to do" state the cast at the outset of this enchanting, troubling musical in its latest reimagining at Southwark and, wow, they're not kidding. Seldom has Stephen Schwartz and Roger O Hirson's 1970s hit been mounted as coherently or excitingly as it is here in Jonathan O'Boyle's exhilarating staging, which feels – in terms of imagination and ambition – a natural progression from the same director'sWhatsOnStage Award-winning take on Hair, which also originated at Manchester's enterprising Hope Mill Theatre. The spirit of Bob Fosse, who staged the iconic original version, hangs heavy over this version. He's there in the splayed hands, dropped shoulders and angular limbs of the dancers and in the sardonic, dark tone of the show as a whole.

Schwartz is the composer and lyricist of Wicked but, at the risk of incurring the wrath of countless stagey teenagers, I would opine that Pippin is a much better score. From the bombast and sparkle of the thrilling opening number "Magic To Do", which introduces the troupe of players bringing to life the mythical story of Pippin, the questioning son of medieval emperor Charlemagne, through gorgeous, lyrical ballads and flamboyant production numbers to a bittersweet, unsettling finale, this is a gem. The lyrics match the glorious quicksilver of the music with ingenuity, wit and, at times, surprising depth.

Hirson's book is more problematic – who are these players that alternately cajole, torment or seduce the emotionally and spiritually lost hero? Does Pippin really assassinate his warmongering father or is that another grand illusion? – but it has heart and a sort of raucous affability that potently throws the darker elements of the piece into vivid relief. The show is open to any number of treatments - the original production referenced commedia dell'arte and the Menier version treated it as a giant computer game with all the characters reduced to avatars. Here we get a band of players in semi-Pierrot get-ups, like a mash-up of end-of-the-pier and Cabaret (the movie). It works!

More than any other production I've seen O'Boyle seems to have interpreted Pippin as a study of depression, with the Leading Player as a not-quite-human manifestation of the proverbial Black Dog, egging the lost hero on to stage his own death as the finale to their show. The juxtaposition of showbiz razzle-dazzle with existential despair transforms difficult themes into powerful, compulsive entertainment that moves and stirs but, amazingly, seldom trivialises.

The New York revival was the first to cast a woman as the Leading Player and O'Boyle follows suit here: Genevieve Nicole is absolute dynamite. A commanding, statuesque presence with the precision and grace of a true dancer and Bassey-like belt, she starts out playful and sassy, but by the end of the show has become downright terrifying. It's a star-making performance. Opposite her, Jonathan Carlton is a lovely, open-faced Pippin who grows in stature as the story deepens and darkens.

Other stand-outs include Bradley Judge as his smarmy half-brother, Rhidian Marc's likeable Welsh-accented King, and Tessa Kadler, delightful as the young widow who throws Pippin an emotional lifeline, but really there is no weak link in this tightly drilled, hugely talented company. There's another stunning performance from Mairi Barclay doubling as Pippin's scheming, glamorous stepmother and his life-affirming grandmother. The decision to play the latter as a comic grotesque does, unfortunately, rob her joyous big number "No Time At All" of its customary lump-in-the-throat poignancy but it feels appropriate within the context of this being a troupe of players.

Overall, this is a gorgeous piece of theatre: endlessly inventive, the magic tricks are up close and real, William Whelton's Fosse-style choreography is often breathtaking, Maeve Black's garishly beautiful design is lit with great panache by Aaron J Dootson, and the band under Zach Flis sound terrific, if at times a little overwhelming. O'Boyle's triumph, though, is that despite all the folderol, it is the human vulnerability and pain at the musical's core that you ultimately come away with. That's quite a feat in a show this flashy. Don't miss it. Sensational.

Pippin runs at Southwark Playhouse until 24 March.