Go see a show at the Fringe and there's a fairly good chance that you'll find yourself in a non-traditional performance space. There are a few proper theatres in Edinburgh, of course, where work takes place all year round, but the vast majority of companies will be performing in lecture theatres, Portakabins, tents, turrets, wine bars, car parks and upturned ruminant mammals. This system has its flaws – unconventional spaces come with challenges concerning access, ventilation, noise and storage – but it's all part of the fun of the festival that theatre turns up here where you least expect it.

But even in a context where any space that will fit performers and an audience can become a makeshift venue, there's still something special about deliberately site-specific work. Having seen none of this type of show during my stint up here at the beginning of the festival, earlier this week my schedule threw up three site-specific performances in three days and I was struck by how affecting I found each of these very different experiences.

First up was The Ethometric Museum, a sound installation performance in the cellars beneath the Hill Street Theatre, a masonic lodge the rest of the year. Via a demonstration of the ethometric instruments in the collection, strange machines from a mysterious and little known branch of science that emit harmonic frequencies, sound artist Ray Lee creates a sort of electronic symphony. It's an almost overwhelming experience for 'visitors' to the museum, one further enriched by the atmospheric surroundings of the low-ceilinged, vaulted cellars, supposedly the location where one of the instruments in the collection was found during recent excavations.

The second show on the list was The Simple Things in Life, which takes place in a series of sheds at the Royal Botanic Gardens. I say 'show', but it's actually five separate theatre pieces (audiences see three of the five at each of the several performances that run every day). You wander through the lovely (but chilly) gardens visiting the different sheds, enjoying two sets of site-specific scenario: the gardens themselves, which form a sort of promenade prologue to the individual pieces; and the theatrical worlds created within the sheds. The shows themselves offer a range of funny, moving and thought-provoking theatre experiences, while the rural calm that pervades the space and the fresh air you breathe as you walk from shed to shed are a remarkable antidote to the urban chaos and toxic lifestyle of normal Fringe business.

The final site-specific piece I attended was 3rd Ring Out: The Emergency, an interactive gaming-style show in a shipping container placed at one end of the Grassmarket. The premise here is that the 12 audience members at each performance are a disaster response team faced with a climate change crisis affecting the UK in the summer of 2033. The company didn't quite succeed in creating enough of a sense of urgency to force the individuals in my group to fully consider the consequences of a catastrophe of this kind, but the claustrophobic atmosphere of the show's setting was certainly diverting enough to make me forget I was in a box in the middle of a busy shopping street.

There's simply something extraordinary about coming out blinking into the daylight after a site-specific show. The best work in traditional spaces, of course, succeeds in taking its audiences to another place but it's rare that you'll actually be made to forget that you're in a theatre all together. Site-specific work can be so powerful because it does just that.