In terms of accommodation, I'm not much use - I'm from Edinburgh and my trip back this summer is the first one I've done during the festival where I've actually had to find somewhere to stay.
Prices DO go up, but it's not extortionate. Try contacting the university - they rent out the halls of residence for about £25 a night upwards. Most hostels would charge this for a single room, and the uni is at least slightly more private. A lot of people also privately rent out their spare rooms during the festival, which can offer a bit more homeliness than a hostel etc. Otherwise, try the Scottish Youth Hostel Association - they have some great places near the west end of Princes Street that are very central.
What to see is almost an impossible question. I really think that to get the full festival experience, you do have to see at least one truly crap thing! See as much as you can, but don't forget to allow time to just soak up the general atmosphere - there is so much going on outwith the 'official' performances that you don't want to spend all day every day sat in theatres/venues.
If you like musicals, check out the programme for George Square. Their dedication wholly to musicals this year is one of the most exciting things to have happened at the festival for a long, long time IMO. I'll do a shameless plug for
Departure Lounge as it's fabulous!
I appreciate your point about not wanting to have to rely on public transport too much. Most of central Edinburgh is easy to get around on foot (though a bit hilly in places!) but the public transport system is also great. I've lived/spent a lot of time in several cities round the UK - Edinburgh, Newcastle, Sheffield, Nottingham, London, Bristol - and the buses in Edinburgh are better than anywhere else. If you're really on a budget, try looking to stay somewhere in the suburbs (Portobello, Musselburgh etc are all lovely, and right on the coast) - the buses are about 80p for a single trip or £2.20 for a day ticket (or at least they were last year when I was last up there - they might have gone up a little) and these places are only about a 25 minute bus ride into the city centre, so worth looking into.
Hope that helps!