For me it's a subjective thing - when the aesthetics of the page begin to look too cluttered with banner ads, I say enough's enough.

At the moment we're only running four (static) banners on the mainpage (one of which is a link to another of our sites) - and we keep the banners strictly at the bottom - away from the site graphics and nav area.