The movie District 9 is due to be released this coming weekend. Maybe I'll have something more to say about that after I've seen it, but for now I just want to talk about why we wouldn't necessarily know if Earth were hosting visitors from elsewhere. I'm not going to bother arguing that such visitations are actually happening, only that our lack of certainty that they are doesn't constitute evidence that they aren't.
Sure, if some incautious aliens were to park a large ship a couple thousand feet above one of our cities, we'd know about that. But a saucer-shaped craft, with curved upper and lower surfaces that met along a fine edge, would be naturally stealthy, even if it weren't coated with radar absorbing material. Park one of those at 80,000 feet or higher, and about all you'd have to worry about would be rockets headed for space, stratospheric balloons, whatever has replaced the SR-71, and the possibility of coming directly between a spy satellite and something on the ground it was imaging. So long as the craft wasn't particularly large or glowing brightly enough to stand out against the sky, the chances of being noticed from the ground would be pretty slim.
There's plenty of places to hide: deserts (under sand), glaciers and polar ice caps (under snow), rain forests (under the canopy), lakes and oceans (under water), rugged terrain (in valleys and canyons), and even cities (amid abandoned industrial facilities and warehouses). Given the abilities to hover without disturbing the surrounding air, detect approaching aircraft, and maneuver to avoid being hit by them, a craft might simply slip into a large cloud and hang out inside it. Unless it were to show up on radar, we'd have no way of knowing it was there.
Really, the only way to be certain that no such extraterrestrial visitations are happening is to assume inflexibly that they couldn't be, either because Earth is alone in having produced (intelligent) life or because the distances between stars are so great that travel between them is effectively impossible. If you admit that we might not be alone in the universe, and that even if faster-than-light travel is unachievable it might still be possible to make one-way journeys of a few light years at a time, spanning several generations, then you'll find it more difficult to simply discount the possibility that beings from elsewhere are flitting about.
Personally, I'm fond of the idea that there might be a benign, wildly diverse, pan-species galactic civilization, to which we have yet to be admitted because we still fight among ourselves, although my recalcitrant, pessimistic streak continues to insist that its more likely any galactic civilization would be an empire, sustained by threat of force, to which we are simply insignificant. Either way, with the exception of the occasional warship, any visitors would probably be here because they find us fascinating, or else because they're attempting to avoid the notice of the empire.