If you look at the photo links over in my thread, you'll probably notice a gradual build-up of poop, especially in the first batch of squabs, when we had the small flowerpot, before we switched to the larger flower box. Early on, we bought into the myth that the parents would reject the squabs if there was human scent around. We know now that's not true, but we didn't know it then. But even after we realized it, we left the nest alone. While we cleaned the balcony at least once a week, we simply switched out the flower box after every couple of batches. But the poop seemed to get mulched into the nest and dirt of the flower box over time. (Putting down a layer of dirt might be a good idea for you.)
We did not have a problem with smell, though. The breezes here carry that away. If you're worried about being outed, I don't think it could hurt to do a little housekeeping in the nest.