50 birds in the same chimney are more likely non-breeding birds.
Swifts never go very far away from chimney when nesting. That can be detected in june and early july. If birds fly all day without disapearing from your sight, it might not be nesting. The easier way is to first find out where they spend the night. In the next days you can see if birds are frequently coming back to the chimney during the day. If so, they are nesting. If they stay up in the air all day, they feed for themselves only.
I'm pretty sure number of birds has gone up last year after 2006 big crash.