Hundreds of objects should not be an issue. With the perftest, my phone only starts losing frames at 100,000 objects and my computer at 150,000. And that's just from max frames per second, it probably wouldn't even be noticeable until significantly more.
You should definitely isolate your events in scope though, there's no reason for any events to not filter out only the things they should be affecting.
On that note, even creating/destroying hundreds of objects at the same time normally wouldn't cause any noticeable lag, so there was probably something else acting on it in that case as well.