It has nothing to do with the signalling server if your host and peers were connected and communicating at any point in time. My understanding is that once a peer successfully joins a room, the signalling server should not have anything to do with the peer anymore (it wouldn't know the connection status of any peer either).
Are all testing hosts/peers on a local network? Sounds like a NAT problem. WebRTC is generally pretty good about maneuvering through NATs, but things can go bonkers through double NATs or other more complicated scenarios.
Another possibility is the host connection is just too unreliable to keep the connections alive - this has nothing to do with speed or bandwidth, but more to do with latency jitter and packet loss.
Outside of that I would suspect an event sheet mistake.