I restarted the build service again and it should be working again. (Every time I restart it, I run a test build to verify it's working, and it was working for me after the last restart.)
Apologies for the inconvenience everyone. Unfortunately it's extremely difficult to figure out why this happens. When it stops responding, I can log in to the build service and see that it has low CPU, free memory, free storage space etc. and everything looks normal, and there are no errors coming from the build service. I will continue investigating and try to narrow down what is going on. In the mean time for anything urgent remember you can do offline builds with the Cordova CLI.