Right, so I also updated to the latest beta version since I hadn't done that already, then I changed the game instance name to be unique and tried again. This time I joined as host the first time in one tab. The second tab I entered as a peer, but again, no other peer present, I was the only one listed in there. Same in host tab, the peer didn't show up in the list.
After a few seconds, I got kicked from the peer tab. The host tab kept running.
I don't know what the unrelated connectivity issue there could be to be honest. My internet connection is working fine, maybe I accidentally set some setting somewhere.
EDIT: The messages shows I connected to the chat room both times, one time as host and one time as peer. But for some reason none of them shows up in eachothers lists and no messages are shown for the other user when typing them in and sending them.