If there are multiple users then that may mean it’s something on Twitch’s end, but the vast number of users without issue still doesn’t rule out it being a transit issue, or an issue on your side with things you’re running, somewhere that’s causing a connection drop.
It’s hard to diagnose without seeing network traffic.