I did some research and it looks like this problem affects all the extensions, including big ones like Hearthstone Deck Tracker or new Borderlands 3 extension. It seems like the twitch extension documentation should more explicitly say about the need of unlistening events.
I also believe that the twitch pub/sub connection is shared between all the extensions, so even if one extension behaves correctly, the other one may drain number of topics, which will result in both extensions not working for the viewer.