In theory: Only the ones that you have permission to be in.
Like you, in theory, shouldn’t join and collect data from ninja’s chat without ninja’s permission.
So you should only join
That know your bot is joining.
There is no “hard limit” just what your connection/script/bot can keep up with.
See also the developer TOS on data collection Twitch.tv - Developer Agreement