You do need to be connect to one of Twitch’s GroupChat servers to send/received whispers but you do not need to join a room/channel, you can simply send a PRIVMSG to jtv, using /w as the command:
PRIVMSG jtv :/w recipient message
recipient: the user to send the whisper to
message: the message to send
And whispers recieved come in formatted as:
@msgtags user!user@user.tmi.twitch.tv WHISPER self :message
@msgtags: only sent if you request IRCv3 message tags viaCAP REQ :twitch.tv/tags
user!user@user.tmi.twitch.tv: The user that sent the whisper
self: the recipiant(you, your bot, etc)
message: the whispered message
(@BarryCarlyon: I didn’t mean to reply to you; meant to reply to the original post, I apologize.)
Edits: More info, spelling, formatting