If one of you is using TwitchApp/CurseApp I have observed that in some cases it doesn’t pop on the website, but does fire on TwitchApp (TwitchApp winning the “yay I received this” war and stopping website notify)
That said, via IRSSI, I only ever reply to and don’t initiate whispers. I don’t really send “unsolicited” whispers per sae.