It is because the IRC spec sort of says you can. It specifies that parameters to a command can start with a colon to allow spaces or not have a colon, but not allow spaces. Parameters are space separated. However there’s only 1 parameter allowed in PRIVMSG, so doing PRVIMSG #channel Test Message\r\n will result in you specifying two parameters. That isn’t a valid PRIVMSG anymore. If you use the colon on the parameter, further spaces get eaten as part of the parameter rather than becoming a delimiter.
You can do PRIVMSG #channel TestMessage\r\n and you’ve got one parameter following the no-space syntax.
You can do PRIVMSG #channel :Test Message\r\n and you’ve got one parameter of Test Message
Now, correct me if I’m wrong, but whispers aren’t formally documented. You’re lucky they work if you actually sent it without the colon.
Note: I’d always send it with a colon. The non-colon case is pretty rare, really, as everyone in normal IRC world just blindly adds colon everyone to be safe and to make it easier.