The IRC spec actually says nothing about needing to JOIN a channel before being able to PRIVMSG at it. JOINs are meant only to “start listening to a specific channel”: http://tools.ietf.org/html/rfc1459#section-4.2.1
If you think of IRC not as a set of individual chat rooms but as a generic publish-subscribe system, with PRIVMSGs being publishes and JOINs subscribes, this idea may not seem so reeling.