This is essentially what is going on. As far as I’ve come to understand, Twitch now heavily caches the followers endpoint now due to the excessive use by alerters. Even so, my alerter polls once per 15 seconds. It’s possible that the longer caching is what is causing your alerts to show up slower.
I run a websocket TMI relay service that allows me to listen for new subscribers in chat. I don’t poll any endpoint for subscribers.