The Twitch API guidelines states that you should avoid hitting the API (any end point) more than once per minute.
You basically cannot get “live” followers, you can only get data that is up to 59 seconds old.
Most follow notifiers appear to be “live” as they get the differences and then spread those differences over the next minute instead.
So in summary, there is nothing to fix, as you can’t get the data any quicker than you are already trying to