- Fetch all followers periodically (to remove unfollows) into an internal list
- Use the webhook to update new followers into your internal list
https://dev.twitch.tv/docs/api/webhooks-guide/
Avoid the issue to poll helix often all together
Would be the more beneficial route