Webhooks: How long should we retain events ids to prevent duplicates?

[BarryCarlyon]

@zero

The thing is, I was hoping for twitch to specify a maximum time after which it never resends the same event instead of choosing blindly a number like e.g.: 24h

Twitch will retry with exponential backoff like 4/5? times before it gives up on a given message if it didn’t get a 2x. So in practice you should NEVER get a duplicate message ID. But you need to protect yourself in case a hacker does the replay attack (not Twitch,Twitch should never resend if it managed to deliver successfully the first time but it does practice “send at least once”, but the second one if sent may have a different ID), personally can’t say I’ve got many “legit” dupes from Twitch.

Generally speaking Twitch won’t ever resend the same event. But a hacker might.

And in the example of a stream.online, you might get a second stream.online with a different message ID. So you may want to deduplicate events recieved based on expected “next” event in an event series. (considering stream on/off)

@zero

Hmm, so my idea was maybe something like a redis cache for storing the IDs of those events (stream.online and stream.offline) and just set a TTL of 24h or something like that and also ignoring events with a timestamp longer than those 24h.

This would be my solution.

@zero

So you wouldn’t accept expired events.

You wouldn’t but it might happen!

@zero

The thing is, I was hoping for twitch to specify a maximum time after which it never resends the same event instead of choosing blindly a number like e.g.: 24h.

TLDR: In practice 24 hours makes sense. As you’d do a time compare on the event and ignore it even if it’s valid (again depends on the topic and use case. But twitch “things” that have “time caches” tend to reside on Twitch for about 5 day. (Thinking about hypetrain events and the like which also don’t have their lifetime noted in the docs and can spontaneousley reset their cache anyway)

Real TLDR: 24 hours makes sense.