Some more information:
As I said the response code is 429.
The response message is empty (length of 0)
There are no twitch specific headers (limit, remaining, etc).
I am hitting the limit even when no other calls are going out, so I am not really sure what has changed, unless I am sending them out too quickly and something on twitches end is wanting me to slow down (that is the next change I am going to implement).
Other ideas?