Appreciate the details here!
Agree on many points here, but wanted to provide a broader image as well to explain why we believe mocks are the right thing (for now).
Oh yes. We are still 8 followers away from affiliate level, which prevents us from fully testing our Golang PubSub library. Unfortunately the PubSub docs appear to be outdated, which doesn’t help either.
Dkamps18created a ticket for it.
Sidenote- if you have specific feedback, feel free to message me on Discord as well; Same username in the Discord server. 
Our bet is that most Twitch developers use readily available CI tools like Github Actions. Github Actions allow egress connections. We’d love to find out, but even game studios like Blizzard will most likely not block egress traffic in their internal CI tooling. We could see a world where Coinbase or other financial or healthcare institutions firewall away their CI suite, but are they really a user-story for this mock?
The point is, from our perspective “not having a (slow) internet connection” doesn’t appear to be a good product requirement.
I can agree with this point to a degree, and it doesn’t really immediately solve the problem, however Twitch is not just smaller developers; we have working relationships with many large multinationals that integrate with our APIs (think like Riot) and these customers are more where avoiding an Internet connection is a plus.
It’s not the entire reason for putting it in the CLI, however- beyond the portability of the data that’s been noted by Barry and Maren below, there’s also the aspect of community engagement that’s not currently available on the docs or otherwise. Meaning if there is drift, for whatever reason, it can be fixed very quickly by myself or others within the community.
There are other reaosns that I’m not sure I can really dive into about this as well, but regardless- it’s an approach that allows us to test the idea quickly. And who knows? Maybe it does get absorbed into an official Sandbox down the road.
If the mock returns responses inconsistent from the production API, developers will lose trust in the mock and instantaneously resort back to the production API for testing. That’s what we would do.
Agree! This is the piece we’re working on figuring out the right process for since this sort of client-side mock server is very atypical. It’s a priority for us to address and address in the right way since it can erode trust- but there’s also the side that means we could have the power to test new functionality prior to going live as well, in addition to having blessed schemas for the API results due to Golang’s strong typing.
To summarize: We’d love to see official client libraries instead of a mock. A sandbox environment would be helpful for integration testing. Good docs with tutorials and guides would be great.
We hope our feedback was helpful and we very much appreciate all of the hard work that goes into building Twitch and its community.
I appreciate the candid feedback here; it’s definitely much appreciated and something we’ll read into. Needless to say, most of this is not explicitly new, but regardless refreshing to see the thoroughness you’ve put into it.