I agree with this. The ability to get all emotes was one of the cooler, though likely less used, endpoints of v5. I used it myself to see how many people had Pog Emotes at one point, and there is at least one site that uses it to catalog the emotes.
Given the Emote Sets endpoint, it might be a good idea to make an endpoint to get all emote set ids, (not including the emotes themselves). Potentially including some metadata about the set, such as emote count and last updated date so that the data can be cached.