Thanks for your quick responses. I am now able to identify the end by checking for the existence of the ‘cursor’ field as well as its contents as noted by @BarryCarlyon. As for hitting the rate limit, it’s still a small problem but since this code is running from a 5 minute cron task, I can afford to spend a few mins waiting for my request count to reset. 30k results / 100 results per request / 120 requests per minute = 2.5 (round up to 3) minutes to search all requests.
My pseudocode now looks something like this:
cursor = “”;
do {
checkWaitOnRateLimit();
getNextBlock();
parseBlock();
if (newCursorExists())
cursor = newCursor;
else
break;
}
while (true);