The reason we include your own ID is as george said, we want to have a consistent model regardless of whether you specify from_id or to_id (or both!).
The max limit of 100 is due to the cost of fetching the following list. That said, it’s not impossible for that limit to be raised in the future.
Finally, I would hope that the following webhook would help you reduce scraping of your follower list and may address your concerns. (releasing Soon™!)