Yes your code can handle that many users. I have a nodejs bot that stores 33000 records of about 1 timestamp in length so it should handle it fine.
I would go with the “store last timestamp of user” method and reward those of the “less than 5 minutes” which would mean you don’t need to fetch the chatters list as all. Since you are only monitoring activity and not presence.