In the case of private data, such as channel subscriptions topic, you should also revoke the User Access Token if you no longer have access to the callback, and there is any potential for that url to be reused by other users.
For public topics you can just let them expire. Ideally though you shouldn’t use webhooks with callbacks that can be lost/changed during the lifetime of a subscription.