How to make the app public?

No, a new version is required to adjust the allowlists

Additionally if an overlay extension different policies apply.