Disabling or deactivating vSphere Supervisor can take some time depending on the number of resources you may have deployed, which all need to be properly cleaned up.

In case the removal is taking longer than expected, there is not much information provided to the user in the vSphere UI.
However, I recently learned about a useful way to understand what is actually happening on the backend by looking at the vSphere Supervisor service logs, especially if you believe the operation is "stuck", which ended up being my situation.

