I just answered an interesting inquiry that came from our field on how to prevent users in vCenter Server from viewing vSphere Tags? The use case here is that the data contained in the vSphere Tags may not be something administrators want general users to be able to see, especially if they contain sensitive information, which hopefully folks are not using to store things like credentials or secrets.
If you navigate to the vSphere Roles, you will see a number of vSphere Tagging privileges, but there is nothing that covers the ability to remove read only access.
One very important thing to understand about the authorization of vSphere Tags is that it is NOT controlled by standard vSphere Permissions that you would assign in the vSphere Inventory but that it is controlled via vSphere Global Permissions, which are outside of the vSphere Inventory, which also includes vSphere Content Library and other vCenter Servers.
If you wish to disable the ability to view vSphere Tags for a VM while still maintaining basic read only view for VM, you need to ensure there is not a read only role assignment for your user under Global Permissions. You can check by navigating to vSphere UI under Administrator->Global Permissions. If the user that you are logging in with does not have a Read Only Global Permission, they will not see any of the vSphere Tagging information nor vSphere Content Library, which is another side affect.