WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

HBR (Host Based Replication) CLI for SRM 5

07.20.2011 by William Lam // 20 Comments

Host based replication (HBR) is a new feature in the upcoming SRM 5.0 which gives user the ability to replicate VM’s between dissimilar storage. Traditionally, SRM mainly relied on array-based replication to backup and recover virtual machines residing on set of LUN(s). This required all virtual machines to be backed up to be in a set of protected and common LUN(s). With HBR, you now have the ability to target specific VM and their respective VMDK(s) and backup to different storage type at the destination such as local storage, iSCSI/FC LUN or NFS datastores.

Another key difference is HBR does not leverage array replication technology but something analogous to CBT (Change Block Tracking) in which the initial backup is a full copy and all subsequent copies will be differentials. The frequency of this differential backup will be solely based on the configured RPO specified by the user.

Now that we have some background on what HBR and how it relates to the new Site Recovery Manager, let's talk about some of the "limited" automation options. As it stands today there is no publicly exposed SDK from VMware that can be consumed from the various toolkits such as vSphere SDK for Perl, PowerCLI, VI java, etc. To configure a VM to be backed using the new HBR functionality, you will still need to manually go through the vSphere Client wizard by simply right clicking on a VM and selecting "Site Recovery Manager HBR Replication" option.

Once you have the initial configuration set for a given virtual machine, there are some limited functionality that has been exposed through the vimsh interface using vim-cmd. A new "hbrsvc" has now been added which provides some limited options in making configuration and state changes for a given VM under HBR management.

~ # vim-cmd hbrsvc
Commands available under hbrsvc/:
vmreplica.abort vmreplica.pause
vmreplica.create vmreplica.queryReplicationState
vmreplica.disable vmreplica.reconfig
vmreplica.diskDisable vmreplica.resume
vmreplica.diskEnable vmreplica.startOfflineInstance
vmreplica.enable vmreplica.stopOfflineInstance
vmreplica.getConfig vmreplica.sync
vmreplica.getState

Note: This is probably not officially supported by VMware, please test this in a development or lab environment before using.

If you have used vim-cmd interface, then you should be pretty familiar with how the options work and since this is applicable for a virtual machine, you will need to know the virtual machine's VmId for all the commands.

To retrieve the HBR configuration for a particular VM, you will use the vmreplica.getConfig option:

Here you can see all the configurations that was made through the GUI such as the RPO, quiesce of guestOS and the VMDK(s) configured for replication. You also get some additional information such as the HBR server and the configured port and some important identifiers such as the "VM Replication ID" and "Replication ID". These two identifiers will be very important later on if you want to make use of the other commands.

To retrieve the state of a given VM, you will use the vmreplica.getState option:

This will provide you the current state of replication and progress if the replication is still going on. You will not only get the progress but also the amount transferred data to the destination site.

To retrieve the current replication state of a VM, you will use the vmreplica.queryReplicationState option:

This should be pretty straight forward command to only get details regarding the replication state and the progress both in percentage and amount of data transferred to the destination site.

To pause replication just like you can using the vSphere Client, you will use the vmreplica.pause option:

To resume replication just like you can using the vSphere Client, you will use the vmreplica.resume option:

To disable replication for a VM, you will use the vmreplica.disable option:

Note: Before attempting to disable replication for a VM, it is extremely important to make sure you take down the two important identifiers we had mentioned earlier: "VM Replication ID" and "Replication ID". The reason for this is when you re-enable replication, you will actually need to specify these ID's else your VM will be in a bad state and the only way to recover is using the vSphere Client to re-enable replication.

To re-enable replication for a VM that was disabled, you will use the vmreplica.enable option:

You will need to specify a few parameters such as the VmId, RPO, Destination HBR Server + Port, Enable Quiesce for guestOS, Enable Opportunistic Updates, VM Replication ID and Disk Replication ID which can all be found by running getConfig prior to disabling replication for a given VM

To manually force a replication sync, you will use the vmreplica.sync option:

You also have the ability to change some of the configurations for a VM for replication using the vmreplica.reconfig option:

Currently this is limited to only the RPO, Destination HBR Server + Port and enabling Quiesce guestOS and Opportunistic Updates. In the example above, you can see the RPO window has been updated to 10 minutes and we can confirm this from the vSphere Client. You will notice that the sync will happen ~10 minutes but the reflect RPO is not updated in the SRM interface, this may be a UI bug or the modification is not pushed up to the HBR servers.

Note: Per the vSphere Client and SRM/HBR documentation, the smallest RPO window is 15minutes but I have found that you can actually go smaller but again, use this at your own risk.

I was also interested to see if I could shrink the RPO window even further to say 1 minute and there was no errors and the ESXi tasks actually confirmed the change

Though after making the change and monitoring the next sync, I noticed it did not actually run every minute but anywhere from 6-11 minutes which seems to be the smallest RPO window.

You can also disable replication for a particular VMDK by using the vmreplica.diskDisable option:

To re-enable replication for a particular VMDK, you will use the vmreplica.diskEnable option:

As mentioned earlier, there are no official SDKs from VMware for SRM but the options provided from hbrsvc are from a hidden HBR API found on ESXi 5.0 host, you can see the new "ha-hbr-manager" using the vSphere MOB. Though you can not fully automate the configuration of HBR for a given VM, you do have the ability to automate the reconfiguration or state change for a given VM if you needed to.

Note: I have never placed with SRM prior to vSphere 5, but I also found WSDL files for what looks to be SRM API under the following URLs: http://[SRM-HOST]:8096/sdk/srm and http://[SRM-HOST]:8096/sdk/drService Once could create an SDK bindings using the WSDL files but I will leave that as task for the reader

There is also one additional HBR utility that can be found on the ESXi Shell of ESXi 5.0 which is the hbrfilterctl which provides some information about disks being replicated in HBR.

~ # hbrfilterctl
Ioctl to device is working.
Usage: hbrfilterctl [command args]

Commands:
ba : Print the active replication bitmap the the specified disk.
bt : Print the inactive replication bitmap the the specified disk.
pr : Print the disk length, bitmap length and extent for the secified disk.
ts []: Extract and transfer a light-weight delta for the specified disk.
li : Returns the File ID, Number of entries, copy index and size of the demand
si : Returns information about the full-sync process
de : Detaches a filter attach for offline replication
log for the specified disk.
fs : Force a full sync of the specified disk.
stats : Returns stats for all (but at most ) groups.

The first two options is pretty verbose as it prints the bitmaps of the specified disk, if you are interested, you can run those to get the output.

Here is an example of running the "pr" option:

Here is an example of running the "li" option:

Here is an example of running the "si" option:

The last option "stats" is probably the only real useful command for users at least which provides the status of replication and by specifying a number, it limits the output. Here is an example

Categories // Uncategorized Tags // ESXi 5.0, hbr, hbrsvc, srm5, vim-cmd, vSphere 5.0

How to Create Custom Roles on Standalone ESX(i) Host

06.22.2011 by William Lam // 9 Comments

There was an interesting question on the VMTN forums that caught my eye this week in which a user posted a question about controlling access to a limited set of virtual machines for users on a standalone ESX(i) host. For those of you who are familiar with vCenter Server know that this can easily be accomplished by using/creating roles which allows for granular access to various resources within a vSphere environment such as Datacenter, Resource Pool, Host, Virtual Machine, etc. This was the suggestion from several community members and it ended up being the simple solution as the user actually had a vCenter Server.

So what if the user did not have a vCenter Server and actually had standalone ESX(i) host, would he/she be out of luck? The answer is no, you can actually utilize roles on a standalone host but the default roles that are available are pretty limited in scope. By default, there are three roles: No access, Read-only and Administrator and these roles can be applied to both local and/or Active Directory users if your ESX(i) host is joined to an AD domain.

By leveraging our good friend vimsh/vim-cmd from either the classic ESX Service Console or Tech Support Mode of ESXi, we can actually create custom roles just like we would on vCenter Server with fine grain permissions to a virtual machine(s). You can not create custom roles using the vSphere Client like you would when connecting to a vCenter Server.

We will be using the commands found under vim-cmd vimsvc/auth:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth
Commands available under vimsvc/auth/:
entity_permission_add lockdown_mode_enter role_permissions
entity_permission_remove lockdown_mode_exit role_remove
entity_permissions permissions roles
lockdown_is_enabled privileges
lockdown_is_possible role_add

Before we get started, it is good to understand what is a permission and how that relates to a particular entity such as a virtual machine. Here are some definitions from VMware:

Privilege - The ability to perform a specific action or read a specific property
Role - A collection of privileges. Roles provide a way to aggregate all the individual privileges that are required to perform a higher-level task, such as administer a virtual machine
Object - An entity upon which actions are performed. VirtualCenter objects are datacenters, folders, resource pools, clusters, hosts, and virtual machines

Permission = User/Group + Role + Object 

Basically we will be creating a custom role which contains a specific set of privileges to administrator a virtual machine and assign this role to a set of users on a standalone ESX(i) host.

To view all available privileges, you can run the following command:

~ # vim-cmd vimsvc/auth/privileges | grep privId | sed 's/^[ \t]*//;s/[ \t]*$//;s/,//;s/"//g' | awk '{print $3}' | sort
Alarm.Acknowledge
Alarm.Create
Alarm.Delete
Alarm.DisableActions
Alarm.Edit
Alarm.SetStatus
Authorization.ModifyPermissions
Authorization.ModifyRoles
Authorization.ReassignRolePermissions
DVPortgroup.Create
DVPortgroup.Delete
DVPortgroup.Modify
DVPortgroup.PolicyOp
DVPortgroup.ScopeOp
DVSwitch.Create
DVSwitch.Delete
DVSwitch.HostOp
.....

To view existing roles and what privileges are associated with a given role, you can run the following command:

vim-cmd vimsvc/auth/roles | less

From the above privilege list, you will need to select the privileges that you would like in your new role, for now you can just copy the entries into a temporarily file (/tmp/vm-privileges) as that will be used later.

In this example, I've selected the basic VM privileges that is associated with the "Virtual Machine User" found in vCenter Server but adding on additional privileges to support snapshots.

[root@vesx41-1 ~]# cat /tmp/vm-privileges
Global.CancelTask
VirtualMachine.Interact.AnswerQuestion
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.SetCDMedia
VirtualMachine.Interact.SetFloppyMedia
VirtualMachine.Interact.ConsoleInteract
VirtualMachine.Interact.DeviceConnection
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.PowerOn
VirtualMachine.Interact.Reset
VirtualMachine.Interact.Suspend
VirtualMachine.Interact.ToolsInstall
VirtualMachine.State.CreateSnapshot
VirtualMachine.State.RemoveSnapshot
VirtualMachine.State.RevertToSnapshot

Next we will need to know the virtual machine's VmId also known as the MoRefID (Managed Object ID) that uniquely identifies a given entity. To display the VmId of all virtual machines residing on your ESX(i) host, you can run the following command:
[root@vesx41-1 ~]# vim-cmd vmsvc/getallvms
Vmid Name File Guest OS Version Annotation
32 SteveJablonsky [datastore1] SteveJablonsky/SteveJablonsky.vmx freebsdGuest vmx-07
48 HansZimmer [datastore1] HansZimmer/HansZimmer.vmx dosGuest vmx-07
64 JamesNewton [datastore1] JamesNewton/JamesNewton.vmx dosGuest vmx-07
80 BrianTyler [datastore1] BrianTyler/BrianTyler.vmx dosGuest vmx-07

Next we need to identify the users in which we will create the new permissions for. In this example, I will use both a local (william) and Active Directory user (PRIMP-IND\primp):

[root@vesx41-1 ~]# id william
uid=501(william) gid=100(users) groups=100(users)

[root@vesx41-1 ~]# id PRIMP-IND\\primp
uid=37225554(PRIMP-IND\primp) gid=37224960(PRIMP-IND\domain^admins) groups=37224960(PRIMP-IND\domain^admins),37225020(PRIMP-IND\denied^rodc^password^replication^group)

In this example, I have a single ESX host that contains 4 virtual machines and the first two will only be visible to a user called "william" and the second two will only be visible to a user called "PRIMP-IND\primp":

  •  Owner: william
    • HansZimmer
    • BrianTyler
  •  Owner: PRIMP-IND\primp
    • SteveJablonsky
    • JamesNewton

We will create a virtual machine administrator role called "VMAdmin" which will then be associated with their respective virtual machines and system owners.

First we need to create the custom "VMAdmin" role by using the file that contains all the necessary privileges, we will run the following command:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/role_add VMAdmin $(cat /tmp/vm-privileges | awk '$1=$1' ORS=' ')

Note: The "role_add" operation takes in a roleName and then a list of privileges which are separated out by a space, this list can be as many as you need and by specifying it in a file, it makes it easier to edit/adjust

You can verify that the new role was created running the following command:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/roles
......
(vim.AuthorizationManager.Role) {
dynamicType = ,
roleId = 12,
system = false,
name = "VMAdmin",
info = (vim.Description) {
dynamicType = ,
label = "VMAdmin",
summary = "VMAdmin",
},
privilege = (string) [
"Global.CancelTask",
"System.Anonymous",
"System.Read",
"System.View",
"VirtualMachine.Interact.AnswerQuestion",
"VirtualMachine.Interact.ConsoleInteract",
"VirtualMachine.Interact.DeviceConnection",
"VirtualMachine.Interact.PowerOff",
"VirtualMachine.Interact.PowerOn",
"VirtualMachine.Interact.Reset",
"VirtualMachine.Interact.SetCDMedia",
"VirtualMachine.Interact.SetFloppyMedia",
"VirtualMachine.Interact.Suspend",
"VirtualMachine.Interact.ToolsInstall",
"VirtualMachine.State.CreateSnapshot",
"VirtualMachine.State.RemoveSnapshot",
"VirtualMachine.State.RevertToSnapshot"
],
}
]

Note: If you created a role that did not contain all the privileges and you need to update the role, you actually have to delete the role first. I have not found a way to "append" privileges, but removing a role is very straight forward and you can use "vim-cmd vimsvc/auth/role_remove" to do so.

Next we will associate this new role with the users and the virtual machines to create the permission mappings. If you recall earlier, we retrieved a list of virtual machines and the first column contains the VM's VmId which will be needed in this next section. We will be using the "vim-cmd vimsvc/auth/entity_permission_add" operation and if you would like to know what arguments it accept, you can just run it by itself without any arguments.To associate the first two virtual machines to the user "william" we will be running the following commands:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:80 william false VMAdmin true
[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:48 william false VMAdmin true

To associate the new two virtual machines with the user "PRIMP-IND\primp" we will be running the following commands:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:64 PRIMP-IND\\primp false VMAdmin true
[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:32 PRIMP-IND\\primp false VMAdmin true

Note: The VM entity syntax is in the form of vim.VirtualMachine:XX where XX is the virtual machine's VmId extracted earlier which will need to be substituted in with your own VmId

We can verify the permissions by running a few commands, we can either check all permissions created with this particular role name, in our example, it is "VMAdmin" and the command to run would be the following:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/role_permissions VMAdmin
(vim.AuthorizationManager.Permission) [
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:32',
principal = "PRIMP-IND\primp",
group = false,
roleId = 12,
propagate = true,
},
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:48',
principal = "william",
group = false,
roleId = 12,
propagate = true,
},
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:64',
principal = "PRIMP-IND\primp",
group = false,
roleId = 12,
propagate = true,
},
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:80',
principal = "william",
group = false,
roleId = 12,
propagate = true,
}
]

You can also check a specific entity such as a virtual machine by running the following command:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permissions vim.VirtualMachine:64
(vim.AuthorizationManager.Permission) [
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:64',
principal = "PRIMP-IND\primp",
group = false,
roleId = 12,
propagate = true,
}
]

Now that we have confirmed the new permissions, we will want to verify that these users can only see the virtual machines we have specified both in the vSphere Client and webAccess (ESX only).

Here is a screenshot of user "william" connecting to the vSphere Client:

Here is a screenshot of user "PRIMP-IND\primp"connecting to the ESX webAccess:

Contrary to the default permissions on a stand alone host, you actually can reate custom roles just like on a vCenter Server to have fine grain access controls for your various users without having to give them full administrative privledges to an ESX(i) host. If you have vCenter Server, then of course that is the recommended approach but if you do not, you still have a way 🙂

Categories // Uncategorized Tags // ESX 4.0, ESXi 4.1, privilege, role, vim-cmd, vimsh

  • « Previous Page
  • 1
  • 2

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025