It is still amazing to see that the number of contributions and suggestions from the community continues to grow for my free and simple VM backup solution called ghettoVCB. I created ghettoVCB almost 8 years ago which now has over 1.2 million views, pretty insane if you ask me! Although I am quite busy these days which includes a new born, I still try to find time to update the script as time permits. A couple of weeks back I received an email from one of my readers who came across ghettoVCB and was quite happy with the free solution. He also had some feedback asking why I did not provide an installable VIB for ghettoVCB?
A totally valid question and the answer was quite simple. When I had first created ghettoVCB back in the classic ESX 3.x days, the concept of a VIB had not existed yet. With the release of ESXi 5.0, the idea of the VIB was introduced but it was only recently in 2012 did VMware publish a method for customers to create custom VIBs for ESXi using the VIB Author Fling. I do have to admit at one point I did think about providing a VIB for ghettoVCB, but I guess I never went through with it for whatever reason. Looking back now, this was a no-brainer to provide a simplified user experience and not to mention the benefit of having ghettoVCB installed as a VIB is that it will automatically persist on ESXi after reboots which was a challenge for new users to ESXI.
So without further ado, here is ghettoVCB provided in either a VIB or offline bundle form:
To install the ghettoVCB VIB, you just need to download the VIB and run the following ESXCLI command and specifying the full path to the VIB:
esxcli software vib install -v /vghetto-ghettoVCB.vib -f
Once installed, you will find all ghettoVCB configuration files located in:
/etc/ghettovcb/ghettoVCB.conf
/etc/ghettovcb/ghettoVCB-restore_vm_restore_configuration_template
/etc/ghettovcb/ghettoVCB-vm_backup_configuration_template
Both ghettoVCB and ghettoVCB-restore scripts are located in:
/opt/ghettovcb/bin/ghettoVCB.sh
/opt/ghettovcb/bin/ghettoVCB-restore.sh
One additional thing I would like to point out is that you can also quickly tell which version of ghettoVCB is running by inspecting the installed VIB by using the following ESXCLI command:
esxcli software vib get -n ghettoVCB
If you look at the screenshot above, I have highlighted two important pieces of information in green. The first is the "Description" property which includes the Github commit hash of the particular revision of ghettoVCB and the "Creation Date" property which contains the date of that commit. This can be handy if you want to compare it to the latest ghettoVCB repository found on Github here. Thanks again Markus for the suggestion!
For those of you who are interested in the details for creating your own ghettoVCB VIB, the next section is specifically for you. Earlier this week I blogged about a Docker Container that I have created to help build custom ESXi VIBs and as you can see now, that was the basis for us to be able to quickly create ghettoVCB VIB based on the latest revision of the script.
Step 1 - Create a new Docker Machine following the steps outlined here.
Step 2 - Login to the Docker Machine and create a new Dockerfile which contains the following:
FROM lamw/vibauthor # Due to https://stackoverflow.com/a/49026601 RUN rpm --rebuilddb RUN yum clean all RUN yum update -y nss curl libcurl;yum clean all # Download ghettoVCB VIB build script RUN curl -O https://raw.githubusercontent.com/lamw/vghetto-scripts/master/shell/create_ghettoVCB_vib.sh && chmod +x create_ghettoVCB_vib.sh # Run ghettoVCB VIB build script RUN /root/create_ghettoVCB_vib.sh CMD ["/bin/bash"]
Step 3 - Next we need to build our new Docker Container which will use the VIB Author Container by running the following command:
docker build -t lamw/ghettovcb .
The output will be quite verbose, but what you will be looking for is text highlighted in green as shown in the screenshot above. You should see the successful build of both the VIB and offline bundle as well as Docker Container showing a successful build.
Step 4 - After a successful build of our Docker Container, we can now launch the container by running the following command:
docker run --rm -it lamw/ghettovcb
Once logged into the Docker Container, you will see the generated VIB and the offline bundle for ghettoVCB as shown in the screenshot above.
If you wish to copy the VIB and offline bundle out of the Docker Container into the Docker Host, you can use Docker Volumes. I found this useful thread over on Stack overflow which I have modified to include the copying of the ghettoVCB VIB and offline bundle out to Docker Host by running the following command:
docker run -i -v ${PWD}/artifacts:/artifacts lamw/ghettovcb sh << COMMANDS
cp vghetto-ghettoVCB* /artifacts
COMMANDS
Finally, to copy the ghettoVCB VIB from the Docker Host to your desktop, we first need to identify the IP Address given to our Docker Machine by running the following command:
docker-machine ip osxdock
Currently, Docker Machine does not include a simple "scp" command so we will need to use regular scp command and specify the private SSH keys which you can find by running "docker-machine inspect [NAME-OF-DOCKER-HOST]" and connecting to our Docker Host to copy the ghettoVCB VIB by running the following command:
scp -i /Users/lamw/.docker/machine/machines/osxdock/id_rsa [email protected]:artifacts/vghetto-ghettoVCB.vib .
Nice! thanks for the effort!
using it via cron to backup my vms weekly... to an nfs data store. which I write to tape the day after using bacula.
works all like a charm!
Can it do Incremental or Differential backup for VM? Thanks!
No, ghettoVCB only supports full backups as it clones the source VM. For more advanced/efficient backup mechanism you'll need to look at commercial products like vSphere Data Protection as an example which is included in all paid versions of vSphere
esxcli software install -v /vghetto-ghettoVCB.vib -f = esxcli software vib install -v /vghetto-ghettoVCB.vib -f
Thanks, I've fixed the typo
In the blog post you say there is a config file at /etc/ghetto/ghettoVCB.conf which is not the case. It's under /etc/ghettovcb/ghettoVCB.conf
But the bigger issue is that I can't edit and of the files in that folder. If I try to use vi to edit any of the files under /etc/ghettovcb I get a message saying "Operation not permitted"
But I was able to copy ghettoVCB.conf to myconfig.conf and then edit that. Which is likely what you intended in the first place, but it's not clear at first.
2) Could you add the outgoing port 25 firewall rule to your VIB as well? Currently I have my own VIB that I use for that, but it would be really nice if both ghettoVCB and the port 25 firewall rule could just be a single VIB.
Thanks!
Love this script! Been working flawlessly for years...we just replaced the VMWare host hardware and I thought I'd try the vib method, but it's not happy:
# esxcli software vib install -v /vghetto-ghettoVCB.vib
[DependencyError]
VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibility rule checks: [u'(line 23: col 0) Element vib failed to validate content']
Please refer to the log file for more details.
This is ESXi 5.5 fyi. Any ideas?
Thank you so much. -- MB
You need to specify -f option (this is mentioned in the instructions above)
Duhhhh. That worked of course. Thank you!!
Many thanks for VIB/bundle of this great tool! Just one question, how is it now with config-file? I think /etc is non-persistent, so any change I do to /etc/ghettovcb/ghettoVCB.conf will be lost after reboot. Or am I wrong?
i was wondering the same thing...
No,if you want to backup your configs, you can use the auto-backup.sh before rebooting. Thats backup your configs, but not all. Shell commands can be restore the configs by commands in /etc/rc.local.d/local.sh. Put in this your routines to restore the config in /etc's. I have in datastore a folder with scripts and configs. You can't edit the etc-files directly, edit this from datastore with copy promt to send etc-folders.
I managed to get this up and running in ESXI 6
I use a NFS share here is my install Guide
#First mount share and install vib
esxcli storage nfs add --host=10.10.10.206 --share=/var/nfs --volume-name=RemoteNFS || true
esxcli software vib install -v /vmfs/volumes/RemoteNFS/BackupSolution/vghetto-ghettoVCB.vib -f
#add to /etc/rc.local.d/local.sh## THis sets up the crontab and makes a local script at boot##
mkdir /etc/cron
echo '#!/bin/sh' > /etc/cron/script.sh
echo 'esxcli storage nfs add --host=10.10.10.206 --share=/var/nfs --volume-name=RemoteNFS || true' >> /etc/cron/script.sh
echo 'sh /opt/ghettovcb/bin/ghettoVCB.sh -a -g /vmfs/volumes/RemoteNFS/BackupSolution/config/esxi105.conf >> /vmfs/volumes/RemoteNFS/BackupSolution/log/cron.esxi105.log' >> /etc/cron/script.sh
chmod +x /etc/cron/script.sh
/bin/kill $(cat /var/run/crond.pid)
/bin/echo '* 2 * * * /etc/cron/script.sh' >> /var/spool/cron/crontabs/root
/usr/lib/vmware/busybox/bin/busybox crond
#see that I utilize the remote share for my logs in the script. The esxi file system requires the script be built every boot.
# I'm sure there are better ways to do this but this works for now.
#please not this is a single line from the above setup
echo ‘esxcli storage nfs add –host=10.10.10.206 –share=/var/nfs –volume-name=RemoteNFS || true’ >> /etc/cron/script.sh
Question
If I change the number of backups I want to keep per VM from 3 to 2 will it automatically delete the third one from the folders or do I have to do this manually?
This has been a great solution for our remote offices. Thanks for all that you have done.
I ran into an issue today testing upgrading hosts to ESXI 6 via the vSphere Update Manager. During the upgrade process, it errors out with the following error. Is there a way around this without uninstalling ghettoVCB?
VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibilty rule checks
irtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0's acceptance level is community, which is not compliant with the ImageProfile acceptance level partner
I found 1 error( no white space before ]]) and 1 warning (no double quote) in ghettoVCB.sh.
I hope already someone had modified.
$ diff ghettoVCB.sh ghettoVCB.sh.org
- if [[ "${EMAIL_LOG}" -eq 1 ]] || [[ "${EMAIL_ALERT}" -eq 1 ]] ; then
+ if [[ "${EMAIL_LOG}" -eq 1 ]] || [[ "${EMAIL_ALERT}" -eq 1]] ; then
- [[ "${BACKUP_ALL_VMS}" -eq 1 ]] && touch ${VM_FILE}
+ [[ $BACKUP_ALL_VMS -eq 1 ]] && touch ${VM_FILE}
Also I added SMTP=0 in function sendMail().
diff ghettoVCB.sh ghettoVCB.sh.org
--- ghettoVCB.sh
+++ ghettoVCB.sh.org
@@ -1323,8 +1323,7 @@
sendMail() {
#close email message
- SMTP=0
Is ghettoVCB compatible with VMware vSphere Enterprise 6.0? I am seeking a backup solution, but have heard it is compatible with 5.0 and maybe 5.5 and not 6.0.
Please advise
Yes, vSphere 6.0 is supported
Hello William ,
do you have any Videos regard use your toll.
I'm not Master to understand your Post can you just share step by step video pls
I don't see complete installation method for ghetto.vib, Could you please share the link for backup and restore and details ASAP.
Regards,
Thirumalai.M
In Esxi 6 free, I start my script with crontab. My script savevm.sh calls ghettovcb.sh but it doesn't wait the end of this. How I can call ghettovcb.sh and wait the results BEFORE continuing with the other lines of my script ? Thanks you for your answer and for your blog (and for ghetto, sure !)
Regards,
Stefano
For anyone interested, I created a Dockerfile that build the vib and offline bundle upon running (instead of building) the container. It also puts the vib/zip in /output which means you can run like 'docker run --rm -v /tmp:/output ' and then find the biv/bundle in /tmp on your docker host. Repo is at https://github.com/tssgery/ghetto-container
Hi,
Just for Information. in the ghettoVCB.sh (LAST_MODIFIED_DATE=2016_11_20) you have to modify line 781 from:
if [[ ${ESX_RELEASE} == "5.5.0" ]] || [[ ${ESX_RELEASE} == "6.0.0" ]] ; then
to:
if [[ ${ESX_RELEASE} == "5.5.0" ]] || [[ ${ESX_RELEASE} == "6.0.0" ]] || [[ ${ESX_RELEASE} == "6.5.0" ]] ; then
otherwise ESXi 6.5 host will not mount the NFS Share and you probably get an Error like:
Add a NAS datastore.
readonly is a boolean value, 1 for readonly and 0 for rw access.
NFS Version must be nfs for nfs v3 or nfsv41 for version 4.1 or later
This has been a great solution for our remote offices. Thanks for all that you have done.
I ran into an issue today testing upgrading hosts to ESXI 6 via the vSphere Update Manager. During the upgrade process, it errors out with the following error. Is there a way around this without uninstalling ghettoVCB?
VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibilty rule checks
irtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0’s acceptance level is community, which is not compliant with the ImageProfile acceptance level partner
The errors you getting are due to ESXi now using python 3.x instead of 2.7
If you are looking to keep using ghettoUI, I have made some crude changes and put the results at https://github.com/tssgery/ghettoVCB/blob/master/ghettoUI. It seems to run fine on ESXi 6.5. You will need to make sure that that your ghettoVCB.sh works on ESXi 6.x as well. I'd suggest you use the version in the base project repo but the version in mine should work as well.
If you are looking for steps to get the scripts working well, the instructions at https://communities.vmware.com/docs/DOC-8760 are pretty good. I followed them to get started and have since come up with my own process that make it really easy to change the configuration.
Start simple and post back if you have issues or questions
Hello, William.
Are you have ghettoVCB-restore.sh with ESXi 6.5 support?
I'm download it from github and after run:
./ghettoVCB-restore.sh -c vms_to_restore
You're not running ESX(i) 3.5, 4.x, 5.x & 6.x!
Thnx for the answear.
If you'd like to make the change yourself, it's a pretty easy fix. Change line 96 of ghettoVCB-restore.sh:
from
6.0.0) VER=6; break;;
to
6.0.0|6.5.0) VER=6; break;;
Thnx, dude, its realy works!
Thanks for suggesting the quick fix Eric.
I've not had time to go and update the script but its pretty easy as nothing has changed from what ghettoVCB relies upon 🙂
I've just rev'ed the scripts as well as the VIB/offline bundle for 6.5 on Github. Enjoy
Great work. Thanks, William, it very perfect tool!
I'm getting errors when I try to install this on ESXi 6.0
esxcli software vib install -d vghetto-ghettoVCB.vib -f
[MetadataDownloadError]
Could not download from depot at /vghetto-ghettoVCB.vib/index.xml, skipping (('/vghetto-ghettoVCB.vib/index.xml', '', "[Errno 4] IOError: "))
url = /vghetto-ghettoVCB.vib/index.xml
Please refer to the log file for more details.
I had the same issue. I ended up uninstalling ghettoVCB and then reinstalling after the upgrade.
Hello,
I have install ghettoVCB in esxi 6.5 but recieve this error mesage: in older version work perfect.
[root@nod04:/vmfs/volumes/5752ee24-ade32f92-a423-001517747800] esxcli software vib install -d /vmfs/volumes/5752ee24-ade32f92-a423-001517747800/vghetto-ghettoVCB-offline-bundle.zip
[DependencyError]
VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibility rule checks: ['(line 26: col 0) Element vib failed to validate content'] Please refer to the log file for more details.
any sugestion?
Thanks.
SOLUTION:
esxcli software vib install -v https://github.com/lamw/ghettoVCB/raw/master/vghetto-ghettoVCB.vib -f
and it is ok.
After upgrade ESXi 5.0.0 to 6.5.0 ghetto vcb script can't send email notifications.
2017-05-30 23:21:39 -- info: ###### Final status: WARNING: All VMs backed up, but some disk(s) failed! ######
2017-05-30 23:21:39 -- info: ============================== ghettoVCB LOG END ================================
2017-05-30 23:22:56 -- info: ERROR: Failed to email log output to 192.168.120.205:25 to *protected email*
Make sure that outbound access on port 25 is enabled in the firewall. The upgrade probably undid any configuration change you did there.
Hello
I was attempting to upgrade by ESXI server with 6.5.0 4564106 server with the command sxcli software profile update -p ESXi-6.5.0-20170404001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml and I got the following error.
[DependencyError]
VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibility rule checks: ['(line 24: col 0) Element vib failed to validate content']
Please refer to the log file for more details. [root@VHost1:~] VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibility rule VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibility rule
Any ideas how I can work around this error?
Thanks
Hello Friends,
We have been running ghettovcb in user interactive mode(ghettoUI) so far successfully till vsphere version 6.0. However, after upgrading our hosts to vsphere 6.5 , the ghettoUI script stopped working giving errors "invalid syntax" . Its probably not going to work it that way anymore. However, can someone provide me with some steps about how to run ghettovcb script for taking backup of VMs on vsphere 6.5?? I would highly appreciate as I am not good with scripts myself.
A simplified step by step procedure if provided will surely help.
Best Regards
The errors you getting are due to ESXi now using python 3.x instead of 2.7
If you are looking to keep using ghettoUI, I have made some crude changes and put the results at https://github.com/tssgery/ghettoVCB/blob/master/ghettoUI. It seems to run fine on ESXi 6.5. You will need to make sure that that your ghettoVCB.sh works on ESXi 6.x as well. I’d suggest you use the version in the base project repo but the version in mine should work as well.
If you are looking for steps to get the scripts working well, the instructions at https://communities.vmware.com/docs/DOC-8760 are pretty good. I followed them to get started and have since come up with my own process that make it really easy to change the configuration.
Start simple and post back if you have issues or questions
Hello William,
Thank you very much for a great simple ESXi backup program.
I did find a flaw in the ghettoVCB-restore.sh script. On line 170 in the current git version, the line reads:
VM_ORIG_VMX=$(ls "${VM_TO_RESTORE}" | grep ".vmx")
If there is a .vmxf file in the folder, subsequent usage of the VM_ORIG_VMX variable fails because it contains ".vmxLF.vmxf". I fixed this by adding a $ in your statement as follows:
VM_ORIG_VMX=$(ls "${VM_TO_RESTORE}" | grep ".vmx$")
I'm surprised nobody else came across this. Is there something unique as to why I have .vmxf files?
Hope this helps.
Thank you again.
Best,
Chuck
Hello William,
Thanks for this amazing Backup Script.
There's a problem with snapshots on ESXi 6.5 and 6.7 VMs, the script doesn't see that the VM has a Snapshot on it.
So ghettoVCB takes a Backup of the VM with the snapshot, but we won't backup a VM with a snapshot.
We figured out to change delta/.vmdk to sesparse/.vmdk in 3 lines of the ghettoVCB.sh, the script see the snapshot and don't take a backup.
But at the end of the log, it tells us everthing is fine.
Best Regards
Dear William!
I have run Esxi 5.5 until yesterday (I knwo, I know) and updatet it now to 6.0 (server HW to old)
Backup was done by ghettovcb for 5 years now perfectly, now its telling me, I am not running 1 esxi 5
So I updated with the actual vib, but still the same issue?
Whats wrong in my way of solution.
Thx
Alex
I have no idea how to follow the install instructions. I've tried both
esxcli software vib install -d vghetto-ghettoVCB.vib -f
and
esxcli software vib install -v /vghetto-ghettoVCB.vib -f
But neither do anything.
First one gives:
[MetadataDownloadError]
Could not download from depot at vghetto-ghettoVCB.vib/index.xml, skipping (('vghetto-ghettoVCB.vib/index.xml', '', "unknown url type: 'vghetto-ghettoVCB.vib/index.xml'"))
url = vghetto-ghettoVCB.vib/index.xml
Please refer to the log file for more details.
Second gives:
[VibDownloadError]
('/vghetto-ghettoVCB.vib', '/tmp/vib_o5ppqp28', "unknown url type: '/vghetto-ghettoVCB.vib'")
url = /vghetto-ghettoVCB.vib
Please refer to the log file for more details.
What concept am I missing?
-d is for installing offline bundle and -v is for installing a VIB. In either option, you need to pass the FULL path to the package you're installing. It looks like you're trying to install the VIB and assuming you properly downloaded the file, then you need to use "esxcli software vib install -v /vghetto-ghettoVCB.vib -f". I'd double check to make sure your download was done properly by right-clicking and download from https://github.com/lamw/ghettoVCB which also outlines the instructions
So feeling a bit stupid here (Really new to Esxi) but download it to where and with what? Do I download that with a normal browser and upload it to esxi using Datastore browser? (Which is what I tried) Or do I download it from the esxi cli? (I tried curl but that's not present in the cli, so not sure how to download it.)
Thanks for helping a newb! 🙂
If you have direct internet access from ESXi host, you can use `wget` as cURl isn't available. You'll need to be running ESXi 7.0 which has wget version supporting HTTPS
Easiest method is to simply download via your browser and then upload via SCP to your ESXi host. Depending on where it goes on the ESXi filesystem, that'll tell you path. I recommend just dropping it in / so you can follow the instructions
Updated to esxi 7. Downloaded and installed latest version of ghettovcb when running the dry run command it comes back as says not running esxi 4, 5, or 6.5. Have been running ghettovcb on others 6 & 6.5 no issues is there something special about version 7?
Hi Julian, I got the same message after ESXi upgrade to 7.0 "You're not running ESX(i) 3.5, 4.x, 5.x & 6.x!"
what solved it for me on 7.0 was simply removing the old vib, and then downloading and installing latest version of it. Also ghettoVCB v1.0 but slightly newer filestamp etc and works beautifully on 7.0 !
Many thanks to Mr. William Lam for this awsome little package that still saves our days after all these years 🙂
It looks like the -n option is no longer supported with esxcli vib list. Might need to change the instructions to esxcli software vib list | grep ghettoVCB
Thanks Dennis. I've updated to use `get` operation which does support the -n flag
Quick question. When removing ghettoVCB do I need to put the host in maintenance mode and then reboot it after the removal?
Thanks.
Nope. You not install/remove
1 Question, Is the ghetto able to send an email through the 587 ssl port. I will enable smtp port on esxi. nc : nc smtp.gmail.com 587
220 smtp.gmail.com ESMTP 8-20020a2e1548000000b002463639d0f2sm1088889ljv.68 - gsmtp
Hi everyone.
@William, great job! I use your script since the beginning. Thanks a lot!
Just to say that I installed the latest vib that is on the github and noticed that the "ghettoVCB.conf" is not in the /etc/ghettovcb directory but in /opt/ghettovcb.
Regards.
Hi ! Any chance of getting this to work on ESXi 8 somehow ? 🙂
[ProfileValidationError] In ImageProfile (Updated) ESXi-8.0.0-20513097-standard, the payload(s) in VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 does not have sha-256 gunzip checksum.
HI!,
I tried to install it but I get the following error:
esxcli software vib install -v /vghetto-ghettoVCB.vib -f
[InstallationError]
Calculated digest at EOF does not match expected result: 07d3fbf45049ba932a2f06e590adfcf268a3e4036be56c8dcec82c79b611bd92 calculated, 4c235dce439ffdf02c877849b1247b3adeb113ce38e3a0d26c08be018c7ea56b expected.
vibs = virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0
Please refer to the log file for more details.
Can't find any online solution for it.
Thank you
I took an older version and it works fine. But it was also an older ESXi 6.x.
Hi everyone.
@William, great job! I use your script since the beginning. Thanks a lot!
I modified the script to send Emails over ssl on o365.
openssl s_client -connect dns-fr.mail.protection.outlook.com:25 -starttls smtp "${EMAIL_LOG_HEADER}"
by
echo -ne "HELO $(hostname)\r\n" > "${EMAIL_LOG_HEADER}"
to be more compatible
any ideas ?
all the best
openssl s_client -connect dns-fr.mail.protection.outlook.com:25 -starttls smtp < /tmp/A_mail.work/email-3969468.content
I have no authentication, I am writing to myself.
Does it only work for 1 of my servers?
it is not a PB of SPF.
I don't have a log on M$
on a postfix I have logs.
postfix/smtpd[58391]: improper command pipelining after EHLO from
to be more compatible