WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Hardware Options
    • Hardware Reviews
    • Lab Deployment Scripts
    • Nested Virtualization
    • Homelab Podcasts
  • VMware Nostalgia
  • Apple

Creating Custom vSphere Reports is a Breeze with CloudPhysics

06.19.2013 by William Lam // 2 Comments

Creating reports is a common task that every vSphere administrator must deal with at least once if not many more times in their career. Whether you are tasked to provide an inventory report of all your virtual machines and their configurations to your manager or to provide a compliance report for your security team to ensure that all virtual machines are hardened according to the vSphere Security Hardening Guide, report creations can be a challenge.

The vSphere platform provides a very powerful and rich set of APIs (Application Programming Interface) that can be consumed by both vSphere administrators as well as developers. However, there is a high learning curve when using the API and it takes quite a bit of time to learn and of course your manager is expecting the report to be done in the next 5 minutes. Even with abstraction tools such as PowerCLI, quickly building a robust, scalable and performant script is not always a trivial task, not to mention the maintenance and updates to the script because your manager wants to continually add more things to the report.

So how can we make reporting so easy that vSphere administrators will no longer have to spend time digging through API documentation and instead they will be able to quickly put together reports within minutes? Well, this is something that the CloudPhysics team has been working on as part of their CloudPhysics card platform and they have built a very unique card solution to help solve this problem. I had the opportunity to get an early preview of this new card and I have to say that I am very impressed at how easy and intuitive the interface is to build simple to very complex reports using this new card solution. The coolest part of this solution is that no programming or scripting skills are required!

To give you an example on how easy it is to use the interface, I recently helped a customer with a script to identify all virtual machines that had a virtual disk using the 2gbsparse disk format. I would like to think I know the vSphere API pretty well, so putting the script together took just a few minutes because I knew exactly where to look for this information. That evening I decided to go through the same scenario, but using the new CloudPhysics card solution and I was literally able to create the report in seconds! It probably took me longer to name the report than to actually create it. As you can see, I am pretty excited about the new card solution and it will be interesting to see all the cool new reports customers can now create and share with each other.

Here is a sneak peak at what the interface looks like when creating your own custom reports:

Here is the final report that is produced:

As you can see, I have quickly narrowed down the specific virtual machine that contains a 2gbsparse VMDK and I am able to see exactly which virtual disk that is. 

If you would like to learn more about the new card solution, Irfan Ahmad, CTO of CloudPhysics will be hosting a live webinar to go over this new solution next Tuesday, June 25th at 9am PST and I would highly recommend you register for it to learn more.

Below is a bit more details on what you can expect from the webinar and you can register here.

vSphere Analytics Without Writing Code: The Quest for Missing Reports

While vSphere, the best-of-class virtualization platform, brings great efficiencies to the datacenter, reporting still presents challenges and pain to sys admins on a daily basis. CloudPhysics offers a radical new way to complete reporting for your virtual infrastructure. In addition to 20 high-impact reports, you can easily build your own and share the report template and output. When asked for asset reports, trending, activity, auditing and more, you’re never more than a few clicks away from delivery.
  
In addition to best practices and secrets to amazing mashups, you’ll learn to:
  • Create easy, visual reports for your vSphere environment
  • Add multiple vCenters in one view
  • Automate the report generation

Categories // Uncategorized Tags // api, cloudphysics, reporting, sdk, vSphere

Will Intel’s VMCS Shadowing Feature Benefit VMware’s Nested Virtualization?

06.18.2013 by William Lam // 1 Comment

For many years now, VMware customers have been using Nested Virtualization, which is the ability to run a hypervisor such as vSphere ESXi within a virtual machine. Even though Nested Virtualization is not officially supported by VMware, customers have come to rely upon this technology for their lab environments and sometimes even production environments. VMware also heavily relies on this technology for their own internal development as well as their Hands On Lab for VMworld, which is now offered as an online SaaS (Software-as-a-Service) solution called Hands On lab Online.

Performance of Nested Virtualization has come a long way since its first introduction and it continues to get better with advancements made in hardware from both Intel and AMD. A couple of months back, I came across an article discussing a new feature from the upcoming Intel Haswell processor’s called VMCS Shadowing which aims to improve the performance of Nested Virtualization. This got me thinking about whether VMCS Shadowing could benefit VMware’s Nested Virtualization.

VMCS (Virtual Machine Control Structure) Shadowing works by reducing the frequency in which the guest VMM (virtual machine) requires assistance from the parent VMM. Its goal is to eliminate the VM-exits due to VMREAD and VMWRITE instructions executed by the guest hypervisor but this comes at a slight expense.

I reached out to one of the core engineers who helped to develop VMware’s Nested Virtualization technology, Jim Mattson, and asked whether or not we would benefit from the VMCS Shadowing feature. Well, it turns out that VMCS Shadowing can help, but we have also done some research in this area and developed some technology that would allow us to eliminate about 75% due to VMREAD and VMWRITE when running guest VMware Hypervisors using some interesting software techniques. The details of these software techniques are actually published in a research paper called Software Techniques for Avoiding Hardware Virtualization Exits on VMware’s Academic Program which is part of VMware Labs. Jim is one of the authors of the research paper and I would highly recommend you check it out if you are interested in more details.

To summarize, because of the techniques described in the paper, VMCS Shadowing will provide only a small benefit when running a VMware Hypervisor as virtual machine. However, it will greatly benefit other non-VMware Hypervisors running as a virtual machine, this is particular true for Hypervisors that perform egregious number of VMREAD and VMWRITE operations and that do not cluster well, such as VirtualBox for example.

The coolest part about the research and software techniques developed by Jim and team, is that the technology has already been incorporated into the existing VMware vSphere ESXi, Workstation and Fusion products. I often times forget that all the awesome-sauce technology that is being developed by VMware starts out in research academia and you can learn about other research topics by visiting the VMware’s Academic Program which includes publications, research papers and the popular VMware Technical Journals.

Categories // Uncategorized Tags // AMD, Intel, nested, nested virtualization, vhv, VMCS, vmware

Forwarding Logs From The vCloud Suite To vCenter Log Insight

06.17.2013 by William Lam // 18 Comments

An exciting new product was just announced last week by VMware called vCenter Log Insight, which will be part of the vCenter Operations Management Suite when released. The announcement also includes a public beta for customers to try out the new log analytics product that allows administrators to easily get an understanding of both their physical and virtual infrastructure through the collection of log data. You can get more details on how vCenter Log Insight works by checking out this article by the Jon Herlocker, who is in the Office of CTO and focusing on vCenter Log Insight.

I had known about vCenter Log Insight for quite sometime now and like others within VMware, I had the opportunity to test drive the product early on and provide feedback to the engineering team. One of neatest thing about vCenter Log Insight, in my opinion, is the simplistic setup and the tight integration between vCenter Server and vCenter Operations Manager. During the setup of vCenter Log Insight, I was reminded about an article that I had written about forwarding vCenter Server logs to a syslog server. I thought, would it not be cool if we could forward logs from other products within the vCloud Suite to vCenter Log Insight using the same syslog-ng trick? I decided to compile a list of logs from each of the products within the vCloud Suite shared that internally and thanks to my colleague Michael White who also help vet the list by circulating it within engineering.

I then decided to create a very simple script called configurevCloudSuiteSyslog.sh that would allow users to easily configure each of the vCloud Suite products to forward their appropriate logs to vCenter Log Insight. The script is very simple to use, you just need to scp the script to one of the supported appliances within the vCloud Suite and specify the VMware solution name and the IP Address of your vCenter Log Insight Server.

Here is an example of running the script on the VCSA (vCenter Server Appliance):

Based on the VMware solution selected, the appropriate logs will be appended to /etc/syslog-ng/syslog-ng.conf to be forwarded off to your vCenter Log Insight Server. The syslog-ng client will automatically be restarted for the changes to go into effect as part of the script. In my environment, I have deployed the majority of products within the vCloud Suite installed and have configured each of them to forward their logs to vCenter Log Insight. This can be very useful from a troubleshooting perspective and being able to view and filter through all the relevant logs from a single location.

It was really interesting to see what the next "chattiest" VMware solution was from a log perspective in my environment, which turned out to be VIN after vCenter Server and ESXi host. I hope to see deeper integration between vCenter Log Insight and the rest of the vCloud Suite in future releases, but for now, if you have not tried out vCenter Log Insight, I would highly recommend you give it a try and provide any feedback you may have in the dedicated VMTN community forum.

If you are interested in the specifics logs that are being collected for each of VMware products, you can find the complete list below. Not all products from the vCloud Suite are listed here and some such as vCloud Director and vCloud Networking & Security provide native syslog configuration from the application standpoint which can be configured using either their UIs or APIs.

vCenter Operations Manager Analytics (VCOPS):

/var/log/vmware/diskadd.log
/var/log/vmware/vcops-admin.log
/var/log/vmware/vcops-firstboot.log
/var/log/vmware/vcops-watch.log 

vCenter Operations Manager UI (VCOPS):

/var/log/vmware/admin.log
/var/log/vmware/ciq-firstboot.log
/var/log/vmware/ciq.log
/var/log/vmware/diskadd.log
/var/log/vmware/lastupdate.log
/var/log/vmware/mod_jk.log
/var/log/vmware/vcops-admin.cmd.log
/var/log/vmware/vcops-admin.log
/var/log/vmware/vcops-firstboot.log
/var/log/vmware/vcops-watch.log
/var/log/vmware/diskadd.log
/var/log/vmware/vcops-admin.log
/var/log/vmware/vcops-firstboot.log
/var/log/vmware/vcops-watch.log 

vCenter Orchestrator (VCO):

/opt/vmo/app-server/server/vmo/log/boot.log
/opt/vmo/app-server/server/vmo/log/console.log
/opt/vmo/app-server/server/vmo/log/server.log
/opt/vmo/app-server/server/vmo/log/script-logs.log
/opt/vmo/configuration/jetty/logs/jetty.log 

vCenter Server Appliance (VCSA):

/var/log/vmware/vpx/vpxd.log
/var/log/vmware/vpx/vpxd-alert.log
/var/log/vmware/vpx/vws.log
/var/log/vmware/vpx/vmware-vpxd.log
/var/log/vmware/vpx/inventoryservice/ds.log 

vCloud Connector Node (VCC):

/opt/vmware/hcagent/logs/hca.log 

vCloud Connector Server (VCC):

/opt/vmware/hcserver/logs/hcs.log 

vSphere Data Protection (VDP):

/space/avamar/var/log/av_boot.rb.log
/space/avamar/var/log/dpnctl.log
/space/avamar/var/log/dpnnetutil-av_boot.log
/usr/local/avamar/var/log/dpnctl.log
/usr/local/avamar/var/log/av_boot.rb.log
/usr/local/avamar/var/log/av_boot.rb.err.log
/usr/local/avamar/var/log/dpnnetutil-av_boot.log
/usr/local/avamar/var/avi/server_log/flush.log
/usr/local/avamar/var/avi/server_log/avinstaller.log.0
/usr/local/avamar/var/vdr/server_logs/vdr-server.log
/usr/local/avamar/var/vdr/server_logs/vdr-configure.log
/usr/local/avamar/var/flr/server_logs/flr-server.log
/data01/cur/err.log
/usr/local/avamarclient/bin/logs/VmMgr.log
/usr/local/avamarclient/bin/logs/MountMgr.log
/usr/local/avamarclient/bin/logs/VmwareFlrWs.log
/usr/local/avamarclient/bin/logs/VmwareFlr.log 

vCloud Director (VCD):

/opt/vmware/vcloud-director/logs/vcloud-container-debug.log
/opt/vmware/vcloud-director/logs/vcloud-container-info.log
/opt/vmware/vcloud-director/logs/jmx.log 

vSphere Infrastructure Navigator (VIN):

/var/log/vadm/system.log
/var/log/vadm/engine.log
/var/log/vadm/activecollector.log
/var/log/vadm/dbconfig.log
/var/log/vadm/db/postgresql.log 

vSphere Management Assistance (VMA):

/var/log/vmware/vma/vifpd.log 

vSphere Replication (VR):

/var/log/vmware/hbrsrv.log 

Categories // Uncategorized Tags // syslog, vC Log, vCenter Log Insight, vcloud suite

  • « Previous Page
  • 1
  • …
  • 454
  • 455
  • 456
  • 457
  • 458
  • …
  • 565
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • PowerCLI remediation script for running NSX Edge on AMD Ryzen for VCF 9.0 06/20/2025
  • Failed to locate kickstart on Nested ESXi VM CD-ROM in VCF 9.0 06/20/2025
  • NVMe Tiering with Nested Virtualization in VCF 9.0 06/20/2025
  • VCF 9.0 Installer workaround for ESXi hosts with different vendor 06/19/2025
  • NVMe Tiering with AMD Ryzen CPU workaround for VCF 9.0 06/19/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025