WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple
You are here: Home / Uncategorized / Forwarding Logs From The vCloud Suite To vCenter Log Insight

Forwarding Logs From The vCloud Suite To vCenter Log Insight

06.17.2013 by William Lam // 18 Comments

An exciting new product was just announced last week by VMware called vCenter Log Insight, which will be part of the vCenter Operations Management Suite when released. The announcement also includes a public beta for customers to try out the new log analytics product that allows administrators to easily get an understanding of both their physical and virtual infrastructure through the collection of log data. You can get more details on how vCenter Log Insight works by checking out this article by the Jon Herlocker, who is in the Office of CTO and focusing on vCenter Log Insight.

I had known about vCenter Log Insight for quite sometime now and like others within VMware, I had the opportunity to test drive the product early on and provide feedback to the engineering team. One of neatest thing about vCenter Log Insight, in my opinion, is the simplistic setup and the tight integration between vCenter Server and vCenter Operations Manager. During the setup of vCenter Log Insight, I was reminded about an article that I had written about forwarding vCenter Server logs to a syslog server. I thought, would it not be cool if we could forward logs from other products within the vCloud Suite to vCenter Log Insight using the same syslog-ng trick? I decided to compile a list of logs from each of the products within the vCloud Suite shared that internally and thanks to my colleague Michael White who also help vet the list by circulating it within engineering.

I then decided to create a very simple script called configurevCloudSuiteSyslog.sh that would allow users to easily configure each of the vCloud Suite products to forward their appropriate logs to vCenter Log Insight. The script is very simple to use, you just need to scp the script to one of the supported appliances within the vCloud Suite and specify the VMware solution name and the IP Address of your vCenter Log Insight Server.

Here is an example of running the script on the VCSA (vCenter Server Appliance):

Based on the VMware solution selected, the appropriate logs will be appended to /etc/syslog-ng/syslog-ng.conf to be forwarded off to your vCenter Log Insight Server. The syslog-ng client will automatically be restarted for the changes to go into effect as part of the script. In my environment, I have deployed the majority of products within the vCloud Suite installed and have configured each of them to forward their logs to vCenter Log Insight. This can be very useful from a troubleshooting perspective and being able to view and filter through all the relevant logs from a single location.

It was really interesting to see what the next "chattiest" VMware solution was from a log perspective in my environment, which turned out to be VIN after vCenter Server and ESXi host. I hope to see deeper integration between vCenter Log Insight and the rest of the vCloud Suite in future releases, but for now, if you have not tried out vCenter Log Insight, I would highly recommend you give it a try and provide any feedback you may have in the dedicated VMTN community forum.

If you are interested in the specifics logs that are being collected for each of VMware products, you can find the complete list below. Not all products from the vCloud Suite are listed here and some such as vCloud Director and vCloud Networking & Security provide native syslog configuration from the application standpoint which can be configured using either their UIs or APIs.

vCenter Operations Manager Analytics (VCOPS):

/var/log/vmware/diskadd.log
/var/log/vmware/vcops-admin.log
/var/log/vmware/vcops-firstboot.log
/var/log/vmware/vcops-watch.log 

vCenter Operations Manager UI (VCOPS):

/var/log/vmware/admin.log
/var/log/vmware/ciq-firstboot.log
/var/log/vmware/ciq.log
/var/log/vmware/diskadd.log
/var/log/vmware/lastupdate.log
/var/log/vmware/mod_jk.log
/var/log/vmware/vcops-admin.cmd.log
/var/log/vmware/vcops-admin.log
/var/log/vmware/vcops-firstboot.log
/var/log/vmware/vcops-watch.log
/var/log/vmware/diskadd.log
/var/log/vmware/vcops-admin.log
/var/log/vmware/vcops-firstboot.log
/var/log/vmware/vcops-watch.log 

vCenter Orchestrator (VCO):

/opt/vmo/app-server/server/vmo/log/boot.log
/opt/vmo/app-server/server/vmo/log/console.log
/opt/vmo/app-server/server/vmo/log/server.log
/opt/vmo/app-server/server/vmo/log/script-logs.log
/opt/vmo/configuration/jetty/logs/jetty.log 

vCenter Server Appliance (VCSA):

/var/log/vmware/vpx/vpxd.log
/var/log/vmware/vpx/vpxd-alert.log
/var/log/vmware/vpx/vws.log
/var/log/vmware/vpx/vmware-vpxd.log
/var/log/vmware/vpx/inventoryservice/ds.log 

vCloud Connector Node (VCC):

/opt/vmware/hcagent/logs/hca.log 

vCloud Connector Server (VCC):

/opt/vmware/hcserver/logs/hcs.log 

vSphere Data Protection (VDP):

/space/avamar/var/log/av_boot.rb.log
/space/avamar/var/log/dpnctl.log
/space/avamar/var/log/dpnnetutil-av_boot.log
/usr/local/avamar/var/log/dpnctl.log
/usr/local/avamar/var/log/av_boot.rb.log
/usr/local/avamar/var/log/av_boot.rb.err.log
/usr/local/avamar/var/log/dpnnetutil-av_boot.log
/usr/local/avamar/var/avi/server_log/flush.log
/usr/local/avamar/var/avi/server_log/avinstaller.log.0
/usr/local/avamar/var/vdr/server_logs/vdr-server.log
/usr/local/avamar/var/vdr/server_logs/vdr-configure.log
/usr/local/avamar/var/flr/server_logs/flr-server.log
/data01/cur/err.log
/usr/local/avamarclient/bin/logs/VmMgr.log
/usr/local/avamarclient/bin/logs/MountMgr.log
/usr/local/avamarclient/bin/logs/VmwareFlrWs.log
/usr/local/avamarclient/bin/logs/VmwareFlr.log 

vCloud Director (VCD):

/opt/vmware/vcloud-director/logs/vcloud-container-debug.log
/opt/vmware/vcloud-director/logs/vcloud-container-info.log
/opt/vmware/vcloud-director/logs/jmx.log 

vSphere Infrastructure Navigator (VIN):

/var/log/vadm/system.log
/var/log/vadm/engine.log
/var/log/vadm/activecollector.log
/var/log/vadm/dbconfig.log
/var/log/vadm/db/postgresql.log 

vSphere Management Assistance (VMA):

/var/log/vmware/vma/vifpd.log 

vSphere Replication (VR):

/var/log/vmware/hbrsrv.log 

More from my site

  • A Hidden vSphere 5.1 Gem - Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 2
  • A Hidden vSphere 5.1 Gem - Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 1
  • A preview of native syslog support in VCSA 6.0
  • How to Quickly Get Started with VMware vSphere & OpenStack?
  • Quick Tip - Monitoring ESXi remote syslog forwarding

Categories // Uncategorized Tags // syslog, vC Log, vCenter Log Insight, vcloud suite

Comments

  1. *protectedAnonymous says

    06/18/2013 at 3:05 pm

    Great post!

    An additional log I think bears monitoring on the VSVA is the vSphere Web Client log at /var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log

    - David

    Reply
  2. *protectedViktor says

    06/19/2013 at 1:44 pm

    Hi William, trying to get your script working on vCloud Director 5.1. Running into some troubles; vCloud Director is using rsyslog instead of syslog, the syslog configuration file is in a different location (/etc/rsyslog.conf). Maybe I misinterpreted your script....any advise is welcome!

    Best regards - Viktor

    Reply
    • *protectedWilliam Lam says

      06/19/2013 at 1:52 pm

      Hi Viktor,

      Are you using the VCD appliance or you've installed your own? If it's the latter, that would explain why you may not see /etc/syslog-ng/syslog-ng.conf

      Reply
    • *protectedViktor says

      06/19/2013 at 2:04 pm

      Hi William. It's the latter, so that would explain things. There are also some challenges regarding IP Tables/Firewall. Will do some additional on this 🙂

      Reply
    • *protectedViktor says

      06/19/2013 at 2:04 pm

      +investigation

      Reply
  3. *protectedRyan Kelly says

    06/25/2013 at 4:38 pm

    Hi William, I think it would also be beneficial to also add vShield edge syslogs.

    Reply
    • *protectedWilliam Lam says

      06/26/2013 at 3:43 pm

      Ryan,

      The article was specific to virtual appliances that did not support native syslog forwarding. Since vCNS supports native syslog for both the Manager & Edge, I intentionally left that out.

      Reply
  4. *protectedIwan 'e1' Rahabok says

    09/19/2013 at 2:05 am

    Thanks William. Very useful.
    Will LogInsight know which log entries is from which products? I hope each entry is prefixed with the product names, or does Log Insight "knows" it by way of configuration.
    Thanks from Singapore.

    Reply
    • *protectedWilliam Lam says

      09/19/2013 at 3:00 am

      No it does not, you can do so by adding "prefix" to the VMware solution or any other solution for that matter. Take a look at this article here http://www.virtuallyghetto.com/2013/05/how-to-add-tag-log-prefix-to-syslog.html for more details

      Reply
  5. *protectedChristian Dickmann says

    10/05/2013 at 7:13 pm

    William, thanks a lot for this post! You may want to call out one caveat:
    If I am observing this correctly syslog is first catching up after being restarted, i.e. it is streaming the old logs into LogInsight. I was surprised because I couldn't find a log, but then I noticed it hadn't finished streaming the old log lines yet. So I gave it some time (30-40 minutes) and then it was live as usual.

    Reply

Trackbacks

  1. Newsletter: May 25, 2014 | Notes from MWhite says:
    05/26/2014 at 8:22 pm

    […] of that since I am using it now and quite like it.  The info for the vCSA logs can be found in this and you can find the Windows log file in C:ProgramDataVMwareVMware VirtualCenterLogs.  […]

    Reply
  2. vCenter Operations 5.7.1 + vCenter Log Insight = Maximum Visibility - viktorious.nl - Virtualization & Cloud Management says:
    07/05/2014 at 8:43 am

    […] Insight, also don't forget to check this video recorded by Eric Sloof.William Lam wrote this nice article on forwarding logs of other VMware products to Log […]

    Reply
  3. David Davis on vRealize Operations – Post #20 – Log Insight 2.0 and vROps 6.0 Integration in VMware Hands On Labs | VMware Cloud Management - VMware Blogs says:
    11/25/2014 at 2:00 pm

    […] send vCenter server log data to Log Insight, a script must be run. William Lam's blog post on Forwarding Logs from the vCloud Suite to Log Insight makes this easy. I put the script on my vCenter Server Appliance (in the lab of course) and it […]

    Reply
  4. Newsletter: January 10, 2015 | Notes from MWhite says:
    01/10/2015 at 11:45 pm

    […] of different Windows or Windows related apps – like vC in this article. Here is a good article to configure appliances in the vCloud […]

    Reply
  5. Automate forwarding of vRealize Operations Manager logs to Syslog Server | virtuallyGhetto says:
    01/18/2015 at 5:59 am

    […] Operations Manager has been greatly simplified with the latest 6.0 release. Instead of having to manually tweak syslog-ng.conf with past releases, there is now a new UI option under Administration->Support->Logs which allows you to easily […]

    Reply
  6. Vcloud Suite | mesotheliomalawyertexas says:
    02/03/2016 at 2:28 pm

    […] Forwarding Logs From The vCloud Suite To vCenter Log … – An exciting new product was just announced last week by VMware called vCenter Log Insight, which will be part of the vCenter Operations Management Suite when released. […]

    Reply
  7. David Davis on vRealize Operations – Post #20 – Log Insight 2.0 and vROps 6.0 Integration in VMware Hands On Labs - VMware Cloud Management says:
    06/24/2016 at 4:57 pm

    […] send vCenter server log data to Log Insight, a script must be run. William Lam's blog post on Forwarding Logs from the vCloud Suite to Log Insight makes this easy. I put the script on my vCenter Server Appliance (in the lab of course) and it […]

    Reply
  8. vCenter Operations 5.7.1 + vCenter Log Insight = Maximum Visibility | viktorious.nl – Virtualization & Cloud Management says:
    01/11/2017 at 8:49 am

    […] Insight, also don't forget to check this video recorded by Eric Sloof.William Lam wrote this nice article on forwarding logs of other VMware products to Log […]

    Reply

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025