In the last few days, I have been ramping back up on NSX-T usage, especially as I take a closer look at VMware Pivotal Container Service (PKS) which hopefully I will be able to share more with an upcoming blog series (at least, thats the plan when I find some spare time). While working on some Automation using the NSX-T REST APIs and the new NSX-T PowerCLI cmdlets, I found it to be pretty challenging and frustrating to troubleshoot NSX-T error messages thrown from the PowerCLI cmdlets.
For example, below is a PowerCLI snippet which I wrote to create a new T1 Logical Router in NSX-T.
$edgeClusterService = Get-NsxtService -Name "com.vmware.nsx.edge_clusters"
$edgeCluster = $edgeClusterService.list().results | where { $_.display_name -eq "Edge-Cluster-01" }
$logicalRouterService = Get-NsxtService -Name "com.vmware.nsx.logical_routers"
$lrSpec = $logicalRouterService.Help.create.logical_router.Create()
$lrSpec.display_name = "Foo"
$lrSpec.router_type = "TIER1"
$lrSpec.high_availability_mode = "ACTIVE_STANDBY"
$lrSpec.failover_mode = "FAILOVER_MODE_PREEMPTIVE"
$lrSpec.edge_cluster_id = "ea655f88-e184-47fe-9352-8ab9d8c955a7"
$lrAdd = $logicalRouterService.create($lrSpec)
However, when I ran the code, PowerCLI threw back a pretty generic and useless error which I am sure some of you have probably seen before "A server error occurred: 'com.vmware.vapi.std.errors.invalid.frequest': . Check $Error[0].Exception.ServerError". Even checking the error variable as stated in the error message did not yield any further details or specific to the issue I was running into.
Simliar to other vCenter Server REST APIs, as part of adding PowerCLI support, there are helper methods called "Create()" which help create the PowerShell objects required to pass to a specific operation. In my case, $logicalRouterService.Help.create.logical_router.Create() will create the PowerShell spec for creating a new Logical Router. You can also explore each of the properties within the object to get more information as the documentation of the API itself is built right into the cmdlet.
In the screenshot below, I am exploring the failover_mode property which PowerCLI is coming back stating that the possible values are either FAILOVER_MODE_PREEMPTIVE or FAILOVER_MODE_NON_PREEMPTIVE. However, as I had mentioned already when executing the code above, the generic error is thrown.
So how do we actually go about troubleshooting this?
Well, I ended logging into NSX-T Manager and tail'ing /var/log/vmware/nsx-manager.log and watch for my particular API request. Sure enough, within seconds I quickly found the actual error and the resolution was quite clear. It turns out I was using an invalid value for failover_mode property and the server also provided me with the correct value. It looks like the issue is a bug with the PowerCLI generated documentation as the NSX-T API docs do include the correct value. I normally do not rely on external documentation, especially as its built right into the cmdlets, but in this case, it may have been incorrectly generated. I have already filed a few bugs internally as this is not the only instance where the keywords are incorrect compared to the REST API documentation.
IMHO, this solution is more of a workaround. We should actually be propagating the server side error messages back to the client through the API. I believe the NSX-T API is doing the right thing but perhaps the NSX-T cmdlets are not passing back the information or they have not been design to include this. In any case, I definitely plan to share this with both our PowerCLI and NSX-T Product Managers 🙂
Thanks for the comment!