Summary: A dell server/appliance is showing missing in the physical config view. PowerTool.log outputs "bind: Address already in use"


Article Content

Issue


A Dell server/appliance node is missing in the physical config display, but is fine as far as vCenter is concerned.

The connectors-cluster.log will have entries similar to this:

DellPTAdapterPool.getDellPTAdapter:56 - Fail to initialize the PT Agent adapter for host: hostnamegoeshere.com
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'DellPTAdapter': Invocation of init method failed; nested exception is java.lang.IllegalStateException: Fail to get any response from PT agent on host: esxi01.xxxxx.yyyy. Please check the status of PT Agent.

You will also see this in the same log: Fail to get any response from PT agent on host: hostnamegoeshere.com. Please check the status of PT Agent.


The following log on the server:

/var/run/log/PowerTools.log.

Shows this error message:
ERROR - bind: Address already in use


"esxcli network ip connection list | grep LISTEN | grep Dell" does not show any output
[root@esxivxpwr1-1:~] esxcli network ip connection list | grep LISTEN | grep Dell
[root@esxivxpwr1-1:~]


"esxcli network ip connection list | grep LISTEN | grep 8086" shows outputs like below, 
(No DellPTAgent at the World Name column)

[root@esxivxpwr1-1:~] esxcli network ip connection list | grep LISTEN | grep 8086
tcp         0       0  192.168.21.11:8086               0.0.0.0:0            LISTEN        47865086  newreno       [root@esxivxpwr1-1:~]

And there is no such a process by ps command,
[root@esxivxpwr1-1:~] ps -cjJ | grep 47865086 | grep -v grep
[root@esxivxpwr1-1:~]
Cause
This is most likely caused by another service or staled process using the same port (8086) or the service did not stop correctly.
Resolution
1. Restart iSM & PTAgent, and then reset iDRAC
# /etc/init.d/dcism-netmon-watchdog restart
# /etc/init.d/DellPTAgent restart
# /opt/dell/DellPTAgent/bin/ipmitool_static mc reset cold

*** if you cannot find /opt/dell/DellPTAgent/bin/ipmitool_static, please search other path

  • /opt/vxrail/tools/ipmitool
  • /scratch/dell/DellPTAgent/bin/ipmitool_static (for old PTAgent versions)
 
*** VxRail: How can I reset iDRAC and restart PTagent ?
KB#511713 (Dell EMC Level 40 internal article - visible for internal users only)

Check if it's resolved by "esxcli network ip connection list | grep LISTEN | grep 8086"
2. If still problematic, perform following action.
Put the node in maintenance mode and reboot it.
After it comes back restart the VxRail manager VM.
Wait 5 to 10 minutes and check for the node status.