Couldn’t connect to localhost at port 9735 on HMC

Couldn't connect to localhost at port 9735.

*  When i going to access my Aix LPAR throuch HMC Console on Putty.

*  I got Couldn’t connect to localhost at port 9735 error.

*  Frame connected to two HMC which is having latest same HMC Version.

* From one of the HMC i can access my LPAR’s without any problem. DLPAR operation also working fine.

Error Message:

Enter Number of Running Partition (q to quit): 10
    Opening Virtual Terminal On Partition TEST-LPAR . . .
connect: Connection refused
Couldn't connect to localhost at port 9735.

    Hit <Enter> To Continue, q to quit: q

Problem Identification:

Step 1: Need to check both HMC firewall network ports.

From linux desktop we can verify what are the network ports are opened using nmap utility.

bash-4.2# nmap -p "*" 10.10.1.101 #----> HMC 1
bash-4.2# nmap -p "*" 10.10.1.102 #----> HMC 2

Manually verified both HMC all required ports already opened. both are having same configurations.

Step 2: One of the HMC DLPAR Operation working, another HMC DLPAR operations not working.

Step 3: Login to any of the aix LPAR which is don’t have any HMC and DLPAR issue and verify RSCT Connection output.

Properly Running Server RSCT Connection Details:

bash-4.2# lsrsrc IBM.MCP
Resource Persistent Attributes for IBM.MCP
resource 1:
        MNName            = "192.168.5.100"
        NodeID            = 6902741367313452150
        KeyToken          = "PRODHMC1"
        IPAddresses       = {"10.10.1.25"}
        ConnectivityNames = {"192.168.5.100"}
        HMCName           = "7041CA6*8402CCD"
        HMCIPAddr         = "10.10.1.45"
        HMCAddIPs         = "10.10.50.20"
        HMCAddIPv6s       = ""
        ActivePeerDomain  = ""
        NodeNameList      = {"PRODVIOSERVER"}
resource 2:
        MNName            = "192.168.5.100"
        NodeID            = 8161732012622481528
        KeyToken          = "PRODHMC2"
        IPAddresses       = {"10.10.1.22"}
        ConnectivityNames = {"192.168.5.100"}
        HMCName           = "7041CA7*8402EED"
        HMCIPAddr         = "10.10.1.92"
        HMCAddIPs         = "10.10.50.21"
        HMCAddIPv6s       = ""
        ActivePeerDomain  = ""
        NodeNameList      = {"PRODVIOSERVER"}

Problematic Server RSCT Connection Details:

bash-4.2# lsrsrc IBM.MCP
Resource Persistent Attributes for IBM.MCP
resource 1:
        MNName            = "192.168.5.246"
        NodeID            = 2529692024237623612
        KeyToken          = "localhost"
        IPAddresses       = {"10.10.1.101"}
        ConnectivityNames = {"192.168.5.246"}
        HMCName           = "7023AE6*064CCLC"
        HMCIPAddr         = "10.10.1.101"
        HMCAddIPs         = "10.10.50.17"
        HMCAddIPv6s       = ""
        ActivePeerDomain  = ""
        NodeNameList      = {"TEST-LPAR"}

Step 4: Verify problematic both HMC’s Network details. login to both hmc through putty.

Run “lshmc -n ” command. HMC output as per below.

HMC 1 output : 10.10.1.101

hscroot@localhost:~> lshmc -n
hostname=localhost,domain=localdomain,ipaddr=10.10.1.101,

HMC 2 output : 10.10.1.102

hscroot@localhost:~> lshmc -n
hostname=localhost,domain=localdomain,ipaddr=10.10.1.102,

Note :

* Here i identified one of the issue. both the HMC having same host name.

* That is the reason we are facing port 9735 error and DLPAR arror on one of the HMC.

* Why because RSCT is a cluster daemon. it won’t allow same HMC’s host name with in the  LPAR.

Solution for the Problem:

* Change host name of one of the problematic HMC and take reboot.

* Again login to any one of the server from HMC vtmenu console it will work. HMC vtmenu port 9735 error resolved.

Enter Number of Running Partition (q to quit): 12 
 
    Opening Virtual Terminal On Partition VIOSERVER_02 . . .

 Open in progress

 Open Completed.
  IBM Virtual I/O Server
                        login:

Result :

Couldn’t connect to localhost at port 9735 on HMC problem has been successfully resolved.

Leave a Comment