[Question] nodectl.exe security login failed to execut "transport: authentication handshake failed: context deadline exceeded"

[Willygap1572]

I am trying to start a AksEdge kubernetes cluster in a Windows Server 2022 with no connection to internet but I have some troubles with the nodectl.exe security login program call.

Full trace:

Install certificates

Get-ChildItem -Path Cert:\LocalMachine\Root | Where-Object {$_.Subject -like "CN=Microsoft Root Certificate Authority 2011"}
PSParentPath: Microsoft.PowerShell.Security\Certificate::LocalMachine\Root

Thumbprint Subject
8F43288AD272F3103B6FB1428485EA3014C0BCFE CN=Microsoft Root Certificate Authority 2011, O=Microsoft Corporation, L=R...

Get-ChildItem -Path Cert:\LocalMachine\CA | Where-Object {$_.Subject -like "CN=Microsoft Code Signing PCA 2011"}

PSParentPath: Microsoft.PowerShell.Security\Certificate::LocalMachine\CA

Thumbprint Subject
F252E794FE438E35ACE6E53762C0A234A2C52135 CN=Microsoft Code Signing PCA 2011, O=Microsoft Corporation, L=Redmond, S=...

---

Install AKSEdge K3

msiexec.exe /i AksEdge-K3s-1.26.6-1.5.203.0.msi /l*v InstallK3Log.txt

---

Install host features

Install-AksEdgeHostFeatures

Confirm
Are you sure you want to perform this action?
Performing the operation "Install the required features" on target "AksEdge Deployment".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): a

• Checking host for required features
• Checking the status of 'Microsoft-Hyper-V'
• Checking the status of 'Microsoft-Hyper-V-Management-PowerShell'
• Checking the status of 'VirtualMachinePlatform'
• Checking the status of 'OpenSSH.Client*'
• Checking power management settings of the Host
• Checking HNS version of the Host
• Checking OpenSSH version of the Host
• Checking Nested Virtualization of the Host
  True

---

SingleMachine deployment

New-AksEdgeConfig -DeploymentType SingleMachineCluster -outFile .\aksedge-config.json | Out-Null

Change Network.InternetDisable to true :

aksedge-config.json:

{
  "SchemaVersion": "1.9",
  "Version": "1.0",
  "DeploymentType": "SingleMachineCluster",
  "Init": {
    "ServiceIPRangeSize": 0
  },
  "Arc": {
    "ClusterName": null,
    "Location": null,
    "ResourceGroupName": null,
    "SubscriptionId": null,
    "TenantId": null,
    "ClientId": null,
    "ClientSecret": null
  },
  "Network": {
    "NetworkPlugin": "flannel",
    "Ip4AddressPrefix": null,
    "InternetDisabled": true,
    "SkipDnsCheck": false,
    "Proxy": {
      "Http": "http://proxy.com:8080",
      "Https": "http://proxy.com:8080",
      "No": "localhost,127.0.0.0/8,192.168.0.0/16,172.17.0.0/16,10.42.0.0/16,10.43.0.0/16,10.96.0.0/12,10.244.0.0/16,.svc"
    }
  },
  "User": {
    "AcceptEula": null,
    "AcceptOptionalTelemetry": null,
    "VolumeLicense": {
      "EnrollmentID": null,
      "PartNumber": null
    }
  },
  "Machines": [
    {
      "LinuxNode": {
        "CpuCount": 4,
        "MemoryInMB": 4096,
        "DataSizeInGB": 10,
        "LogSizeInGB": 1,
        "TimeoutSeconds": 300,
        "TpmPassthrough": false,
        "SecondaryNetworks": [
          {
            "VMSwitchName": null,
            "Ip4Address": null,
            "Ip4GatewayAddress": null,
            "Ip4PrefixLength": null
          }
        ]
      }
    }
  ]
}

New-AksEdgeDeployment -JsonConfigFilePath .\aksedge-config.json

• Checking host for required features
• Checking the status of 'Microsoft-Hyper-V'
• Checking the status of 'Microsoft-Hyper-V-Management-PowerShell'
• Checking the status of 'VirtualMachinePlatform'
• Checking the status of 'OpenSSH.Client*'
• Checking power management settings of the Host
• Checking HNS version of the Host
• Checking OpenSSH version of the Host
• Checking Nested Virtualization of the Host

[09/25/2024 10:16:15] All required host features are installed

[09/25/2024 10:16:15] Attention - Azure Arc properties are specified. These will not be used. The cluster currently needs to be connected in a separate step after deployment via Connect-AksEdgeArc
[09/25/2024 10:16:15] Validating AksEdge network parameters...

• Selecting private subnet in the '192.168' network segment...
• Identified candidate for private subnet: '192.168.0.0'. Validating subnet's gateway IP '192.168.0.1' is free...
• private subnet '192.168.0.0' is available

[09/25/2024 10:16:19] ***0 errors found in the deployment configuration.
[09/25/2024 10:16:39] Checking the required certificates for offline installation...

[09/25/2024 10:16:40] Verifying Host Requirements for Linux node(s)

• Verifying host requirements for selected configuration (19.5 GB disk size, 4596 MB memory, 4 CPUs)

[09/25/2024 10:16:40] Verifying Host OS can support requested configuration

[09/25/2024 10:16:40] Verifying required storage, RAM and number of cores are available

• Drive 'C:' has 40 GB free
• A minimum of 19 GB disk space is required on drive 'C:'
• Host has 23589 MB free memory
• A minimum of 4596 MB memory is required
• Host has 4 CPU cores
• A minimum of 4 CPU cores is required
• Verifying certificate requirements for AKS-EE
• Starting Internet Disabled Deployment

[09/25/2024 10:16:41] AksEdge - deploying a new Linux single machine k3s cluster

[09/25/2024 10:16:41] Creating single machine cluster vmms network

• Selecting private subnet in the '192.168' network segment...
• Identified candidate for private subnet: '192.168.0.0'. Validating subnet's gateway IP '192.168.0.1' is free...
• private subnet '192.168.0.0' is available
• Successfully selected private subnet '192.168.0.0'.
• AksEdge - private network carved:

Name : ip4GatewayAddress
Value : 192.168.0.1

Name : WindowsVmIp4Address
Value : 192.168.0.3

Name : LinuxVmIp4Address
Value : 192.168.0.2

Name : ip4Subnet
Value : 192.168.0.0

Name : ip4PrefixLength
Value : 24

[09/25/2024 10:16:46] Deploying AKS Edge Essentials - K3s
[09/25/2024 10:16:46] Step 1: Preparing host for AKS Edge Essentials - K3s
[09/25/2024 10:16:46] Enabling Microsoft Update. This will allow AKS Edge Essentials - K3s to receive updates.

• WARNING: Microsoft Update is not enabled. Please enable manually to ensure AKS Edge Essentials - K3s stays up to date.

[09/25/2024 10:16:47] Checking for virtual switch with name 'aksedgesw-int'

• The virtual switch 'aksedgesw-int' of type 'Internal' is present

[09/25/2024 10:16:47] Associating wssdagent service with nodectl

[09/25/2024 10:17:07] Exception Caught!!!

** - C:\Program Files\AksEdge\nodectl.exe security login --loginpath c:\programdata\wssdagent\nodelogin.yaml --identity failed to execute [Error: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"] (AksEdge-Core.psm1: line 5438)**

[09/25/2024 10:17:07] Collecting logs from deployment...

[09/25/2024 10:17:07] Collecting 'AKS Edge Essentials - K3s' configuration

[09/25/2024 10:17:07] Collecting 'AKS Edge Essentials - K3s' deployment configuration

[09/25/2024 10:17:07] Collecting 'AKS Edge Essentials - K3s' event logs
[09/25/2024 10:17:07] Collecting wssdagent configuration [09/25/2024 10:17:07] Collecting wssdagent logs
[09/25/2024 10:17:07] Collecting node logs

[09/25/2024 10:17:28] Exception Caught!!!

** - C:\Program Files\AksEdge\nodectl.exe compute vm list -o tsv --query "[*].name" failed to execute [Error: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"] (AksEdge-Core.psm1: line 5438)**

[09/25/2024 10:17:28] Compressing logs
[09/25/2024 10:17:28] Zip file is located at "C:\ProgramData\AksEdge\logs\aksedgelogs-240925-1017.zip"
C:\ProgramData\AksEdge\logs\aksedgelogs-240925-1017.zip
[09/25/2024 10:17:28] Attempting to remove vmms single machine cluster network

[09/25/2024 10:17:48] Exception Caught!!!

- C:\Program Files\AksEdge\nodectl.exe network vnet show --name "aksedgesw-int" failed to execute [Error: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"] (AksEdge-Core.psm1: line 5438)

• Cleaning up single machine cluster NAT object 'aksedge_NAT' ...
• Cleaning up single machine cluster virtual switch 'aksedgesw-int' ...

nodelogin.yaml: (token not expired)

name: Admin
token: <TOKEN>
certificate: <CERTIFICATE>
clienttype: ""
cloudfqdn: ""
cloudport: 0
cloudauthport: 0
cacerthash: ""
location: ""
type: ""

[scholz]

Hi @Willygap1572 , since i've been working a lot with AKSEE lately i had a look to see if i could offer some help. But I have never encountered the error you show in my installations before (& have installed in all envs so far: online, offline, proxied but OS only win10 ltsc iot).

Can you maybe try to update to a new version of AKSEE first? The version you are referring to is nearly 1yr old and the team has made some significant bug fixes / improvements in the meantime.

BTW: In the latest version version of AKSEE (1.8.202.0), which I am running in a VM (vSphere) behind a corporate proxy and is successfully connected to Azure, the nodectl network vnet show --name "aksedgesw-int" also fails (manually executed) but it does not seem to hurt the installation.