When I was trying to use VSCode to connect discovery cluster. It would be find in the first 10 minutes, but ssh connect in VSCode will definitely fail every 10 minutes.
Here is the log in VSCode ssh. I cannot appreciate it more anyone could answer my question.
The connection was built at 18:00, but it disconnected at 18:10.
[18:00:43.227] [Forwarding server 3853] Got connection 0
[18:00:43.238] ------
[18:00:43.267] [Forwarding server 3853] Got connection 1
[18:00:43.332] [Forwarding server 3853] Got connection 2
[18:00:44.569] [Forwarding server 3853] Got connection 3
[18:10:08.767] ------
[18:10:08.768] SSH Resolver called for "ssh-remote+discovery2.usc.edu", attempt 2, (Reconnection)
[18:10:08.769] [Forwarding server 3853] Got connection 4
[18:10:08.771] SSH Resolver called for host: discovery2.usc.edu
[18:10:08.771] Setting up SSH remote "discovery2.usc.edu"
[18:10:08.774] Acquiring local install lock: C:\Users\zjin\AppData\Local\Temp\vscode-remote-ssh-05328798-install.lock
[18:10:08.775] Looking for existing server data file at c:\Users\zjin\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-ssh\vscode-ssh-host-05328798-695af097c7bd098fbf017ce3ac85e09bbc5dda06-0.102.0\data.json
[18:10:08.776] Found local server running: {"remoteListeningOn":{"port":36908},"osReleaseId":"centos","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","display":"","tmpDir":"/run/user/602602","platform":"linux","connectionToken":"1a1a1a1a-11a1-11a1-11aa-11aa111111aa","pid":25932,"ipcHandlePath":"\\\\.\\pipe\\vscode-ssh-askpass-b456c0c4d5d261e460543dc337c8213e6677266f-sock","socksPort":3850,"startupTime":1687136440645}
[18:10:08.776] Running server is stale. Ignoring
[18:10:08.777] Using commit id "695af097c7bd098fbf017ce3ac85e09bbc5dda06" and quality "stable" for server
[18:10:08.781] Install and start server if needed
[18:10:08.806] Failed to set up socket for dynamic port forward to remote port 36908: Socket closed. Is the remote port correct?
[18:10:08.809] Using SSH config file "C:\Users\zjin\.ssh\config"
[18:10:08.810] askpass server listening on \\.\pipe\vscode-ssh-askpass-67c41fc80b1f49b10296442b58d9618ab66e0491-sock
[18:10:08.810] Spawning local server with {"serverId":2,"ipcHandlePath":"\\\\.\\pipe\\vscode-ssh-askpass-d60e44ed22c77e7456a49ac46ac2541df71e7bff-sock","sshCommand":"C:\\Softwares\\Git\\Git\\usr\\bin\\ssh.exe","sshArgs":["-v","-T","-D","4104","-F","C:\\Users\\zjin\\.ssh\\config","discovery2.usc.edu","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"c:\\Users\\zjin\\AppData\\Roaming\\Code\\User\\globalStorage\\ms-vscode-remote.remote-ssh\\vscode-ssh-host-05328798-695af097c7bd098fbf017ce3ac85e09bbc5dda06-0.102.0\\data.json"}
[18:10:08.811] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"c:\\Users\\zjin\\.vscode\\extensions\\ms-vscode-remote.remote-ssh-0.102.0\\out\\local-server\\askpass.bat","VSCODE_SSH_ASKPASS_NODE":"C:\\Softwares\\VSCode\\Microsoft VS Code\\Code.exe","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"--ms-enable-electron-run-as-node","VSCODE_SSH_ASKPASS_MAIN":"c:\\Users\\zjin\\.vscode\\extensions\\ms-vscode-remote.remote-ssh-0.102.0\\out\\askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"\\\\.\\pipe\\vscode-ssh-askpass-67c41fc80b1f49b10296442b58d9618ab66e0491-sock"}
[18:10:08.825] Spawned 10660
[18:10:08.927] > local-server-2> Running ssh connection command: "-v -T -D 4104 -F C:\\Users\\zjin\\.ssh\\config discovery2.usc.edu bash"
[18:10:08.935] > local-server-2> Spawned ssh, pid=26564
[18:10:09.060] stderr> OpenSSH_9.3p1, OpenSSL 1.1.1u 30 May 2023
[18:10:09.276] stderr> debug1: Server host key: ecdsa-sha2-nistp256 SHA256:pO0FfPYahmoh/8UcgHpLKxdJe52fNuX9bCodsHI4rVE
[18:10:09.463] stderr> Authenticated to discovery2.usc.edu ([10.72.0.14]:22) using "publickey".
[18:10:12.340] > local-server-1> Timed out
[18:10:12.370] Local server exit: 0
[18:10:25.835] Terminating local server
[18:10:25.839] Resolver error: Error: Connecting with SSH timed out
at m.Timeout (c:\Users\zjin\.vscode\extensions\ms-vscode-remote.remote-ssh-0.102.0\out\extension.js:1:584271)
at Timeout._onTimeout (c:\Users\zjin\.vscode\extensions\ms-vscode-remote.remote-ssh-0.102.0\out\extension.js:1:630486)
at listOnTimeout (node:internal/timers:559:17)
at process.processTimers (node:internal/timers:502:7)
[18:10:25.845] ------
[18:10:25.849] Local server exit: null
[18:10:30.887] ------
[18:10:30.887] SSH Resolver called for "ssh-remote+discovery2.usc.edu", attempt 3, (Reconnection)
[18:10:30.889] SSH Resolver called for host: discovery2.usc.edu
[18:10:30.889] Setting up SSH remote "discovery2.usc.edu"
[18:10:30.890] Acquiring local install lock: C:\Users\zjin\AppData\Local\Temp\vscode-remote-ssh-05328798-install.lock
[18:10:30.894] Looking for existing server data file at c:\Users\zjin\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-ssh\vscode-ssh-host-05328798-695af097c7bd098fbf017ce3ac85e09bbc5dda06-0.102.0\data.json
[18:10:30.894] Using commit id "695af097c7bd098fbf017ce3ac85e09bbc5dda06" and quality "stable" for server
[18:10:30.898] Install and start server if needed
[18:10:30.903] Using SSH config file "C:\Users\zjin\.ssh\config"
[18:10:30.903] askpass server listening on \\.\pipe\vscode-ssh-askpass-fe4f01f771cc72abbe31f2ba74b3eb7f9b137529-sock
[18:10:30.903] Spawning local server with {"serverId":3,"ipcHandlePath":"\\\\.\\pipe\\vscode-ssh-askpass-ae34101a4bc9ccc8c4e8bf14da787fcdaf1d5d78-sock","sshCommand":"C:\\Softwares\\Git\\Git\\usr\\bin\\ssh.exe","sshArgs":["-v","-T","-D","4117","-F","C:\\Users\\zjin\\.ssh\\config","discovery2.usc.edu","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"c:\\Users\\zjin\\AppData\\Roaming\\Code\\User\\globalStorage\\ms-vscode-remote.remote-ssh\\vscode-ssh-host-05328798-695af097c7bd098fbf017ce3ac85e09bbc5dda06-0.102.0\\data.json"}
[18:10:30.904] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"c:\\Users\\zjin\\.vscode\\extensions\\ms-vscode-remote.remote-ssh-0.102.0\\out\\local-server\\askpass.bat","VSCODE_SSH_ASKPASS_NODE":"C:\\Softwares\\VSCode\\Microsoft VS Code\\Code.exe","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"--ms-enable-electron-run-as-node","VSCODE_SSH_ASKPASS_MAIN":"c:\\Users\\zjin\\.vscode\\extensions\\ms-vscode-remote.remote-ssh-0.102.0\\out\\askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"\\\\.\\pipe\\vscode-ssh-askpass-fe4f01f771cc72abbe31f2ba74b3eb7f9b137529-sock"}
[18:10:30.916] Spawned 5936
[18:10:31.015] > local-server-3> Running ssh connection command: "-v -T -D 4117 -F C:\\Users\\zjin\\.ssh\\config discovery2.usc.edu bash"
[18:10:31.022] > local-server-3> Spawned ssh, pid=13224
[18:10:31.089] stderr> OpenSSH_9.3p1, OpenSSL 1.1.1u 30 May 2023
[18:10:31.301] stderr> debug1: Server host key: ecdsa-sha2-nistp256 SHA256:pO0FfPYahmoh/8UcgHpLKxdJe52fNuX9bCodsHI4rVE
[18:10:31.488] stderr> Authenticated to discovery2.usc.edu ([10.72.0.14]:22) using "publickey".
[18:10:35.691] stderr> Lmod has detected the following error: Unable to load module because of error
[18:10:35.691] stderr> when evaluating modulefile:
[18:10:35.691] stderr> /spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0/tmux/3.3a.lua:
[18:10:35.691] stderr> [string "-- -*- lua -*-..."]:28: attempt to concatenate a nil value (local
[18:10:35.692] stderr> 'hostname')
[18:10:35.692] stderr> Please check the modulefile and especially if there is a the line number
[18:10:35.692] stderr> specified in the above message
[18:10:35.692] stderr> While processing the following module(s):
[18:10:35.692] stderr> Module fullname Module Filename
[18:10:35.692] stderr> --------------- ---------------
[18:10:35.693] stderr> tmux/3.3a /spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0/tmux/3.3a.lua
[18:10:35.693] stderr>
[18:10:36.060] > ready: a07d847cb150
[18:10:36.101] > Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020
[18:10:36.102] Platform: linux
[18:10:36.131] > /bin/bash
[18:10:36.131] Shell: bash
[18:10:36.131] Creating bash subshell inside "bash"
[18:10:36.164] > bash version: 4.2.46(2)-release
[18:10:36.348] > a07d847cb150: running
[18:10:36.365] > Acquiring lock on /home1/zjin8285/.vscode-server/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06/vscode-remote-lock.zjin8285.695af097c7bd098fbf017ce3ac85e09bbc5dda06
[18:10:36.369] > Found existing installation at /home1/zjin8285/.vscode-server/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06...
> Checking /home1/zjin8285/.vscode-server/.695af097c7bd098fbf017ce3ac85e09bbc5dda06.log and /home1/zjin8285/.vscode-server/.695af097c7bd098fbf017ce3ac85e09bbc5dda06.pid for a running server
> Looking for server with pid: 52757
[18:10:36.512] > Running ssh connection command... /home1/zjin8285/.vscode-server/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06/bin/code-server --start-server --host=127.0.0.1 --accept-server-license-terms --enable-remote-auto-shutdown --port=0 --telemetry-level all &> "/home1/zjin8285/.vscode-server/.695af097c7bd098fbf017ce3ac85e09bbc5dda06.log" < /dev/null
> printenv:
[18:10:36.516] > LMOD_FAMILY_COMPILER_VERSION=11.3.0
> CONDA_AUTO_ACTIVATE_BASE=false
> MANPATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r/share/man:/spack/apps/gcc/11.3.0/share/man:/usr/share/man
> XDG_SESSION_ID=151434
> _ModuleTable003_=Y2svYXBwcy9sbW9kL2xpbnV4LWNlbnRvczcteDg2XzY0L0NvcmUiLH0sWyJzeXN0ZW1CYXNlTVBBVEgiXT0iL3NwYWNrL2FwcHMvbG1vZC9saW51eC1jZW50b3M3LXg4Nl82NC9Db3JlIix9
> ZSH_ROOT=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r
> __LMOD_REF_COUNT_MODULEPATH=/spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0:1;/spack/apps/lmod/linux-centos7-x86_64/Core:1
> SHELL=/bin/bash
> LMOD_ROOT=/spack/apps/linux-centos7-x86_64/gcc-4.8.5/lmod-8.2-qzzv4g6g7wfwzl4aoqzz6fgqaoszbd65/lmod
> SSH_CLIENT=10.49.131.31 4118 22
> CONDA_SHLVL=0
> LMOD_PKG=/spack/apps/linux-centos7-x86_64/gcc-4.8.5/lmod-8.2-qzzv4g6g7wfwzl4aoqzz6fgqaoszbd65/lmod/lmod
> LMOD_VERSION=8.2
> GIT_ROOT=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/git-2.36.1-ctkif3c
> __LMOD_REF_COUNT_LOADEDMODULES=gcc/11.3.0:1;zsh/5.9:1;git/2.36.1:1
> __LMOD_REF_COUNT_CMAKE_PREFIX_PATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/git-2.36.1-ctkif3c:1;/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r:1;/spack/apps/gcc/11.3.0:1
> USER=zjin8285
> LD_LIBRARY_PATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r/lib:/spack/apps/gcc/11.3.0/lib64
> CONDA_EXE=/home1/zjin8285/miniconda3/bin/conda
> __LMOD_REF_COUNT__LMFILES_=/spack/apps/lmod/linux-centos7-x86_64/Core/gcc/11.3.0.lua:1;/spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0/zsh/5.9.lua:1;/spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0/git/2.36.1.lua:1
> VSCODE_AGENT_FOLDER=/home1/zjin8285/.vscode-server
> _CE_CONDA=
> _ModuleTable001_=X01vZHVsZVRhYmxlXz17WyJNVHZlcnNpb24iXT0zLFsiY19yZWJ1aWxkVGltZSJdPWZhbHNlLFsiY19zaG9ydFRpbWUiXT1mYWxzZSxkZXB0aFQ9e30sZmFtaWx5PXtbImNvbXBpbGVyIl09ImdjYyIsfSxtVD17Z2NjPXtbImZuIl09Ii9zcGFjay9hcHBzL2xtb2QvbGludXgtY2VudG9zNy14ODZfNjQvQ29yZS9nY2MvMTEuMy4wLmx1YSIsWyJmdWxsTmFtZSJdPSJnY2MvMTEuMy4wIixbImxvYWRPcmRlciJdPTEscHJvcFQ9e30sWyJzdGFja0RlcHRoIl09MCxbInN0YXR1cyJdPSJhY3RpdmUiLFsidXNlck5hbWUiXT0iZ2NjLzExLjMuMCIsfSxnaXQ9e1siZm4iXT0iL3NwYWNrLzIyMDYvYXBwcy9sbW9kL2xpbnV4LWNlbnRvczcteDg2XzY0L2djYy8xMS4zLjAvZ2l0LzIuMzYu
> MAIL=/var/mail/zjin8285
> PATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/git-2.36.1-ctkif3c/bin:/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r/bin:/spack/apps/gcc/11.3.0/bin:/home1/zjin8285/miniconda3/condabin:/spack/utilities:/usr/local/bin:/usr/bin
> PWD=/home1/zjin8285
> _LMFILES_=/spack/apps/lmod/linux-centos7-x86_64/Core/gcc/11.3.0.lua:/spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0/zsh/5.9.lua:/spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0/git/2.36.1.lua
> MODULEPATH=/spack/2206/apps/lmod/linux-centos7-x86_64/gcc/11.3.0:/spack/apps/lmod/linux-centos7-x86_64/Core
> LOADEDMODULES=gcc/11.3.0:zsh/5.9:git/2.36.1
> _ModuleTable_Sz_=3
> LMOD_CMD=/spack/apps/linux-centos7-x86_64/gcc-4.8.5/lmod-8.2-qzzv4g6g7wfwzl4aoqzz6fgqaoszbd65/lmod/lmod/libexec/lmod
> _CE_M=
> GCC_ROOT=/spack/apps/gcc/11.3.0
> SHLVL=3
> HOME=/home1/zjin8285
> __LMOD_REF_COUNT_PATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/git-2.36.1-ctkif3c/bin:1;/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r/bin:1;/spack/apps/gcc/11.3.0/bin:1;/home1/zjin8285/miniconda3/condabin:1;/spack/utilities:1;/usr/local/bin:1;/usr/bin:1
> _ModuleTable002_=MS5sdWEiLFsiZnVsbE5hbWUiXT0iZ2l0LzIuMzYuMSIsWyJsb2FkT3JkZXIiXT0zLHByb3BUPXt9LFsic3RhY2tEZXB0aCJdPTAsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImdpdC8yLjM2LjEiLH0senNoPXtbImZuIl09Ii9zcGFjay8yMjA2L2FwcHMvbG1vZC9saW51eC1jZW50b3M3LXg4Nl82NC9nY2MvMTEuMy4wL3pzaC81LjkubHVhIixbImZ1bGxOYW1lIl09InpzaC81LjkiLFsibG9hZE9yZGVyIl09Mixwcm9wVD17fSxbInN0YWNrRGVwdGgiXT0wLFsic3RhdHVzIl09ImFjdGl2ZSIsWyJ1c2VyTmFtZSJdPSJ6c2gvNS45Iix9LH0sbXBhdGhBPXsiL3NwYWNrLzIyMDYvYXBwcy9sbW9kL2xpbnV4LWNlbnRvczcteDg2XzY0L2djYy8xMS4zLjAiLCIvc3Bh
> CONDA_PYTHON_EXE=/home1/zjin8285/miniconda3/bin/python
> LOGNAME=zjin8285
> XDG_DATA_DIRS=/home1/zjin8285/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
> SSH_CONNECTION=10.49.131.31 4118 10.72.0.14 22
> MODULESHOME=/spack/apps/linux-centos7-x86_64/gcc-4.8.5/lmod-8.2-qzzv4g6g7wfwzl4aoqzz6fgqaoszbd65/lmod/lmod
> __LMOD_REF_COUNT_LD_LIBRARY_PATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r/lib:1;/spack/apps/gcc/11.3.0/lib64:1
> LMOD_SETTARG_FULL_SUPPORT=no
> LESSOPEN=||/usr/bin/lesspipe.sh %s
> LMOD_FAMILY_COMPILER=gcc
> CMAKE_PREFIX_PATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/git-2.36.1-ctkif3c:/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r:/spack/apps/gcc/11.3.0
> XDG_RUNTIME_DIR=/run/user/602602
> __LMOD_REF_COUNT_MANPATH=/spack/2206/apps/linux-centos7-x86_64_v3/gcc-11.3.0/zsh-5.9-eepz55r/share/man:1;/spack/apps/gcc/11.3.0/share/man:1;/usr/share/man:1
> LMOD_DIR=/spack/apps/linux-centos7-x86_64/gcc-4.8.5/lmod-8.2-qzzv4g6g7wfwzl4aoqzz6fgqaoszbd65/lmod/lmod/libexec
> BASH_FUNC_module()=() { eval $($LMOD_CMD bash "$@") && eval $(${LMOD_SETTARG_CMD:-:} -s sh)
> }
> BASH_FUNC_ml()=() { eval $($LMOD_DIR/ml_cmd "$@")
> }
> _=/usr/bin/printenv
[18:10:36.519] > Removing old logfile at /home1/zjin8285/.vscode-server/.695af097c7bd098fbf017ce3ac85e09bbc5dda06.log
[18:10:36.521] > Spawned remote server: 63510
[18:10:36.522] > Waiting for server log...
[18:10:36.557] > Waiting for server log...
[18:10:36.592] >
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>
[18:10:36.596] > a07d847cb150: start
> SSH_AUTH_SOCK====
> DISPLAY====
> webUiAccessToken====
> listeningOn==46618==
> osReleaseId==centos==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/run/user/602602==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime==78==
> connectionToken==111a1aa1-a111-1aa1-1a11-a1a111a1111a==
> a07d847cb150: end
[18:10:36.596] Received install output:
SSH_AUTH_SOCK====
DISPLAY====
webUiAccessToken====
listeningOn==46618==
osReleaseId==centos==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/run/user/602602==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime==78==
connectionToken==111a1aa1-a111-1aa1-1a11-a1a111a1111a==
[18:10:36.597] Remote server is listening on 46618
[18:10:36.597] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":46618},"osReleaseId":"centos","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","display":"","tmpDir":"/run/user/602602","platform":"linux","connectionToken":"111a1aa1-a111-1aa1-1a11-a1a111a1111a"},"serverStartTime":78,"installUnpackCode":""}
[18:10:36.598] Persisting server connection details to c:\Users\zjin\AppData\Roaming\Code\User\globalStorage\ms-vscode-remote.remote-ssh\vscode-ssh-host-05328798-695af097c7bd098fbf017ce3ac85e09bbc5dda06-0.102.0\data.json
[18:10:36.600] Starting forwarding server. localPort 4121 -> socksPort 4117 -> remotePort 46618
[18:10:36.601] Forwarding server listening on 4121
[18:10:36.601] Waiting for ssh tunnel to be ready
[18:10:36.602] Tunneled 46618 to local port 4121
[18:10:36.603] Resolved "ssh-remote+discovery2.usc.edu" to "127.0.0.1:4121"
[18:10:36.603] [Forwarding server 4121] Got connection 0
[18:10:36.606] ------
[18:10:36.613] [Forwarding server 4121] Got connection 1