Skip to content

Ssh command deadlock with a bad host connectionΒ #688

@aniknaemmm

Description

@aniknaemmm
using (var command = client.CreateCommand(CommandText))
{
     command.CommandTimeout = TimeSpan.FromMinutes(2);
     command.Execute();
}

I have problems with this code. Sometimes a thread with this part of the code may hang. I began to research this problem and found that the problem happens when I had a bad connection with the host (about 60-70 pocket loss).

Deep in the library code, I found this method with infinite wait time:

 private static int TrySocketRead(Socket socket, byte[] buffer, int offset, int length)
 {
      return SocketAbstraction.Read(socket, buffer, offset, length, InfiniteTimeSpan);
  }

And inside of this method in SocketAbstraction.cs

public static int Read(Socket socket, byte[] buffer, int offset, int size, TimeSpan timeout)
{
    var totalBytesRead = 0;
    var totalBytesToRead = size;
    socket.ReceiveTimeout = (int) timeout.TotalMilliseconds;
    do
    {
        try
        {
            var bytesRead = socket.Receive(buffer, offset + totalBytesRead, totalBytesToRead - totalBytesRead, SocketFlags.None);
            if (bytesRead == 0)
                return 0;
            totalBytesRead += bytesRead;
        }

We will have socket.Recive() call with infinity timeout.

Therefore, in case of a bad connection, we will hang in the Receive method, and in another thread, we will call Dispose of SshCommand and hang when _socketReadLock is locked because this lock object is used when we call TrySocketRead in Sesseion.cs

private bool IsSocketConnected()
 {
    lock (_socketDisposeLock)
    {
#if FEATURE_SOCKET_POLL
      if (!_socket.IsConnected())
      {
          return false;
      }
      lock (_socketReadLock)
      {
         var connectionClosedOrDataAvailable = _socket.Poll(0, SelectMode.SelectRead);
         return !(connectionClosedOrDataAvailable && _socket.Available == 0);
      }
#else
        return _socket.IsConnected();
#endif // FEATURE_SOCKET_POLL
    }
}

I found 2 workarounds for this case:

  1. We can set a timeout for this method from Session.cs
 private static int TrySocketRead(Socket socket, byte[] buffer, int offset, int length)
 {
     return SocketAbstraction.Read(socket, buffer, offset, length, InfiniteTimeSpan);
  }
  1. We can disable FEATURE_SOCKET_POLL

What do you think about this?
Can you explain why FEATURE_SOCKET_POLL is necessary or what is bad if disable FEATURE_SOCKET_POLL?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions