PR: Add support to edit remote files (Editor/Files)

[hlouzada]

Description of Changes

• Wrote at least one-line docstrings (for any new functions)
• Added unit test(s) covering the changes (if testable)

• Included a screenshot or animation (if affecting the UI, see Licecap)

• Created a Helper class RemoteFileHelper using AsyncDispatcher to interact with async api SpyderRemoteFileServicesAPI:
• keeps tracks of the connected APIs per server;
• centralize files operations (read, write);
• provide metadata;
• store remote files metadata with RemoteFileHandle that is stored in FileInfo;
• EditorMainWidget and EditorStack makes use of RemoteFileHelper to create RemoteFileHandle for IO operations and status updates;
• Allow ExplorerWidget and RemoteExplorer to open remote files.

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @hlouzada

[ccordoba12]

@dalthviz, please give this a thorough manual review to see if you find issues @hlouzada's work.

To open remote files in the editor you need to double click on a text file in the Files pane. Linting and code completion should be working for remote Python files too.

[dalthviz]

Gave this a check and seems to me that overall things are working 👍 There are a couple of things that require more work (although some are probably beyond the scope of this PR) but one thing that seems like would require changes here is the handling of a server disconnection/server being unavailable. A possible expected behavior when the connection gets closed is that the files related to that connection should be closed after a number of unsuccessful reconnection attempts. Maybe a similar dialog than the one is shown when a local file is deleted/removed from outside Spyder could be used 🤔. Anyhow, from my testing, when the connection with the server stops/server gets unavailable some errors are being raised:

• Stop server connection while a remote file is open and focus on the file:

An attempt to reconnect is done each time the file gains focus and although it can be successfull an error report dialog can also appear until the connection is stablished again:

Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2402, in focus_changed
    self.refresh()
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2657, in refresh
    self.__check_file_status(index)
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2493, in __check_file_status
    self._check_remote_file_status(index, finfo, name)
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2575, in _check_remote_file_status
    updated = helper.stat(handle)
              ^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\utils\remote.py", line 156, in stat
    info = self._info_runner(handle.client_id, handle.path.as_posix())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 266, in wrapper
    return task.result()
           ^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 678, in result
    return super().result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\concurrent\futures\_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\utils\remote.py", line 206, in _info_async
    return await api.info(PurePosixPath(posix_path))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\modules\file_services.py", line 389, in info
    return await response.json()
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\site-packages\aiohttp\client_reqrep.py", line 758, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 200, message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8', url='http://127.0.0.1:65509/login?next=/spyder-services/fs/info?path%3Dfile:///home/ubuntu/testing/testing.py'

• Lost server connection (stop server itself for example) while a file is open and focus on the file:

An attempt to reconnect is done each time the file gains focus. However, since the server is not available anymore, an error report dialog will keep appearing each time the remote file gains focus.

Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\manager\base.py", line 345, in create_new_connection
    if await self.__connection_task:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\manager\ssh.py", line 351, in _create_new_connection
    self._ssh_connection = await asyncssh.connect(
                           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\site-packages\asyncssh\connection.py", line 9188, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\site-packages\asyncssh\connection.py", line 516, in _connect
    _, session = await loop.create_connection(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\base_events.py", line 1132, in create_connection
    raise exceptions[0]
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\base_events.py", line 1107, in create_connection
    sock = await self._connect_sock(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\base_events.py", line 1010, in _connect_sock
    await self.sock_connect(sock, address)
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\proactor_events.py", line 729, in sock_connect
    return await self._proactor.connect(sock, address)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\windows_events.py", line 803, in _poll
    value = callback(transferred, key, ov)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\windows_events.py", line 599, in finish_connect
    ov.getresult()
ConnectionRefusedError: [WinError 1225] El equipo remoto rechazó la conexión de red
Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\utils\remote.py", line 222, in _is_writable_async
    remote_file = await api.open(posix_path, mode="ab")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\modules\file_services.py", line 488, in open
    await file.connect()
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\modules\file_services.py", line 127, in connect
    await super().connect()
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\modules\base.py", line 121, in connect
    raise RuntimeError("Failed to connect to Jupyter server")
RuntimeError: Failed to connect to Jupyter server

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2402, in focus_changed
    self.refresh()
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2656, in refresh
    self.__refresh_readonly(index)
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\widgets\editorstack\editorstack.py", line 2449, in __refresh_readonly
    read_only = not self._require_remote_helper().is_writable(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\utils\remote.py", line 166, in is_writable
    return self._is_writable_runner(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 266, in wrapper
    return task.result()
           ^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 678, in result
    return super().result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\concurrent\futures\_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\editor\utils\remote.py", line 228, in _is_writable_async
    await remote_file.close()
          ^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'remote_file' where it is not associated with a value

Besides those errors that trigger an error report dialog, I was able to trigger some error dialogs:

Over the debug output I can see the following traceback:

Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\manager\base.py", line 345, in create_new_connection
    if await self.__connection_task:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\manager\ssh.py", line 351, in _create_new_connection
    self._ssh_connection = await asyncssh.connect(
                           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\site-packages\asyncssh\connection.py", line 9188, in connect
    return await asyncio.wait_for(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\site-packages\asyncssh\connection.py", line 516, in _connect
    _, session = await loop.create_connection(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\base_events.py", line 1132, in create_connection
    raise exceptions[0]
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\base_events.py", line 1107, in create_connection
    sock = await self._connect_sock(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\base_events.py", line 1010, in _connect_sock
    await self.sock_connect(sock, address)
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\proactor_events.py", line 729, in sock_connect
    return await self._proactor.connect(sock, address)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\windows_events.py", line 803, in _poll
    value = callback(transferred, key, ov)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dalth\anaconda3\envs\spyder-dev\Lib\asyncio\windows_events.py", line 599, in finish_connect
    ov.getresult()
ConnectionRefusedError: [WinError 1225] El equipo remoto rechazó la conexión de red

In this last case, seems like the error output gets printed in the manage remote connections dialog (no error report dialog is shown). Not totally sure if that is expected so leaving a note for that.

Some other notes (maybe out of the scope of this PR, probably related actions should be disabled until logic to proper handle them is done?):

• The recent files menu entries doesn't include remote files.

• Trying to run/debug/profile from a remote file seems like works although no arrow indicating the debugging line is shown (and seems like breakpoints are ignored too). Also, in case you run/debug/profile in a console that is not in the remote server you can see a message related with the working directory not existing (maybe if a remote file is detected runs should always be done on a dedicated remote console? 🤔):

Local	Remote

• Trying to run code analysis doesn't do anything. It shows: