how to handle a timeout issue with retries

[MCO-SOCOMEC]

Hi,
I'm using pyxcp to communicate with an embedded target via TCP. Everything works fine after a point my target decides to reply in delay and I get a timeout (socket.timeout: timed out).
I would like to put in place a retry mechanism. I took my functio

def read_characteristic(self,
characteristic2read=string) -> int:
"""
It reads a characteristic based on the address found in the a2l file

    Parameters
    ----------
    characteristic2read : _type_, optional
        _description_, by default string

    Returns
    -------
    _type_
        _description_
    """

    with self.ap.run() as xcp_connection:
        try:
            self.__connect()
            xcp_connection.connect()
            data = self.__read_characteristic(
                xcp_connection, self.a2ldb, characteristic2read)
            xcp_connection.disconnect()
            self.__disconnect()
        except Exception as e:
            self._logger.error(msg=f"Error reading characteristic: {e}")
            data = None

    return data

and I modified it in this way but it doens't work.

def read_characteristic(self,
characteristic2read=string) -> int:
"""
It reads a characteristic based on the address found in the a2l file

    Parameters
    ----------
    characteristic2read : _type_, optional
        _description_, by default string

    Returns
    -------
    _type_
        _description_
    """

    retries = self.COMM_RETRIES + 1

    while retries > 0:

        try:
            with self.ap.run() as xcp_connection:
                try:
                    self.__connect()
                    xcp_connection.connect()
                    data = self.__read_characteristic(
                        xcp_connection, self.a2ldb, characteristic2read)
                    xcp_connection.disconnect()
                    self.__disconnect()
                except Exception as e:
                    self._logger.error(
                        msg=f"Error reading characteristic: {e}")
                    data = None
        except Exception as e:
            print("Error initializing XCP connection:", e)
            data = None

        if data is not None:
            break

        self._logger.debug(("Retry on ExceptionResponse ({e}) - "
                            "{retries}"))

        if self.COMM_BACKOFF:
            delay = 2 ** (self.COMM_RETRIES - retries) * self.COMM_BACKOFF
            sleep(delay)
            self._logger.debug("Sleeping {}".format(delay))

        retries = retries - 1

    return data

Any suggestion?
Thanks
Michele

[MCO-SOCOMEC]

sorry, I forgot that self.ap is defined at init as

self.ap = ArgumentParser(description=source)

[MCO-SOCOMEC]

Sorry Cristoph to bother you. Do you have any suggestion?
thanks a lot
Michele

[christoph2]

@MCO-SOCOMEC
Hello,
Have you tried the c.Transport.timeout parameter in your config file?
It's also very useful to use the command line option -l DEBUG to see what exactly causes the problems.

[MCO-SOCOMEC]

Hello Chrisoph, no I didn't try the parameter. I will try and come back to you.
thanks a lot for the supprot

[MCO-SOCOMEC]

Hello,
I tried increasing the parameter in the configuration file but I still get a retransmission.

[PyXCP] Execute(Master.disconnect -- Arguments(ARGS = (), KWS = {}))
[PyXCP] DISCONNECT
[PyXCP] -> [01 00 43 00 fe]
[PyXCP] <- L1 C11351 [ff]
[PyXCP] XCPonEth - Connected to: 172.17.1.72:17725 local address: 172.17.10.84:59940 [AF_INET][SOCK_STREAM]
[PyXCP] Execute(Master.connect -- Arguments(ARGS = (), KWS = {}))
[PyXCP] CONNECT
[PyXCP] -> [02 00 00 00 ff 00]
[PyXCP] <- L8 C11352 [ff 05 00 ff fc ff 01 01]
[PyXCP] Execute(Master.setMta -- Arguments(ARGS = (6995372,), KWS = {}))
[PyXCP] SET_MTA
[PyXCP] -> [08 00 01 00 f6 00 00 00 ac bd 6a 00]
[PyXCP] <- L1 C11353 [ff]
[PyXCP] Execute(Master.upload -- Arguments(ARGS = (4,), KWS = {}))
[PyXCP] UPLOAD
[PyXCP] -> [02 00 02 00 f5 04]
[PyXCP] <- L5 C11354 [ff 98 00 00 00]
[PyXCP] Execute(Master.disconnect -- Arguments(ARGS = (), KWS = {}))
[PyXCP] DISCONNECT
[PyXCP] -> [01 00 03 00 fe]
[PyXCP] <- L1 C11355 [ff]
Backend Qt5Agg is interactive backend. Turning interactive mode on.
Exception in thread Thread-83:
Traceback (most recent call last):
File "C:\Program Files\Python\lib\threading.py", line 973, in _bootstrap_inner
self.run()
File "C:\Program Files\Python\lib\threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "C:\Dev\2024_hil\atp_core_hil\hiloscilloscope.py", line 736, in get_signals
time, start_acq_index, res = self.get_hil_signals()
File "C:\Dev\2024_hil\atp_core_hil\hiloscilloscope.py", line 711, in get_hil_signals
self.hil.write_characteristic(
File "C:\Dev\2024_hil\atp_core_hil\hw_utilities\interfaces\hil_interface.py", line 419, in write_characteristic
with self.ap.run() as xcp_connection:
File "c:\Dev\2024_hil\atp_core_hil\socomecenv\socomecenv\lib\site-packages\pyxcp\master\master.py", line 116, in enter
self.transport.connect()
File "c:\Dev\2024_hil\atp_core_hil\socomecenv\socomecenv\lib\site-packages\pyxcp\transport\eth.py", line 102, in connect
self.sock.connect(self.sockaddr)
socket.timeout: timed out

and here it is the wireshark trace: looking at it, it seems like there is a retransmission (.84 is my pc) before the stop of communication but there is a reply from the board (.72) then am acknowledge from the pc and nothing else.

If after the error I restart my application everything works fine so it seems like the embedded board is still running correctly.
thanks a lot
Michele

[MCO-SOCOMEC]

Hello Christoph,
do you have any idea? It seems to me that the initial problem comes from the embedded board (.72) that doens't reply to the request but after it does and it's the application that is hanging not being able to restart communication.
thanks
Michele

[christoph2]

@MCO-SOCOMEC
Hello Michele,
I think, you should first run xcp-info to see if basic communication is OK; the information you get, may also helpful for further steps.
The missing reply may indicate that you are writing to an illegal memory area and messing things up...
You should should carefully check the addresses you are writing to.

[MCO-SOCOMEC]

Hello Christoph,
finally I think I identified the problem. The problem is that sometimes I got an error entering in the master class and doing the connection: I got a socket error. I need to do a close to restart the communication otherwise the slave (my embedded board) doens't close the connection and it doens't reply anymore until I close.
Modifyinf the __enter function like this and putting in place a retry mechanism in my application it seeems to work fine.

def __enter__(self):
    """Context manager entry part."""
    try:
        self.transport.connect()
    except Exception as e:
        self.logger.error(msg=f"Error open connection: {e}")
        self.close()
    return self

Sometimes I catch an error but it's able to restart the communication.
What do you think?
thanks
Michele

[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 1 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left
[PyXCP] ERROR | Error open connection: timed out
ERROR:hw_utilities.interfaces.hil_interface:Error reading measurement: [WinError 10038] Tentativo di operazione su un elemento diverso dal socket
Read Measurement ... 2 attempts left

[MCO-SOCOMEC]

Hi Christoph,
what do you think about my workaround? Could be integrated in the library?
thanks
Michele

[christoph2]

@MCO-SOCOMEC
No, certainly not. The error-handling works as required by the XCP specification and there is also no pyXCP related issue.

[MCO-SOCOMEC]

Hi Christoph,
thank you for your support, I will search how to manage it in a different way.
Michele