storeFile raise SMBTimeout exception on files > 1k

[dmlos]

Our code started to fail with SMBTimeout:

  File "/upload_to_fileshare.py", line 62, in copy_file_from_s3_to_network_share
    self.connection.storeFile(fs_share_name, destination_folder, source_data_fp)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/smb/SMBConnection.py", line 376, in storeFile
    return self.storeFileFromOffset(service_name, path, file_obj, 0, True, timeout, show_progress = show_progress, tqdm_kwargs = tqdm_kwargs)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/smb/SMBConnection.py", line 408, in storeFileFromOffset
    self._pollForNetBIOSPacket(timeout)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/smb/SMBConnection.py", line 607, in _pollForNetBIOSPacket
    raise SMBTimeout
smb.base.SMBTimeout

After investigation appeared that it able to store 1k of byte data, hence the workaround was:

offset = 0
while offset < source_data_fp.getbuffer().nbytes:
    offset = self.connection.storeFileFromOffset(
        service_name=fs_share_name,
        path=destination_folder,
        file_obj=io.BytesIO(source_data_fp.read(1024)),
        offset=offset,
        truncate=True if offset == 0 else False,
    )

[miketeo]

@dmlos

1. If it's convenient, can you share what does the original code look like?
2. Have you tried uploading the file to the SMB server from a local filesystem instead of from a remote cloud storage?

[dmlos]

@dmlos

1. If it's convenient, can you share what does the original code look like?
2. Have you tried uploading the file to the SMB server from a local filesystem instead of from a remote cloud storage?

1. Original code:

with closing(io.BytesIO()) as source_data_fp:
    source_data_fp.write(
        s3_hook.get_key(bucket_name=s3_bucket_name, key=s3_key_source)
        .get()["Body"]
        .read()
    )

    source_data_fp.seek(0)

    self.connection.storeFile(fs_share_name, destination_folder, source_data_fp)

2. I tested similar code on Windows Cloud PC and it works, issue appear on AWS MWAA environment.