S3 IAM_AUTH issue

[faiqramzan]

I am replacing minio with S3 while deploying karton-playground. I am using IAM role. If i remove access_key from karton.docker.ini I get the following error although iam_auth is set to true:

karton-playground-karton-mwdb-reporter-1     | Traceback (most recent call last):
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/bin/karton-mwdb-reporter", line 8, in <module>
karton-playground-karton-mwdb-reporter-1     |     sys.exit(MWDBReporter.main())
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/utils.py", line 122, in newfunc
karton-playground-karton-mwdb-reporter-1     |     return self.func(owner, *args, **kwargs)
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/base.py", line 242, in main
karton-playground-karton-mwdb-reporter-1     |     service = cls.karton_from_args()
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/base.py", line 187, in karton_from_args
karton-playground-karton-mwdb-reporter-1     |     return cls(config=config)
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/mwdb_reporter/mwdb_reporter.py", line 104, in __init__
karton-playground-karton-mwdb-reporter-1     |     super().__init__(*args, **kwargs)
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/karton.py", line 417, in __init__
karton-playground-karton-mwdb-reporter-1     |     super().__init__(config=config, identity=identity, backend=backend)
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/karton.py", line 118, in __init__
karton-playground-karton-mwdb-reporter-1     |     super().__init__(config=config, identity=identity, backend=backend)
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/base.py", line 208, in __init__
karton-playground-karton-mwdb-reporter-1     |     super().__init__(
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/karton.py", line 58, in __init__
karton-playground-karton-mwdb-reporter-1     |     super().__init__(config=config, identity=identity, backend=backend)
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/base.py", line 55, in __init__
karton-playground-karton-mwdb-reporter-1     |     self.backend = backend or KartonBackend(
karton-playground-karton-mwdb-reporter-1     |   File "/usr/local/lib/python3.9/site-packages/karton/core/backend.py", line 119, in __init__
karton-playground-karton-mwdb-reporter-1     |     aws_access_key_id=config["s3"]["access_key"],
karton-playground-karton-mwdb-reporter-1     | KeyError: 'access_key'

But when I set access_key empty I get the following error:

karton-playground-karton-classifier-1        | [2025-10-22 14:26:58,274][INFO] Service karton.classifier started
karton-playground-karton-classifier-1        | /usr/local/lib/python3.9/site-packages/karton/core/logger.py:66: UserWarning: There is no active log consumer to receive logged messages.
karton-playground-karton-classifier-1        |   warnings.warn("There is no active log consumer to receive logged messages.")
karton-playground-karton-classifier-1        | [2025-10-22 14:26:58,275][INFO] Binding on: {'type': 'sample', 'kind': 'raw'}
karton-playground-karton-classifier-1        | [2025-10-22 14:26:58,279][INFO] Received new task - {6278b90b-afa7-4e0e-82ce-2c932b8405a2}:0251517e-45d0-4435-9200-b691689779c6
karton-playground-karton-classifier-1        | [2025-10-22 14:26:58,465][ERROR] Failed to process task - {6278b90b-afa7-4e0e-82ce-2c932b8405a2}:0251517e-45d0-4435-9200-b691689779c6
karton-playground-karton-classifier-1        | Traceback (most recent call last):
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/karton/core/karton.py", line 179, in internal_process
karton-playground-karton-classifier-1        |     self.process(self.current_task)
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/karton/classifier/classifier.py", line 87, in process
karton-playground-karton-classifier-1        |     sample_class = self._classify(task)
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/karton/classifier/classifier.py", line 157, in _classify
karton-playground-karton-classifier-1        |     content = cast(bytes, sample.content)
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/karton/core/resource.py", line 430, in content
karton-playground-karton-classifier-1        |     return self.download()
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/karton/core/resource.py", line 484, in download
karton-playground-karton-classifier-1        |     self._content = self.backend.download_object(self.bucket, self.uid)
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/karton/core/backend.py", line 752, in download_object
karton-playground-karton-classifier-1        |     with self.s3.get_object(Bucket=bucket, Key=object_uid)["Body"] as f:
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 530, in _api_call
karton-playground-karton-classifier-1        |     return self._make_api_call(operation_name, kwargs)
karton-playground-karton-classifier-1        |   File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 964, in _make_api_call
karton-playground-karton-classifier-1        |     raise error_class(parsed_response, operation_name)
karton-playground-karton-classifier-1        | botocore.exceptions.ClientError: An error occurred (AuthorizationHeaderMalformed) when calling the GetObject operation: The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.

I can't figure out what I am doing wrong, here is my karton.docker.ini file:

[redis]
host=redis

[s3]
access_key =
secret_key =
address = https://s3.us-east-1.amazonaws.com
bucket = bucket-name
region = us-east-1
iam_auth = True

[mwdb]
api_url = http://mwdb.:8080/api/
username = admin
password = admin

[yankovs]

You're using an old version of Karton in your karton-playground-karton-mwdb-reporter instance which didn't support IAM authentication. Newer Karton has this feature implemented (see: https://github.com/CERT-Polska/karton/blob/master/karton/core/backend.py#L323)

I'd try to force rebuild in your docker compose command so it takes the latest release of Karton. After doing that, I'd exec into the containers of the Kartons and do pip list | grep karton to see what version of Karton you have in each Karton, this is helpful for debugging.

[yankovs]

And by the way, access_key and secret_key should be omitted when configuring iam_auth = True. They can still be present as fallback, but at least put values in them. If not needed, remove them.