ECS .NET SDK request signature does not match bug

[sballarati]

There are 22 ASCII characters that will work with the JAVA SDK but not with the .NET SDK when being set as as part of the metadata keys.
ECS returns Http code 403 with message:

“The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.”

Analysis

When the SDK signs the request that will be verified on the server, it orders the keys (headers) using "StringComparer.OrdinalIgnoreCase". But, on the server, the JAVA version generates a different signature, thus, it does not match and throws a 403 Signature Does not match

Method: static string BuildCanonicalizedHeaders(IDictionary<string, string> headers)

Class:
S3SignerECS

Version: v1.0.1

Repro steps

You can find attached two unit test projects showcasing the bug, one for NET and one for JAVA.

If you run the same example, you will notice with the JAVA SDK, you won't get the Signature match error. Most likely because the JAVA SDK uses the TreeMap class that performs a natural ordering.

dotnetUnderscoreECSTest.zip

JavaUnderscoreECSTest2.zip

Solution

We'd need a fix on the .NET SDK that signs the request same way the JAVA one does.

[sballarati]

Hi @ben-schumacher how are you doing? is this addressed on https://github.com/EMCECS/ecs-object-client-dotnet/releases/tag/v1.0.2 ?

thanks and have a nice day!

[ben-schumacher]

Hi @sballarati, yes this was resolved in v1.0.2. Please let me know if you have any questions or further issues.

[sballarati]

Hi @ben-schumacher thank you, we've re-tested and some were fixed but not all. Below the ones working and supported by the JAVA SDK:

Continue to fail

(") 34 02/02 42 22 QUOTATION MARK
(() 40 02/08 50 28 LEFT PARENTHESIS
()) 41 02/09 51 29 RIGHT PARENTHESIS
(,) 44 02/12 54 2C COMMA
(/) 47 02/15 57 2F SOLIDUS, SLASH
(;) 59 03/11 73 3B SEMICOLON
(<) 60 03/12 74 3C LESS-THAN SIGN, LEFT ANGLE BRACKET
(=) 61 03/13 75 3D EQUALS SIGN
(>) 62 03/14 76 3E GREATER-THAN SIGN, RIGHT ANGLE BRACKET
(?) 63 03/15 77 3F QUESTION MARK
(@) 64 04/00 100 40 COMMERCIAL AT SIGN
([) 91 05/11 133 5B LEFT SQUARE BRACKET
() 92 05/12 134 5C REVERSE SOLIDUS (BACKSLASH)
(]) 93 05/13 135 5D RIGHT SQUARE BRACKET
({) 123 07/11 173 7B LEFT CURLY BRACKET, LEFT BRACE
(}) 125 07/13 175 7D RIGHT CURLY BRACKET, RIGHT BRACE
(') 39 02/07 47 27 APOSTROPHE

By comparing the JAVA source code we noticed the special chars are being handled but not done in the NET SDK.

Thanks for looking into it!
Seba

[ben-schumacher]

Hi @sballarati ,

Because the ecs-object-client-dotnet SDK is merely an extension (using .net extension methods) of the real AWS SDK, we're still letting the AWSSDK.Core library handle all other aspects of the request/response logic. In other words, you will encounter this same issue using the AWS SDK. I agree, this doesn't align exactly to how the Java SDK works. If that's needed, we would need to incorporate some sort of handler for these specific metadata characters. Have you looked at possibly incorporating that code to accomplish what you're asking for? As of our latest release, the SDK is working as intended.

Ben

[sballarati]

hi Ben @ben-schumacher ,

I totally follow, we've also tried that out and found that we would end up opening an issue to the AWS SDK.

Our main concern is that someone using the JAVA SDK can set those special chars within the metadata keys but then, someone trying to update the metadata using the NET SDK won't be able to do it. Right?

[ben-schumacher]

@sballarati ,
That is correct and a valid point. It would be best to push AWS for a fix to address this issue and bring both SDKs into alignment on how they handle these characters.