INTPYTHON-527 Add Queryable Encryption support #329
Conversation
|
Wrong commit message for 65bd15a and I don't want to force push yet. It should have said:
I'm aware that
|
It's not working as you think it is. As I said elsewhere, Does this fix the "command not supported for auto encryption: buildinfo" error? If so, it's perhaps because I'd suggest to use my patch is as a starting point for maintaining two connections. |
I don't disagree, but it feels a lot like
Yes it works by design, not a side effect. I'm
I'd make a few passes at it but did not get anywhere, I'll try again though. |
|
Your "stumble" theory of how it's working isn't correct. |
Copy that, thanks! I've removed Still working on an unencrypted connection, but perhaps the only time we need it is for the version check. |
|
@ShaneHarvey @Jibola @timgraham FYI here is the And here is the error again with some additional debug: And the full traceback: Test settings: This is happening in the |
Move the encryption checks for patient to test_patient.
Encryption tests will fail if the schema is wrong.
Ideally in schema.py instead of indexing KMS_CREDENTIALS with provider, configure and use on-demand credentials. However, the implementation in libmongocrypt appears to be that given a provider, the credentials are acquired and used by PyMongo, which may not be suitable for use in schema.py. However it may be possible to call a function in libmongocrypt, instead of indexing KMS_CREDENTIALS with provider.
| KMS_CREDENTIALS = { | ||
| "aws": { | ||
| "key": os.getenv("AWS_KEY_ARN", ""), | ||
| "region": os.getenv("AWS_KEY_REGION", ""), | ||
| }, | ||
| "azure": { | ||
| "keyName": os.getenv("AZURE_KEY_NAME", ""), | ||
| "keyVaultEndpoint": os.getenv("AZURE_KEY_VAULT_ENDPOINT", ""), | ||
| }, | ||
| "gcp": { | ||
| "projectId": os.getenv("GCP_PROJECT_ID", ""), | ||
| "location": os.getenv("GCP_LOCATION", ""), | ||
| "keyRing": os.getenv("GCP_KEY_RING", ""), | ||
| "keyName": os.getenv("GCP_KEY_NAME", ""), | ||
| }, | ||
| "kmip": {}, | ||
| "local": {}, | ||
| } |
There was a problem hiding this comment.
I think KMS_CREDENTIALS goes in DATABASES also. Referencing encryption.KMS_CREDENTIALS in schema editor doesn't look good to me for several reasons. It's "global state" as we discussed with kms providers, where it eliminates the possibility to use different provider credentials for different database aliases.
Also, the use of environment couples the list of providers ("aws", "azure", etc.) as well as each providers options ("accessKeyId", "secretAccessKey", etc.) to this package's release cycle. I remain unconvinced that the environment variables solution is useful and a step toward "making this feature work with minimal effort." How are environment variables less effort than something like this:
DATABASES = {
"alias": {
....
"KMS_CREDENTIALS": {
"aws": {
"accessKeyId": ...,
"secretAccessKey": ...,
},
},
},
}
django_mongodb_backend/management/commands/get_encrypted_fields_map.py
Outdated
Show resolved
Hide resolved
Maybe folks can use the mixin with any Django fields we don't provide ?
Subclassing `dict` to support `queries=EqualityQuery()` API
- Move aws creds to on-demand credentials provided by libmongocrypt (requires `pip install pymongo[aws]`. - Mock boto3 response - Not sure if KMS_CREDENTIALS are being used since the tests succeed after they pass the boto3 mock. - Test var cleanup
- Local provider has no configurable env setting - Kmip provider has configurable provider env only
(see previous attempts in #318, #319 and #323 for additional context)