Issues with Pulumi

[farbodahm]

Currently we are facing the issues below with Pulumi. After having the first milestone and a deliverable product, we should decide if we want to continue with Pulumi or switch to CDK Terraform:

1. Package size of Pulumi is about 1 GB! (~600MB Pulumi Azure Provider)
   
2. In the context of Damavand, writing tests with Pulumi is complex.
3. Dependency management with Poetry/PDM is not straight forward.
4. Some error messages may not be explicit to debug:
5. While building a fat wheel to contain all of the dependencies, we need to include protobuf (it's a dependency for Pulumi). google module doesn't have a .dist-info so Glue can't install it correctly.

[wtfzambo]

@farbodahm regarding the package size I'm afraid there's nothing to do if we stick with pulumi.

I keep being surprised by point 4, as the traceback doesn't point to anything in your code. I've used pulumi for 2 years and never saw anything like this.

I'm not sure about point 3, and regarding point 2, I know that Pulumi has some documentation about it. Maybe take a look here if you haven't already: https://www.pulumi.com/docs/iac/concepts/testing/

[kkiani]

I fight a lot with pulumi with regards to using PDM. Unfortunately, Pulumi is only supporting pip and poetry which means enforcing tools to the end user. Writing tests in Pulumi is way harder than other IaC framework.

I have also faced the non explicit debug messages couple of times. This is expected due to the usage of RPCs under the hood.

This might be more subjective but I find the implementation of JS promise pattern in python makes a complex and non pythonic interface to work with.

[wtfzambo]

This might be more subjective but I find the implementation of JS promise pattern in python makes a complex and non pythonic interface to work with.

I agree, it can be confusing.

This is expected due to the usage of RPCs under the hood.

@kkiani can you elaborate on this? I don't understand why, what's exactly causing the issue?