Skip to content

cluster-autoscaler: vendor aws-sdk-go-v2 #8265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
56 changes: 56 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,8 @@ To refresh static list, please run `go run ec2_instance_types/gen.go` under

## Using the AWS SDK vendored in the AWS cloudprovider

### v1

If you want to use a newer version of the AWS SDK than the version currently vendored as a direct dependency by Cluster Autoscaler, then you can use the version vendored under this AWS cloudprovider.

The current version vendored is `v1.48.7`.
Expand All @@ -436,6 +438,60 @@ If you want to update the vendored AWS SDK to a newer version, please make sure
3. Update the import statements within the newly-copied AWS SDK to reference the new paths (e.g., `github.com/aws/aws-sdk-go/aws/awsutil` -> `k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws-sdk-go/aws/awsutil`). You can use this command from the aws-sdk-go folder `find . -type f -exec sed -i ‘s#github.com/aws/aws-sdk-go#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws-sdk-go#’ {} \;`
4. Update the version number above to indicate the new vendored version.

### v2
If you want to use a newer version of the AWS SDK than the version currently vendored as a direct dependency by Cluster Autoscaler, then you can use the version vendored under this AWS cloudprovider.

The current version vendored is `2025-06-17`.

The aws-sdk-go-v2 also depends on aws/smithy-go; this is similarly vendored; check the go.mod of the aws-sdk-go-v2 package for the correct version of this package.

The current version vendored is `v1.22.4`

You can use the commands below to update the vendored copy of these modules to a newer version

```shell
export SDK_VERSION="2025-06-17"
export SMITHY_VERSION="1.22.4"

# Prepare clean smithy-go folder
## Clean up old release
rm -rf smithy-go/

## Download and extract specified release
curl -L "https://github.com/aws/smithy-go/archive/refs/tags/v${SMITHY_VERSION}.tar.gz" -o smithy-go.tar.gz
tar xf smithy-go.tar.gz
mv "smithy-go-${SMITHY_VERSION}" smithy-go
rm smithy-go.tar.gz

# Adjust for vendoring within cluster-autoscaler
## Remove unneeded files, directories, and tests to reduce size
rm -r ./smithy-go/.github ./smithy-go/codegen ./smithy-go/.travis.yml
find ./smithy-go \( -name "*_test.go" -o -name 'go.mod' -o -name 'go.sum' \) -exec rm {} \+

## Update paths for local vendoring
find ./smithy-go -name '*.go' -type f -exec sed -i '' 's#github.com/aws/smithy-go#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/smithy-go#' {} \+

# Prepare clean sdk-v2 folder
## Clean up old release
rm -rf aws-sdk-go-v2/

## Download and extract specified release
curl -L "https://github.com/aws/aws-sdk-go-v2/archive/refs/tags/release-${SDK_VERSION}.tar.gz" -o aws-sdk-go-v2.tar.gz
tar xf aws-sdk-go-v2.tar.gz
mv "aws-sdk-go-v2-release-${SDK_VERSION}" aws-sdk-go-v2
rm aws-sdk-go-v2.tar.gz

# Adjust for vendoring within cluster-autoscaler
## Remove unneeded files, directories, and tests to reduce size
rm ./aws-sdk-go-v2/.travis.yml ./aws-sdk-go-v2/.golangci.toml ./aws-sdk-go-v2/buildspec.yml ./aws-sdk-go-v2/ci-find-smithy-go.sh ./aws-sdk-go-v2/local-mod-replace.sh ./aws-sdk-go-v2/modman.toml
rm -r ./aws-sdk-go-v2/.github ./aws-sdk-go-v2/codegen ./aws-sdk-go-v2/example ./aws-sdk-go-v2/internal/codegen ./aws-sdk-go-v2/internal/repotools
find ./aws-sdk-go-v2 \( -name "*_test.go" -o -name "*.go.snap" -o -name 'go.mod' -o -name 'go.sum' \) -exec rm {} \+

## Update imports to local path
find ./aws-sdk-go-v2 -name '*.go' -type f -exec sed -i '' 's#github.com/aws/aws-sdk-go-v2#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws-sdk-go-v2#' {} \+
find ./aws-sdk-go-v2 -name '*.go' -type f -exec sed -i '' 's#github.com/aws/smithy-go#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/smithy-go#' {} \+
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled this branch and ran the commands:

export SDK_VERSION="2025-06-17"
export SMITHY_VERSION="1.22.4"

# Prepare clean smithy-go folder
## Clean up old release
rm -rf smithy-go/

## Download and extract specified release
curl -L "https://github.com/aws/smithy-go/archive/refs/tags/v${SMITHY_VERSION}.tar.gz" -o smithy-go.tar.gz
tar xf smithy-go.tar.gz
mv "smithy-go-${SMITHY_VERSION}" smithy-go
rm smithy-go.tar.gz

# Adjust for vendoring within cluster-autoscaler 
## Remove unneeded files, directories, and tests to reduce size
rm -r ./smithy-go/.github ./smithy-go/codegen ./smithy-go/.travis.yml
find ./smithy-go \( -name "*_test.go" -o -name 'go.mod' -o -name 'go.sum' \) -exec rm {} \+

## Update paths for local vendoring
find ./smithy-go -name '*.go' -type f -exec sed -i '' 's#github.com/aws/smithy-go#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/smithy-go#' {} \+

# Prepare clean sdk-v2 folder
## Clean up old release
rm -rf aws-sdk-go-v2/

## Download and extract specified release
curl -L "https://github.com/aws/aws-sdk-go-v2/archive/refs/tags/release-${SDK_VERSION}.tar.gz" -o aws-sdk-go-v2.tar.gz
tar xf aws-sdk-go-v2.tar.gz
mv "aws-sdk-go-v2-release-${SDK_VERSION}" aws-sdk-go-v2
rm aws-sdk-go-v2.tar.gz

# Adjust for vendoring within cluster-autoscaler
## Remove unneeded files, directories, and tests to reduce size
rm ./aws-sdk-go-v2/.travis.yml ./aws-sdk-go-v2/.golangci.toml ./aws-sdk-go-v2/buildspec.yml ./aws-sdk-go-v2/ci-find-smithy-go.sh ./aws-sdk-go-v2/local-mod-replace.sh ./aws-sdk-go-v2/modman.toml
rm -r ./aws-sdk-go-v2/.github ./aws-sdk-go-v2/codegen ./aws-sdk-go-v2/example ./aws-sdk-go-v2/internal/codegen ./aws-sdk-go-v2/internal/repotools
find ./aws-sdk-go-v2 \( -name "*_test.go" -o -name "*.go.snap" -o -name 'go.mod' -o -name 'go.sum' \) -exec rm {} \+

## Update imports to local path
find ./aws-sdk-go-v2 -name '*.go' -type f -exec sed -i '' 's#github.com/aws/aws-sdk-go-v2#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws-sdk-go-v2#' {} \+
find ./aws-sdk-go-v2 -name '*.go' -type f -exec sed -i '' 's#github.com/aws/smithy-go#k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/smithy-go#' {} \+

I had no diff, so was successfully able to regenerate the exact same vendors 👍🏾


## Using cloud config with helm

If you want to use custom AWS cloud config e.g. endpoint urls
Expand Down
31 changes: 31 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/aws-sdk-go-v2/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Project-specific ignore
/doc
/doc-staging
/internal/awstesting/integration/smoke/**/importmarker__.go
/internal/awstesting/integration/smoke/_test/
/vendor
/private/model/cli/gen-api/gen-api
build/
bin/

# Mac
.DS_Store/

# Intellij
.idea/
*.iml
*.ipr
*.iws

# Node
dist

# Java
.gradle/

# vscode
.vscode/

# Ruby
.yardoc
Gemfile.lock
19,819 changes: 19,819 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/aws-sdk-go-v2/CHANGELOG.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
## Code of Conduct
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
[email protected] with any additional questions or comments.
161 changes: 161 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/aws-sdk-go-v2/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Contributing to the AWS SDK for Go

Thank you for your interest in contributing to the AWS SDK for Go!
We work hard to provide a high-quality and useful SDK, and we greatly value
feedback and contributions from our community. Whether it's a bug report,
new feature, correction, or additional documentation, we welcome your issues
and pull requests. Please read through this document before submitting any
[issues] or [pull requests][pr] to ensure we have all the necessary information to
effectively respond to your bug report or contribution.

Jump To:

* [Bug Reports](#bug-reports)
* [Feature Requests](#feature-requests)
* [Code Contributions](#code-contributions)

## How to contribute

*Before you send us a pull request, please be sure that:*

1. You're working from the latest source on the `main` branch.
2. You check existing open, and recently closed, pull requests to be sure
that someone else hasn't already addressed the problem.
3. You create an issue before working on a contribution that will take a
significant amount of your time.

*Creating a Pull Request*

1. Fork the repository.
2. In your fork, make your change in a branch that's based on this repo's `main` branch.
3. Commit the change to your fork, using a clear and descriptive commit message.
4. Open a pull request, answering any questions in the pull request form.

For contributions that will take a significant amount of time, open a new
issue to pitch your idea before you get started. Explain the problem and
describe the content you want to see added to the documentation. Let us know
if you'll write it yourself or if you'd like us to help. We'll discuss your
proposal with you and let you know whether we're likely to accept it.

## Bug Reports

You can file bug reports against the SDK on the [GitHub issues][issues] page.

If you are filing a report for a bug or regression in the SDK, it's extremely
helpful to provide as much information as possible when opening the original
issue. This helps us reproduce and investigate the possible bug without having
to wait for this extra information to be provided. Please read the following
guidelines prior to filing a bug report.

1. Search through existing [issues][] to ensure that your specific issue has
not yet been reported. If it is a common issue, it is likely there is
already a bug report for your problem.

2. Ensure that you have tested the latest version of the SDK. Although you
may have an issue against an older version of the SDK, we cannot provide
bug fixes for old versions. It's also possible that the bug may have been
fixed in the latest release.

3. Provide as much information about your environment, SDK version, and
relevant dependencies as possible. For example, let us know what version
of Go you are using, which and version of the operating system, and the
the environment your code is running in. e.g Container.

4. Provide a minimal test case that reproduces your issue or any error
information you related to your problem. We can provide feedback much
more quickly if we know what operations you are calling in the SDK. If
you cannot provide a full test case, provide as much code as you can
to help us diagnose the problem. Any relevant information should be provided
as well, like whether this is a persistent issue, or if it only occurs
some of the time.

## Feature Requests

Open an [issue][issues] with the following:

* A short, descriptive title. Ideally, other community members should be able
to get a good idea of the feature just from reading the title.
* A detailed description of the the proposed feature.
* Why it should be added to the SDK.
* If possible, example code to illustrate how it should work.
* Use Markdown to make the request easier to read;
* If you intend to implement this feature, indicate that you'd like to the issue to be assigned to you.

## Code Contributions

We are always happy to receive code and documentation contributions to the SDK.
Please be aware of the following notes prior to opening a pull request:

1. The SDK is released under the [Apache license][license]. Any code you submit
will be released under that license. For substantial contributions, we may
ask you to sign a [Contributor License Agreement (CLA)][cla].

2. If you would like to implement support for a significant feature that is not
yet available in the SDK, please talk to us beforehand to avoid any
duplication of effort.

3. Wherever possible, pull requests should contain tests as appropriate.
Bugfixes should contain tests that exercise the corrected behavior (i.e., the
test should fail without the bugfix and pass with it), and new features
should be accompanied by tests exercising the feature.

4. Pull requests that contain failing tests will not be merged until the test
failures are addressed. Pull requests that cause a significant drop in the
SDK's test coverage percentage are unlikely to be merged until tests have
been added.

5. The JSON files under the SDK's `models` folder are sourced from outside the SDK.
Such as `models/apis/ec2/2016-11-15/api.json`. We will not accept pull requests
directly on these models. If you discover an issue with the models please
create a [GitHub issue][issues] describing the issue.

### Testing

To run the tests locally, running the `make unit` command will `go get` the
SDK's testing dependencies, and run vet, link and unit tests for the SDK.

```
make unit
```

Standard go testing functionality is supported as well. To test SDK code that
is tagged with `codegen` you'll need to set the build tag in the go test
command. The `make unit` command will do this automatically.

```
go test -tags codegen ./private/...
```

See the `Makefile` for additional testing tags that can be used in testing.

To test on multiple platform the SDK includes several DockerFiles under the
`awstesting/sandbox` folder, and associated make recipes to to execute
unit testing within environments configured for specific Go versions.

```
make sandbox-test-go18
```

To run all sandbox environments use the following make recipe

```
# Optionally update the Go tip that will be used during the batch testing
make update-aws-golang-tip

# Run all SDK tests for supported Go versions in sandboxes
make sandbox-test
```

In addition the sandbox environment include make recipes for interactive modes
so you can run command within the Docker container and context of the SDK.

```
make sandbox-go18
```

[issues]: https://github.com/aws/aws-sdk-go-v2/issues
[pr]: https://github.com/aws/aws-sdk-go-v2/pulls
[license]: http://aws.amazon.com/apache2.0/
[cla]: http://en.wikipedia.org/wiki/Contributor_License_Agreement
[releasenotes]: https://github.com/aws/aws-sdk-go-v2/releases

15 changes: 15 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/aws-sdk-go-v2/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Open Discussions
---
The following issues are currently open for community feedback.
All discourse must adhere to the [Code of Conduct] policy.

* [Refactoring API Client Paginators](https://github.com/aws/aws-sdk-go-v2/issues/439)
* [Refactoring API Client Waiters](https://github.com/aws/aws-sdk-go-v2/issues/442)
* [Refactoring API Client Enums and Types to Discrete Packages](https://github.com/aws/aws-sdk-go-v2/issues/445)
* [SDK Modularization](https://github.com/aws/aws-sdk-go-v2/issues/444)

Past Discussions
---
The issues listed here are for documentation purposes, and is used to capture issues and their associated discussions.

[Code of Conduct]: https://github.com/aws/aws-sdk-go-v2/blob/main/CODE_OF_CONDUCT.md
Loading
Loading