Skip to content

Conversation

jojje
Copy link
Contributor

@jojje jojje commented Feb 27, 2025

New ability to see which GPU type is present in which Secure Cloud data center

Add cloud option to show datacenter for the gpus (closes #181)

Example output:

$ runpodctl get cloud --secure --datacenter

GPU TYPE                        MEM GB  VCPU    SPOT $/HR       ONDEMAND $/HR   DATA CENTERS
1x AMD Instinct MI300X OAM      283     24      1.490           2.490           EU-RO-1
1x NVIDIA A100 80GB PCIe        117     8       0.820           1.640           CA-MTL-3,EU-RO-1,US-KS-2,US-KS-3
1x NVIDIA A100-SXM4-80GB        125     16      0.950           1.890           EU-RO-1,EUR-IS-1,US-KS-2,US-KS-3
1x NVIDIA A40                   50      9       0.280           0.440           CA-MTL-1,EU-SE-1,OC-AU-1,US-KS-3
1x NVIDIA GeForce RTX 3090      125     16      0.220           0.430           EU-CZ-1
1x NVIDIA GeForce RTX 4090      30      6       0.350           0.690           EU-CZ-1,EU-RO-1,EUR-IS-1,EUR-IS-2,EUR-NO-1,US-CA-2,US-IL-1,US-TX-3
1x NVIDIA H100 80GB HBM3        125     16      1.750           2.990           CA-MTL-1,CA-MTL-2,CA-MTL-3,EU-FR-1,EU-NL-1,EUR-IS-3,US-CA-2,US-GA-2,US-KS-2,US-TX-3
1x NVIDIA H100 NVL              94      16      1.650           2.790           OC-AU-1,US-CA-2,US-GA-2,US-KS-2
1x NVIDIA H100 PCIe             188     24      1.250           2.390           US-KS-2
1x NVIDIA H200                  188     12      Reserved        Reserved        EU-FR-1,EUR-IS-2,US-GA-2,US-NC-1,US-TX-4
1x NVIDIA L4                    50      12      0.220           0.430           EU-RO-1,EUR-IS-1,EUR-IS-2
1x NVIDIA L40                   94      8       0.500           0.990           EUR-IS-1,EUR-IS-2,US-KS-2
1x NVIDIA L40S                  62      16      0.430           0.860           EU-NL-1,OC-AU-1,US-IL-1,US-TX-3,US-TX-4
1x NVIDIA RTX 2000 Ada          31      6       0.140           0.280           EU-RO-1,EUR-IS-1
Generation
1x NVIDIA RTX 4000 Ada          47      9       0.190           0.380           EU-RO-1,EUR-IS-1,US-CA-2
Generation
1x NVIDIA RTX 6000 Ada          94      16      0.390           0.770           US-CA-2,US-IL-1,US-KS-2,US-KS-3
Generation
1x NVIDIA RTX A4000             20      5       0.160           0.320           EU-RO-1,EUR-IS-1,EUR-NO-1,US-IL-1,US-TX-2
1x NVIDIA RTX A4500             31      12      0.180           0.340           EU-RO-1
1x NVIDIA RTX A5000             25      9       0.220           0.360           CA-MTL-1,EU-CZ-1,EU-SE-1,US-CA-2,US-IL-1
1x NVIDIA RTX A6000             50      8       0.380           0.760           EU-CZ-1,EU-RO-1,EU-SE-1,US-KS-1,US-KS-2,US-TX-1

And when no datacenter mapping exists on the backend, such as for GPU options that aren't available in a Runpod managed DC

$ runpodctl get cloud --datacenter

GPU TYPE                        MEM GB  VCPU    SPOT $/HR       ONDEMAND $/HR   DATA CENTERS
1x AMD Instinct MI300X OAM      283     24      Reserved        Reserved        EU-RO-1
1x NVIDIA A100 80GB PCIe        117     8       0.600           1.190           CA-MTL-3,EU-RO-1,US-KS-2,US-KS-3
1x NVIDIA A100-SXM4-80GB        125     16      0.890           1.590           EU-RO-1,EUR-IS-1,US-KS-2,US-KS-3
1x NVIDIA A40                   50      9       0.240           0.350           CA-MTL-1,EU-SE-1,OC-AU-1,US-KS-3
1x NVIDIA GeForce RTX 3070      31      16      0.070           0.130
1x NVIDIA GeForce RTX 3080      30      7       0.090           0.170
1x NVIDIA GeForce RTX 3080 Ti   20      8       0.090           0.180
1x NVIDIA GeForce RTX 3090      24      4       0.110           0.220           EU-CZ-1
1x NVIDIA GeForce RTX 4070 Ti   29      8       0.100           0.190
1x NVIDIA GeForce RTX 4080      22      8       0.160           0.270
1x NVIDIA GeForce RTX 4080      35      18      0.170           0.280
SUPER
1x NVIDIA GeForce RTX 4090      29      6       0.200           0.340           EU-CZ-1,EU-RO-1,EUR-IS-1,EUR-IS-2,EUR-NO-1,US-CA-2,US-IL-1,US-TX-3
1x NVIDIA GeForce RTX 5080      41      21      0.250           0.490
1x NVIDIA H100 80GB HBM3        125     20      1.500           2.690           CA-MTL-1,CA-MTL-2,CA-MTL-3,EU-FR-1,EU-NL-1,EUR-IS-3,US-CA-2,US-GA-2,US-KS-2,US-TX-3
1x NVIDIA H100 NVL              94      16      1.400           2.590           OC-AU-1,US-CA-2,US-GA-2,US-KS-2
1x NVIDIA H100 PCIe             188     24      1.350           1.990           US-KS-2
1x NVIDIA H200                  188     12      Reserved        Reserved        EU-FR-1,EUR-IS-2,US-GA-2,US-NC-1,US-TX-4
1x NVIDIA L4                    50      12      Reserved        Reserved        EU-RO-1,EUR-IS-1,EUR-IS-2
1x NVIDIA L40                   94      8       Reserved        Reserved        EUR-IS-1,EUR-IS-2,US-KS-2
1x NVIDIA L40S                  62      16      0.400           0.790           EU-NL-1,OC-AU-1,US-IL-1,US-TX-3,US-TX-4
1x NVIDIA RTX 2000 Ada          31      6       Reserved        Reserved        EU-RO-1,EUR-IS-1
Generation
1x NVIDIA RTX 4000 Ada          31      6       0.100           0.200           EU-RO-1,EUR-IS-1,US-CA-2
Generation
1x NVIDIA RTX A2000             35      9       0.060           0.120
1x NVIDIA RTX A4000             20      4       0.090           0.170           EU-RO-1,EUR-IS-1,EUR-NO-1,US-IL-1,US-TX-2
1x NVIDIA RTX A4500             29      4       0.100           0.190           EU-RO-1
1x NVIDIA RTX A5000             25      4       0.110           0.220           CA-MTL-1,EU-CZ-1,EU-SE-1,US-CA-2,US-IL-1
1x NVIDIA RTX A6000             50      8       0.250           0.440           EU-CZ-1,EU-RO-1,EU-SE-1,US-KS-1,US-KS-2,US-TX-1
1x Tesla V100-PCIE-16GB         43      4       0.100           0.190
1x Tesla V100-SXM2-16GB         62      10      0.120           0.230
1x Tesla V100-SXM2-32GB         93      20      0.170           0.330

The only gotcha people have to keep in mind is that the prices will be the community (lowest) prices when not explicitly giving the -s/--secure flag. But that was already the case before this patch so nothing has changed in this regard. The only reason I'm pointing this out is that it might be a bit deceptive to see the community prices by default, or when "-c" is provided, and still see the DC mapping. If the user doesn't recognize that the DCs are for the listed GPU types, not the prices in those DCs for that type, they may make incorrect assumptions about the price for running the GPU in secure cloud.

I did consider this conundrum a bit and initially only presented the DCs when -s/--secure was provided. However I found that to be rather annoying since it precluded answering the questions "Which GPUs in community cloud do Runpod also offer in a secure cloud variant?", as well as "What are all the possible GPUs, and for those that run in DCs, where can I find them?"

Those two questions prompted the current implementation of showing the DC mapping whenever there is a DC that has that GPU.

Finally, there is a reason I'm not using a comma-space delimiter; Line wrapping. I find that highly annoying for a CLI, and I think the GPU type field is a perfect example. If you grep on the line "4000 Ada" you expect to get the complete type name that you can copy-paste into wherever you need it (like the create pod subcommand), but that doesn't work. The Generation suffix is required and got line-broken so doesn't end up in the grep output, leading to user frustration. As such I wanted to ensure users could grep on their "favorite" DCs and be sure to get the complete context for that DC, with all the GPUs. This would not have been possible if I'd allowed the table writer to line-break the DC CSV strings.

How I tested it

Via interactive debugging of GraphQL interaction, unmarshalling and branching correctness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Show datacenter availability for the different GPU types

1 participant