[WIP][AQUA] GPU Shape Recommendation

[elizjo]

Wrote an additional POST API and aqua command for recommending GPU shapes for a particular model

{
  "model_ocid" : "ocid1.datasciencemodel.oc1.<ocid>"
}

ads aqua recommend which_gpu --model_ocid 'ocid1.datasciencemodel.oc1.<ocid>' "" 

Returns

{
  "recommendations": [
    {
      "shape_details": {
        "core_count": null,
        "memory_in_gbs": null,
        "name": "VM.GPU2.1",
        "shape_series": "GPU",
        "gpu_specs": {
          "gpu_memory_in_gbs": 16,
          "gpu_count": 1,
          "gpu_type": "P100",
          "quantization": [
            "fp16"
          ],
          "ranking": {
            "cost": 10,
            "performance": 10
          }
        }
      },
      "configurations": [
        {
          "model_details": {
            "model_size_gb": 7.04,
            "kv_cache_size_gb": 6.711,
            "total_model_gb": 13.751
          },
          "deployment_params": {
            "quantization": "4bit",
            "max_model_len": 32768,
            "batch_size": 1
          },
          "recommendation": "While the selected compute shape is estimated to work (13.8GB used / 16.0GB allowed), the model configuration is close to the GPU memory limit. This estimation is theoretical; actual memory usage may vary at runtime.\n\nIf you encounter issues with this shape, consider the following options to reduce memory usage:\n\nBoth model weights and KV cache are significant contributors to memory use. (KV cache: 6.7GB, Weights: 7.0GB).\n\nTo reduce model size:\n1. Use a model with fewer parameters.\n2. No smaller quantized version exists. Use a model with fewer parameters."
        }
      ]
    }
  ],
  "troubleshoot": ""
}

TODO:
logic for extracting the config.json from the huggingface model needs to be improved
need to write a whole unit test module + conduct testing (business logic works, but API is untested)

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

📌 Cov diff with main:

📌 Overall coverage: