Skip to content
24 changes: 17 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,29 @@
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/jetstack/version-checker)

version-checker is a Kubernetes utility for observing the current versions of
images running in the cluster, as well as the latest available upstream. These
checks get exposed as Prometheus metrics to be viewed on a dashboard, or _soft_
alert cluster operators.
images running in the cluster, as well as the latest available upstream. Additionally,
it monitors the Kubernetes cluster version against the latest available releases
using official Kubernetes release channels. These checks get exposed as Prometheus
metrics to be viewed on a dashboard, or _soft_ alert cluster operators.

## Features

- **Container Image Version Checking**: Monitor and compare container image versions running in the cluster against their latest upstream versions
- **Kubernetes Version Monitoring**: Track your cluster's Kubernetes version against the latest available releases from official Kubernetes channels
- **Prometheus Metrics Integration**: Export all version information as Prometheus metrics for monitoring and alerting
- **Flexible Channel Selection**: Configure which Kubernetes release channel to track (stable, latest, etc.)

---

## Why Use version-checker?

- **Improved Security**: Ensures images are up-to-date, reducing the risk of using vulnerable or compromised versions.
- **Enhanced Visibility**: Provides a clear overview of all running container versions across clusters.
- **Operational Efficiency**: Automates image tracking and reduces manual intervention in version management.
- **Compliance and Policy**: Enforcement: Helps maintain version consistency and adherence to organizational policies.
- **Improved Security**: Ensures images and Kubernetes clusters are up-to-date, reducing the risk of using vulnerable or compromised versions.
- **Enhanced Visibility**: Provides a clear overview of all running container versions and cluster versions across clusters.
- **Operational Efficiency**: Automates image and Kubernetes version tracking and reduces manual intervention in version management.
- **Compliance and Policy Enforcement**: Helps maintain version consistency and adherence to organizational policies for both applications and infrastructure.
- **Incremental Upgrades**: Facilitates frequent, incremental updates to reduce the risk of large, disruptive upgrades.
- **Add-On Compatibility**: Ensures compatibility with the latest versions of Kubernetes add-ons and dependencies.
- **Proactive Cluster Management**: Stay informed about Kubernetes security updates and new features through automated version monitoring.

---

Expand All @@ -45,6 +54,7 @@ These registries support authentication.

- [Installation Guide](docs/installation.md)
- [Metrics](docs/metrics.md)
- [New Features](docs/new_features.md)
Comment thread
maria-reynoso marked this conversation as resolved.

---

Expand Down
16 changes: 14 additions & 2 deletions cmd/app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,16 +110,28 @@ func NewCommand(ctx context.Context) *cobra.Command {
return fmt.Errorf("failed to setup image registry clients: %s", err)
}

c := controller.NewPodReconciler(opts.CacheTimeout,
_ = client

podController := controller.NewPodReconciler(opts.CacheTimeout,
metricsServer,
client,
mgr.GetClient(),
log,
opts.RequeueDuration,
opts.DefaultTestAll,
)
if err := podController.SetupWithManager(mgr); err != nil {
return err
}

if err := c.SetupWithManager(mgr); err != nil {
kubeController := controller.NewKubeReconciler(
Comment thread
maria-reynoso marked this conversation as resolved.
log,
mgr.GetConfig(),
metricsServer,
opts.KubeInterval,
opts.KubeChannel,
)
if err := mgr.Add(kubeController); err != nil {
return err
}

Expand Down
14 changes: 13 additions & 1 deletion cmd/app/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,10 @@ type Options struct {
CacheSyncPeriod time.Duration
RequeueDuration time.Duration

KubeChannel string
KubeInterval time.Duration

// kubeConfigFlags holds the flags for the kubernetes client
kubeConfigFlags *genericclioptions.ConfigFlags

selfhosted selfhosted.Options
Expand Down Expand Up @@ -141,7 +145,15 @@ func (o *Options) addAppFlags(fs *pflag.FlagSet) {

fs.DurationVarP(&o.CacheSyncPeriod,
"cache-sync-period", "", 5*time.Hour,
"The time in which all resources should be updated.")
"The duration in which all resources should be updated.")

fs.DurationVarP(&o.KubeInterval,
"kube-interval", "", o.CacheSyncPeriod,
"The time in which kubernetes channels updates are checked.")

fs.StringVarP(&o.KubeChannel,
"kube-channel", "", "stable",
"The Kubernetes channel to check against for cluster updates.")

fs.DurationVarP(&o.GracefulShutdownTimeout,
"graceful-shutdown-timeout", "", 10*time.Second,
Expand Down
20 changes: 18 additions & 2 deletions docs/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,32 @@

By default, version-checker exposes the following Prometheus metrics on `0.0.0.0:8080/metrics`:

## Container Image Metrics

- `version_checker_is_latest_version`: Indicates whether the container in use is using the latest upstream registry version.
- `version_checker_last_checked`: Timestamp when the image was last checked.
- `version_checker_image_lookup_duration`: Duration of the image version check.
- `version_checker_image_failures_total`: Total of errors encountered during image version checks.

## Kubernetes Version Metrics

- `version_checker_is_latest_kube_version`: Indicates whether the cluster is running the latest version from the configured Kubernetes release channel.
- Labels: `current_version`, `latest_version`, `channel`
- Value `1`: Cluster is up-to-date
- Value `0`: Update available

---

## Example Prometheus Query
## Example Prometheus Queries

### Check container image versions
```sh
QUERY="version_checker_is_latest_version"
curl -s --get --data-urlencode query=$QUERY <PROMETHEUS_URL>
```
```

### Check Kubernetes cluster version
```sh
QUERY="version_checker_is_latest_kube_version"
curl -s --get --data-urlencode query=$QUERY <PROMETHEUS_URL>
```
4 changes: 2 additions & 2 deletions pkg/client/fallback/fallback.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ func (c *Client) Tags(ctx context.Context, host, repo, image string) (tags []api

remaining := len(c.clients) - i - 1
if remaining == 0 {
c.log.Debugf("failed to lookup via %q, Giving up, no more clients", client.Name())
c.log.Infof("failed to lookup via %q, Giving up, no more clients", client.Name())
} else {
c.log.Debugf("failed to lookup via %q, continuing to search with %v clients remaining", client.Name(), remaining)
c.log.Infof("failed to lookup via %q, continuing to search with %v clients remaining", client.Name(), remaining)
}
}

Expand Down
138 changes: 138 additions & 0 deletions pkg/controller/kube_controller.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
package controller

import (
"context"
"fmt"
"io"
"strings"
"time"

"github.com/sirupsen/logrus"

"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"

"github.com/hashicorp/go-retryablehttp"
"github.com/jetstack/version-checker/pkg/metrics"
"github.com/jetstack/version-checker/pkg/version/semver"
)

const channelURLSuffix = "https://dl.k8s.io/release/"

type ClusterVersionScheduler struct {
client kubernetes.Interface
log *logrus.Entry
metrics *metrics.Metrics
interval time.Duration
channel string
}

func NewKubeReconciler(
log *logrus.Entry,
config *rest.Config,
metrics *metrics.Metrics,
interval time.Duration,
channel string,
) *ClusterVersionScheduler {

return &ClusterVersionScheduler{
log: log,
client: kubernetes.NewForConfigOrDie(config),
interval: interval,
metrics: metrics,
channel: channel,
}
}

func (s *ClusterVersionScheduler) Start(ctx context.Context) error {
go s.runScheduler(ctx)
return s.reconcile(ctx)
}

func (s *ClusterVersionScheduler) runScheduler(ctx context.Context) {
ticker := time.NewTicker(s.interval)
defer ticker.Stop()

s.log.WithField("interval", s.interval).WithField("channel", s.channel).
Info("ClusterVersionScheduler started")

for {
select {
case <-ctx.Done():
s.log.Info("ClusterVersionScheduler stopping")
return
case <-ticker.C:
if err := s.reconcile(ctx); err != nil {
s.log.Error(err, "Failed to reconcile cluster version")
}
}
}
}

func (s *ClusterVersionScheduler) reconcile(_ context.Context) error {
// Get current cluster version
current, err := s.client.Discovery().ServerVersion()
if err != nil {
return fmt.Errorf("getting cluster version: %w", err)
}

// Get latest stable version
latest, err := getLatestStableVersion(s.channel)
Comment thread
maria-reynoso marked this conversation as resolved.
Outdated
if err != nil {
return fmt.Errorf("fetching latest stable version: %w", err)
}

latestSemVer := semver.Parse(latest)
currentSemVer := semver.Parse(current.GitVersion)

// Create version strings without metadata for comparison
currentSemVerNoMeta := fmt.Sprintf("%d.%d.%d", currentSemVer.Major(), currentSemVer.Minor(), currentSemVer.Patch())
latestSemVerNoMeta := fmt.Sprintf("%d.%d.%d", latestSemVer.Major(), latestSemVer.Minor(), latestSemVer.Patch())

// Parse the versions without metadata for comparison
currentComparable := semver.Parse(currentSemVerNoMeta)
latestComparable := semver.Parse(latestSemVerNoMeta)

// Register metrics!
s.metrics.RegisterKubeVersion(!currentComparable.LessThan(latestComparable),
currentSemVerNoMeta, latestSemVerNoMeta,
s.channel,
)

s.log.WithFields(logrus.Fields{
"currentVersion": currentSemVerNoMeta,
"latestStable": latestSemVerNoMeta,
"channel": s.channel,
}).Info("Cluster version check complete")

return nil
}

func getLatestStableVersion(channel string) (string, error) {
Comment thread
maria-reynoso marked this conversation as resolved.
Outdated
if !strings.HasSuffix(channel, ".txt") {
channel += ".txt"
}

// We don't need a `/` here as its should be in the channelURLSuffix
channelURL := fmt.Sprintf("%s%s", channelURLSuffix, channel)
Comment thread
maria-reynoso marked this conversation as resolved.
Outdated

client := retryablehttp.NewClient()
client.RetryMax = 3
client.RetryWaitMin = 1 * time.Second
client.RetryWaitMax = 30 * time.Second
// Optional: Log using your own logrus/logr logger
client.Logger = nil

resp, err := client.Get(channelURL)
if err != nil {
return "", err
}
defer resp.Body.Close()
Comment thread
maria-reynoso marked this conversation as resolved.
Outdated

body, err := io.ReadAll(resp.Body)
Comment thread
maria-reynoso marked this conversation as resolved.
if err != nil {
return "", err
}

return strings.TrimSpace(string(body)), nil
}
21 changes: 21 additions & 0 deletions pkg/metrics/kubernetes.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package metrics

import "github.com/prometheus/client_golang/prometheus"

func (m *Metrics) RegisterKubeVersion(isLatest bool, currentVersion, latestVersion, channel string) {
m.mu.Lock()
defer m.mu.Unlock()

isLatestF := 0.0
if isLatest {
isLatestF = 1.0
}

m.kubernetesVersion.With(
prometheus.Labels{
"current_version": currentVersion,
"latest_version": latestVersion,
"channel": channel,
},
).Set(isLatestF)
}
Loading