Introduction

In our previous discussions Kubernetes Monitoring with Prometheus & PrometheusOperator & KubePrometheus , we focused on the implementation of Prometheus, Prometheus-Operator, and kube-prometheus.

However, as we scaled and our needs evolved, we encountered certain limitations inherent to Prometheus, especially for the long term metrics store.

That’s when I check Thanos.io an Open source, highly available Prometheus setup with long term storage capabilities.

What is Thanos

Thanos emerges as a solution to these challenges, extending Prometheus in several critical areas:

• Global Query View: Thanos provides a unified view of data across multiple Prometheus instances. This feature is crucial for HA deployments, as it ensures consistent query results irrespective of which Prometheus instance the data originates from.
• Unlimited Retention: With Thanos, the constraint of local storage is overcome. It integrates with various cloud storage solutions, allowing for practically unlimited data retention.
• Prometheus Compatibility: Thanos is designed to work seamlessly with existing Prometheus setups. It enhances Prometheus's capabilities without requiring significant changes to the existing monitoring infrastructure.
• Downsampling and Compaction: Thanos implements downsampling, reducing the resolution of historical data to save storage space while maintaining the essence of the data for longer-term trends. It also compacts data over time, optimizing storage utilization and query efficiency.

Thanos Components & Architecture

Thanos has a lot components, which might initially seem daunting. However it’s smart designed to split components with distinct and decoupled purposes. Once we understand the principle we can easily understand each components.

Write Path

Let’s look at the write path first, different components share same goal, write metrics chunks from different sources to the chosen storage.

The Storage*

Thanos uses object storage as primary storage for metrics and metadata related to them. It support major object storages by cloud providers.

Thanos Sidecar

Thanos Sidecar runs on the existing prometheus pod as a sidecar, it directly access the data chunks created by Prometheus and upload to storage