M3 Components

M3 consists of 3 simple components, all purposefully designed for scale, reliability and efficiency.

  • arrow-pic

    Ingestion & streaming aggregation

    Ingest and aggregate a stream of metrics with dynamic configuration

  • Timeseries Database

    Purpose built time series database (M3DB) for better performance, reliability and consistent results

  • arrow-pic

    Real time query engine

    Real time, highly performant and optimized query engine

History

brand-decor
M3
brand-decor
Uber
decor-line-dot
decor-subsection-left

Intro  2014

In 2014, following several years of battling a legacy metrics platform based on Graphite and WhisperDB, the Observability team at Uber began developing an open source metrics platform that would become known as M3. The goal of the project was to provide teams with a highly available and centralized metrics platform that could be easily integrated with existing tooling.

decor-line-dot
decor-subsection-left
decor-subsection-right
subsection-icon

Storage

The first iteration of the storage layer featured Cassandra and ElasticSearch; however, as the ride-hailing business entered its hyper growth phase, the firefighting that had once plagued the team during the Graphite and WhisperDB years had resurfaced. The team then decided to build M3DB, a custom timeseries database with an embedded inverted index, from scratch which became the foundation of the platform. Over the course of about a year and a half, M3DB was developed and rolled out across Uber.

Learn more
decor-line-dot
subsection-icon

Query

OOMs and slow queries became a regular occurrence, which prevented teams from effectively monitoring their services. The new query engine, dubbed M3Query, takes advantage of the fact that M3DB stores data in highly compressed blocks, and decompresses the data client side one datapoint at a time while functions are lazily applied. These optimizations minimize network, compute and memory resources. The new query engine, which supports PromQL and the legacy Graphite language, gave teams an extremely fast and powerful way to gain insights into their metrics.

Learn more
decor-line-dot
subsection-icon

Ingestion

As the number of metrics being ingested skyrocketed, it became clear that there wasn’t a one-size-fits-all methodology when it came to data storage policies. Different teams wanted their metrics stored at different resolutions and retention periods. For example, infrastructure teams needed metrics stored at very high resolutions, but not for too long. Conversely, business teams didn’t need fine granularities, but wanted to retain data for much longer. The solution was a combination of two components, M3Coordinator and M3Aggregator, which work in tandem to ingest metrics and aggregate them in a highly customizable way all while providing high availability and efficiency.

Learn more
decor-line-dot
subsection-icon

Today

Today, the M3 project is maintained by open source contributors around the globe, and is currently used by some of the largest enterprises in the world.

Getting Started

Governance & Proposals

The M3 Technical Steering Committee

  • border-dashed
    jimjag

    @jimjag

    Uber

  • border-dashed
    martin-mao

    @martin-mao

    Chronosphere

  • border-dashed
    mway

    @mway

    Uber

  • border-dashed
    prateek

    @prateek

    Uber

  • border-dashed
    robskillington

    @robskillington

    Chronosphere

feature-background

How to Propose a Feature

If you are interested in proposing a feature for M3, click the link and follow the steps provided.

Propose Feature