György Krajcsovits
cf929d6460
fix(chunkenc): get rid of Iterator.Encoding()
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-02-18 18:51:38 +01:00
György Krajcsovits
7991bcbff9
feat(tsdb): adopt head append changes from 18026
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-02-18 18:51:33 +01:00
György Krajcsovits
7c295dbece
feat(tsdb): query of overlapping chunks with ST by source encoding
...
When querying chunks from storage, take into account the source
encoding instead of losing ST.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-02-18 18:50:00 +01:00
György Krajcsovits
e40f988f5c
feat(tsdb): allow appending to ST capable XOR chunk optionally
...
See PR description for uptodate info on details.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-02-18 18:50:00 +01:00
Owen Williams
b57f5b59b3
tsdb: ST-in-WAL: Counter implementation and benchmarks ( #17671 )
...
Initial implementation of https://github.com/prometheus/prometheus/issues/17790 .
Only implements ST-per-sample for Counters. Tests and benchmarks updated.
Note: This increases the size of the RefSample object for all users, whether st-per-sample is turned on or not.
Signed-off-by: Owen Williams <owen.williams@grafana.com >
2026-02-12 13:17:50 -05:00
Ganesh Vernekar
0ede19e2ee
Replace deprecated google.CredentialsFromJSON with option.WithAuthCredentialsFile
...
Signed-off-by: Ganesh Vernekar <ganesh.vernekar@reddit.com >
2026-02-10 13:03:14 -08:00
Bartlomiej Plotka
5e46e77754
refactor: use Appender mock for otlptranslator tests ( #17999 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-02-06 09:51:40 +00:00
Bartlomiej Plotka
c8e7f4e2a6
tests: Unify TestDiskFillingUpAfterDisablingOOO and avoid hiding errors ( #18017 )
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
Sync repo files / repo_sync (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
* tests: Unify TestDiskFillingUpAfterDisablingOOO and avoid hiding errors
Signed-off-by: bwplotka <bwplotka@gmail.com >
* addressed comments
Signed-off-by: bwplotka <bwplotka@gmail.com >
---------
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-02-05 16:11:35 +00:00
Bartlomiej Plotka
7769495a4a
refactor: switch OTLP handler to AppendableV2 ( #17996 )
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
* refactor: switch OTLP handler to AppendableV2
Signed-off-by: bwplotka <bwplotka@gmail.com >
* addressed comments
Signed-off-by: bwplotka <bwplotka@gmail.com >
---------
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-02-03 16:44:40 +00:00
Arve Knudsen
3c44ca757d
storage: fix ToError() to return nil for empty partial errors ( #18002 )
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com >
2026-02-03 16:58:02 +01:00
Bartlomiej Plotka
44d772b4e7
refactor: use Appender mock for tests ( #17992 )
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-02-02 15:56:11 +00:00
Bartlomiej Plotka
076369fad0
refactor: move OTLP handler to separate file ( #17990 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-02-02 14:36:00 +00:00
Bartlomiej Plotka
9657c23c37
storage: optimized fanoutAppenderV2 ( #17976 )
...
* storage: add BenchmarkFanoutAppenderV2
Signed-off-by: bwplotka <bwplotka@gmail.com >
* fix: optimized fanoutAppenderV2
Signed-off-by: bwplotka <bwplotka@gmail.com >
* optimized more
Signed-off-by: bwplotka <bwplotka@gmail.com >
---------
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-02-02 07:04:30 +00:00
bwplotka
6c18bba5c6
Revert "storage: add BenchmarkFanoutAppenderV2"
...
Sync repo files / repo_sync (push) Has been cancelled
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
This reverts commit 91b0f353b0
2026-01-30 15:54:52 +00:00
bwplotka
91b0f353b0
storage: add BenchmarkFanoutAppenderV2
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-01-30 15:53:59 +00:00
Arve Knudsen
dc34b90f93
otlptranslator: fix silently swallowed error in addSumNumberDataPoints ( #17954 )
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
Sync repo files / repo_sync (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
The createAttributes error was incorrectly returning nil instead of err,
causing errors to be silently discarded. This could lead to silent data
loss for sum metrics during OTLP ingestion.
Fixes #17953
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com >
2026-01-28 13:58:50 +01:00
Callum Styan
97e7ef802c
remote write: simplify readability of timeseries filtering by using the slices package ( #14318 )
...
* simplify readability of timeseries filtering by using the slices package
Signed-off-by: Callum Styan <callumstyan@gmail.com >
* ensure that BenchmarkBuildTimeSeries doesn't account for the building of
the actual proto in the benchmark results, we only care about the
buildTimeSeries call
Signed-off-by: Callum Styan <callumstyan@gmail.com >
---------
Signed-off-by: Callum Styan <callumstyan@gmail.com >
2026-01-28 11:47:34 +00:00
Ganesh Vernekar
9eb78735cf
Merge pull request #16929 from prometheus/codesome/stale-series-compaction
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Sync repo files / repo_sync (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
tsdb: Early compaction of stale series
Closes #13616
Based on https://github.com/prometheus/proposals/pull/55
Stale series tracking was added in #16925 . This PR compacts the stale series into its own block before the normal compaction hits. Here is how the settings:
stale_series_compaction_threshold: As soon as the ratio of stale series in the head block crosses StaleSeriesImmediateCompactionThreshold, TSDB performs a stale series compaction and puts all the stale series into a block and removed it from the head, but it does not remove it from the WAL. (technically this condition is checked every minute and not exactly immediate)
Additional details
WAL replay: after a stale series compaction, tombstones are added with (MinInt64, MaxInt64) for all these stale series. During WAL replay we add a special condition where when we find such tombstone, it immediately removes the series from the memory instead of storing the tombstone. This is required so that we don't spike up memory during WAL replay and also don't keep the compacted stale series in the memory.
Head block truncation ignores this block via the added metadata, similar to out-of-order blocks.
2026-01-24 15:18:17 -08:00
Arve Knudsen
2332962c4b
otlptranslator: filter __name__ from OTLP attributes to prevent duplicates ( #17917 )
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
* otlptranslator: filter __name__ from OTLP attributes to prevent duplicates
OTLP metrics can have a __name__ attribute which, when combined with the
metric name passed via extras, creates duplicate __name__ labels.
This commit implements filtering out of any __name__ metric attribute from OTLP.
Also rename TestCreateAttributes to TestPrometheusConverter_createAttributes
for consistency, and add test cases for __name__, __type__, and __unit__ OTLP metric attributes.
---------
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com >
2026-01-24 16:44:08 +01:00
Ganesh Vernekar
4f3de8da29
tsdb: Add unit tests for stale series compaction
...
Signed-off-by: Ganesh Vernekar <ganesh.vernekar@reddit.com >
2026-01-23 18:07:34 -08:00
Bartlomiej Plotka
0d116b0994
tests(teststorage): Close Storage in the helper ( #17902 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-01-23 08:41:35 +00:00
Ben Kochie
1a0c56993c
Use standard library errors in otlptranslator ( #17894 )
...
Remove direct use of `go.uber.org/multierr` in favor of standard
library `errors` package.
Signed-off-by: SuperQ <superq@gmail.com >
2026-01-20 10:13:40 +01:00
Bartlomiej Plotka
c4b0da94db
Merge pull request #17835 from prometheus/bwplotka/a2-storage-support
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Sync repo files / repo_sync (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
feat(storage)[PART4b]: add AppenderV2 to the rest of storage.Storage implementations + mock exemplar fix
2026-01-16 13:04:14 +00:00
bwplotka
f61a83bcd6
fix: appenderV2 mock exemplar appendErr injection before appending it
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-01-16 10:47:00 +00:00
Arve Knudsen
4afa76d083
OTLP: label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency ( #17860 )
...
* otlptranslator: add label caching for OTLP-to-Prometheus conversion
Add per-request caching to reduce redundant computation and allocations
during OTLP metric conversion:
1. Per-request label sanitization cache: Cache sanitized label names
within a request to avoid repeated string allocations for commonly
repeated labels like __name__, job, instance.
2. Resource-level label caching: Precompute and cache job, instance,
promoted resource attributes, and external labels once per
ResourceMetrics boundary instead of for each datapoint.
3. Scope-level label caching: Precompute and cache scope metadata labels
(otel_scope_name, otel_scope_version, etc.) once per ScopeMetrics
boundary.
4. LabelNamer instance caching: Reuse the LabelNamer struct across
datapoints within the same resource context.
These optimizations significantly reduce allocations and improve latency
for OTLP ingestion workloads with many datapoints per resource/scope.
---------
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com >
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com >
2026-01-16 10:30:05 +00:00
George Krajcsovits
ef350b2b54
Merge pull request #17840 from prometheus/krajo/st-in-chunks
...
feat(tsdb): new Append parameter and AtST interface for chunks
2026-01-16 10:32:53 +01:00
bwplotka
1d3c6210f3
add extra fanout test
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-01-15 14:13:58 +00:00
Arve Knudsen
a85a8998c9
otlptranslator: fix BenchmarkPrometheusConverter_FromMetrics ( #17866 )
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
Sync repo files / repo_sync (push) Has been cancelled
Stale Check / stale (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
The benchmark was passing appendMetadata=false to NewCombinedAppender,
which caused UpdateMetadata to never be called on the underlying
noOpAppender. This resulted in app.metadata always being 0, failing
the assertion that metadata count should be positive.
Fix by enabling metadata appending in the benchmark.
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com >
2026-01-15 09:33:16 +01:00
bwplotka
8a2921e385
addressed feedback
...
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-01-14 13:57:48 +00:00
bwplotka
49c3aea56d
feat(storage)[PART4b]: add AppenderV2 to the rest of storage.Storage impl
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2026-01-14 13:54:27 +00:00
György Krajcsovits
6137de085e
test ListSeriesIterator with ST
...
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:19 +01:00
György Krajcsovits
5ecc0e7062
test that ChainSampleIterator passes on the AtST call
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:18 +01:00
György Krajcsovits
8067b3d60a
add test coverage for buffer.go
...
I've checked that commenting out any one of the new lines produces an error.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:17 +01:00
György Krajcsovits
6647e512ad
update ExpandSamples
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:14 +01:00
György Krajcsovits
a00c0d6a66
auto update f/h/fh sample init with positional fields
...
find . -name "*.go" -type f \
-exec sed -E -i 's/((f|h|fh)Sample\{)([^,{:]+,[^,]+\})/\10, \3/g' {} +
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:11 +01:00
György Krajcsovits
1e77d9ded8
storage/buffer.go: add ST to sample types and iterators
...
Also fix the single multi line fSample definition to be one liner.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:10 +01:00
György Krajcsovits
70bc06718d
feat(tsdb): new AppenderV2 and AtST interface for chunks
...
No implementation yet. Just to test the shape of the interface.
AtST is implemented for trivial cases, anything else is hard coded
to return 0.
Ref: https://github.com/prometheus/prometheus/issues/17791
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-14 13:15:09 +01:00
Ben Kochie
836caf7b16
Refactor storage package errors ( #17846 )
...
Replace use of `tsdb/errors` with standard library `errors`.
Signed-off-by: SuperQ <superq@gmail.com >
2026-01-13 10:21:33 +01:00
Ben Kochie
03a1d7a350
Refactor storage package errors ( #17844 )
...
Replace use of `tsdb/errors` with standard library `errors`.
Signed-off-by: SuperQ <superq@gmail.com >
2026-01-13 09:53:49 +01:00
George Krajcsovits
66bdc88013
fix(remote_read): NHCB not returned over remote read samples ( #17794 )
...
NHCB is native histograms with custom buckets.
prompb is used for both remote write 1.0 and remote read. We do not
support NHCB over remote write 1.0 , however we should absolutely
support it for remote read.
Prometheus remote write 1.0 client already refuses to send NHCB.
Prometheus remote write 1.0 server accepts NHCB, but doesn't store
custom values, corrupting the result. I'm now handling NHCB correctly,
instead of refusing or corrupting.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com >
2026-01-07 08:44:57 +01:00
Ben Kochie
e14795bbf4
Remove copyright date from headers ( #17785 )
...
Remove copyright dates from various files as part of [PROM-50].
[PROM-50]: https://github.com/prometheus/proposals/blob/main/proposals/0050-remove-copyright-dates.md
Signed-off-by: SuperQ <superq@gmail.com >
2026-01-05 13:46:21 +01:00
dependabot[bot]
87401302f6
chore(deps): bump google.golang.org/api from 0.257.0 to 0.258.0 ( #17761 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.257.0 to 0.258.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.257.0...v0.258.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-version: 0.258.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-04 15:47:58 +00:00
Bryan Boreham
950f756e9e
Merge remote-tracking branch 'origin/release-3.8' into merge-3.8.1-to-main
...
Tweak RW2 change for consistency with older lines.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com >
2025-12-19 09:33:41 +00:00
bwplotka
da253bddf5
fix: ensure remote PRWv1 write handler does not send false 0 response headers
...
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Signed-off-by: bwplotka <bwplotka@gmail.com >
2025-12-15 12:27:35 +00:00
bwplotka
ad367b504b
refactor(tsdb/agent)[PART3]: add AppenderV2 support to agent
...
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Signed-off-by: bwplotka <bwplotka@gmail.com >
2025-12-13 05:17:39 +00:00
bwplotka
cd98ded6ea
test: add regression test against remote write handler bad response stats
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2025-12-12 22:43:51 +00:00
bwplotka
0b70a07572
refactor(appenderV2): add TSDB AppenderV2 implementation
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
tmp
Signed-off-by: bwplotka <bwplotka@gmail.com >
2025-12-09 10:39:43 +00:00
bwplotka
129650df9d
refactor(appenderV2): 1:1 copy of head_append.go -> head_append_v2.go (starting point)
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2025-12-09 09:53:39 +00:00
bwplotka
04696703fe
refactor(appenderV2): add AppenderV2 interface
...
Signed-off-by: bwplotka <bwplotka@gmail.com >
2025-12-09 09:53:34 +00:00
Julien
93edf912a7
Merge pull request #17630 from zjumathcode/main
...
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
Lock Threads / action (push) Has been cancelled
chore: Fix function name typo in createBatchSpan comment
2025-12-02 16:23:48 +01:00
