github-mirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2026-04-22 07:31:40 +08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
11,971 Commits 358 Branches 525 Tags
arct2
Commit Graph

30 Commits

Author SHA1 Message Date
Goutham Veeramachaneni
86729d4d7b Update exp package (#12650) 2023-09-21 22:53:51 +02:00
Bryan Boreham
ce153e3fff Replace sort.Sort with faster slices.SortFunc 
The generic version is more efficient.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2023-07-10 09:43:45 +00:00
Alan Protasio
ce6a643ee8 Changing TotalQueryableSamples from int to int64 (#10549) 
* Changing TotalQueryableSamples from int to int64

Signed-off-by: Alan Protasio <approtas@amazon.com>
 2022-04-12 01:22:25 +02:00
Andrew Bloomgarden
20dbb128d9 Switch per-step stats to serialize ints 
Per Julien's feedback on #10369, we're choosing to be consistent with
data types inside the stats structure (ints) rather than with the points
format that is part of the normal query responses (strings). We have
this option because this data cannot be NaN/Inf.

Signed-off-by: Andrew Bloomgarden <blmgrdn@amazon.com>
 2022-03-24 10:44:07 +01:00
Andrew Bloomgarden
a64b9fe323 Report PeakSamples in query statistics 
This exactly corresponds to the statistic compared against MaxSamples
during the course of query execution, so users can see how close their
queries are to a limit.

Co-authored-by: Harkishen Singh <harkishensingh@hotmail.com>
Co-authored-by: Andrew Bloomgarden <blmgrdn@amazon.com>
Signed-off-by: Andrew Bloomgarden <blmgrdn@amazon.com>
 2022-03-21 23:49:17 +01:00
Andrew Bloomgarden
ed091a1fb9 Allow replacing the stats struct before rendering JSON 
This allows other implementations to inject their own statistics that
they're gathering in data linked from the context.Context. For example,
Cortex can inject its stats.Stats value under the `cortex` key.

Signed-off-by: Andrew Bloomgarden <blmgrdn@amazon.com>
 2022-03-21 23:49:17 +01:00
Alan Protasio
606ef33d91 Track and report Samples Queried per query 
We always track total samples queried and add those to the standard set
of stats queries can report.

We also allow optionally tracking per-step samples queried. This must be
enabled both at the engine and query level to be tracked and rendered.
The engine flag is exposed via a Prometheus feature flag, while the
query flag is set when stats=all.

Co-authored-by: Alan Protasio <approtas@amazon.com>
Co-authored-by: Andrew Bloomgarden <blmgrdn@amazon.com>
Co-authored-by: Harkishen Singh <harkishensingh@hotmail.com>
Signed-off-by: Andrew Bloomgarden <blmgrdn@amazon.com>
 2022-03-21 23:49:17 +01:00
Julien Pivotto
9a2e93228e Switch to grafana/regexp everywhere (#10268) 
Let's have a consistent library for regexp.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2022-02-13 00:58:27 +01:00
Matej Gera
2c61d29b2a Tracing: Migrate to OpenTelemetry library (#9724) 
Signed-off-by: Matej Gera <matejgera@gmail.com>
 2022-01-25 11:08:04 +01:00
Paweł Szulik
f5563bfe95 tests: Move from t.Errorf and others. (Part 2) (#9309) 
* Refactor util tests.

Signed-off-by: Paweł Szulik <paul.szulik@gmail.com>
 2021-09-13 21:19:20 +02:00
Julien Pivotto
4e5b1722b3 Move away from testutil, refactor imports (#8087) 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-10-22 11:00:08 +02:00
Julien Pivotto
2b2eb79e8b Add windows tests for query logger (#6653) 
* Add windows tests
* Do not rely on time.Time in timer

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-20 13:17:11 +00:00
陈谭军
0f022aa42b fix unreasonable goimports (#6115) 
Signed-off-by: tanjunchen <2799194073@qq.com>
 2019-10-09 09:43:56 +01:00
zhulongcheng
970ef41bb6 Fix exported function comments (#6002) 
Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>
 2019-09-10 14:45:09 +01:00
Tariq Ibrahim
a188693015 Adding unit tests for the util package (#4534) 
Signed-off-by: Tariq Ibrahim <tariq.ibrahim@microsoft.com>
 2018-08-27 15:55:49 +01:00
Goutham Veeramachaneni
71855a22a4 Add tracing spans to promql (#4436) 
* Add spans to promql

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Simplify timer and span tracking.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
 2018-08-16 13:11:34 +05:30
Brian Brazil
dd6781add2 Optimise PromQL (#3966) 
* Move range logic to 'eval'

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make aggregegate range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* PromQL is statically typed, so don't eval to find the type.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Extend rangewrapper to multiple exprs

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Start making function evaluation ranged

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make instant queries a special case of range queries

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Eliminate evalString

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Evaluate range vector functions one series at a time

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make unary operators range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make binops range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Pass time to range-aware functions.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make simple _over_time functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reduce allocs when working with matrix selectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add basic benchmark for range evaluation

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse objects for function arguments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Do dropmetricname and allocating output vector only once.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add range-aware support for range vector functions with params

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise holt_winters, cut cpu and allocs by ~25%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make rate&friends range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make more functions range aware. Document calling convention.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make date functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make simple math functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Convert more functions to be range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make more functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Specialcase timestamp() with vector selector arg for range awareness

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove transition code for functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove the rest of the engine transition code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove more obselete code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove the last uses of the eval* functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove engine finalizers to prevent corruption

The finalizers set by matrixSelector were being called
just before the value they were retruning to the pool
was then being provided to the caller. Thus a concurrent query
could corrupt the data that the user has just been returned.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add new benchmark suite for range functinos

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Migrate existing benchmarks to new system

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Expand promql benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Simply test by removing unused range code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* When testing instant queries, check range queries too.

To protect against subsequent steps in a range query being
affected by the previous steps, add a test that evaluates
an instant query that we know works again as a range query
with the tiimestamp we care about not being the first step.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse ring for matrix iters. Put query results back in pool.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse buffer when iterating over matrix selectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Unary minus should remove metric name

Cut down benchmarks for faster runs.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reduce repetition in benchmark test cases

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Work series by series when doing normal vectorSelectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise benchmark setup, cuts time by 60%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Have rangeWrapper use an evalNodeHelper to cache across steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Use evalNodeHelper with functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Cache dropMetricName within a node evaluation.

This saves both the calculations and allocs done by dropMetricName
across steps.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse input vectors in rangewrapper

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse the point slices in the matrixes input/output by rangeWrapper

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make benchmark setup faster using AddFast

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Simplify benchmark code.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add caching in VectorBinop

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Use xor to have one-level resultMetric hash key

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add more benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Call Query.Close in apiv1

This allows point slices allocated for the response data
to be reused by later queries, saving allocations.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise histogram_quantile

It's now 5-10% faster with 97% less garbage generated for 1k steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make the input collection in rangeVector linear rather than quadratic

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise label_join, 1.8x faster and 11x less memory for 1k steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Expand benchmarks, cleanup comments, simplify numSteps logic.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address Fabian's comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Comments from Alin.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address jrv's comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove dead code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address Simon's comments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Rename populateIterators, pre-init some sizes

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Handle case where function has non-matrix args first

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Split rangeWrapper out to rangeEval function, improve comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Cleanup and make things more consistent

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make EvalNodeHelper public

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Fabian's comments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2018-06-04 15:47:45 +02:00
ferhat elmas
ffa673f7d8 General simplifications (#3887) 
Another try as in #1516
 2018-02-26 07:58:10 +00:00
David Kaltschmidt
8e01e7e388 Handle runtime errors as fatal test results 2017-11-19 21:15:24 +01:00
David Kaltschmidt
3c5d207a53 Changed test failure handler to Fatalf(), added comment 2017-11-19 21:06:35 +01:00
David Kaltschmidt
79caf5342e Comment styles 2017-11-17 22:33:24 +01:00
David Kaltschmidt
0a3faee526 Pushed query timings into timings field, added tests 2017-11-17 22:30:59 +01:00
David Kaltschmidt
87c46ea6c3 Renamed TotalEvalTime to EvalTotalTime 
* TotalFoo suggested a comprehensive timing, but TotalEvalTime was part
of the Exec timings, together with Queue timings
* The other option was to rename ExecTotalTime to TotalExecTime, but
 there was already ExecQueueTime, suggesting Exec to be some sort of
group
 2017-11-17 17:46:51 +01:00
David Kaltschmidt
af75ce02c1 Review feedback 
* renamed MakeQueryStats
* added stats to query() as well
* gofmt
 2017-11-16 16:30:48 +01:00
David Kaltschmidt
c93e54d240 Adds execution timer stats to the range query 
API consumers should be able to get insight into the query run times.
The UI currently measures total roundtrip times. This PR allows for more
fine grained metrics to be exposed.

* adds new timer for total execution time (queue + eval)

* expose new timer, queue timer, and eval timer in stats field of the
 range query response:
```json
{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [],
    "stats": {
      "execQueueTimeNs": 4683,
      "execTotalTimeNs": 2086587,
      "totalEvalTimeNs": 2077851
    }
  }
}
```

* stats field is optional, only set when query parameter `stats` is not
empty

Try it via
```sh
curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true'
```

Review feedback

* moved query stats json generation to query_stats.go
* use seconds for all query timers
* expose all timers available
* Changed ExecTotalTime string representation from Exec queue total time to Exec total time
 2017-11-16 16:05:10 +01:00
Alberto Cortés
2c9739162e utils/stats: remove dead code 2017-09-26 16:32:45 +02:00
Julius Volz
3bfec97d46 Make the storage interface higher-level. 
See discussion in
https://groups.google.com/forum/#!topic/prometheus-developers/bkuGbVlvQ9g

The main idea is that the user of a storage shouldn't have to deal with
fingerprints anymore, and should not need to do an individual preload
call for each metric. The storage interface needs to be made more
high-level to not expose these details.

This also makes it easier to reuse the same storage interface for remote
storages later, as fewer roundtrips are required and the fingerprint
concept doesn't work well across the network.

NOTE: this deliberately gets rid of a small optimization in the old
query Analyzer, where we dedupe instants and ranges for the same series.
This should have a minor impact, as most queries do not have multiple
selectors loading the same series (and at the same offset).
 2016-07-25 13:59:22 +02:00
beorn7
c740789ce3 Improve predict_linear 
Fixes https://github.com/prometheus/prometheus/issues/1401

This remove the last (and in fact bogus) use of BoundaryValues.

Thus, a whole lot of unused (and arguably sub-optimal / ugly) code can
be removed here, too.
 2016-02-25 12:10:55 +01:00
Julius Volz
9b6d69610a Fix various typos in comments. 
Helpfully reported by
https://goreportcard.com/report/github.com/prometheus/prometheus :)
 2016-02-10 03:47:00 +01:00
Fabian Reinartz
0de6edbdfc Move pkg/ to util/ 2015-06-01 21:12:32 +02:00