github-mirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2026-04-22 07:46:30 +08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
11,934 Commits 358 Branches 525 Tags
784a2d2c7418f6dfd1864781d65acc4e7804a708
Commit Graph

374 Commits

Author SHA1 Message Date
Łukasz Mierzwa
870627fbed Add enable_compression scrape config option 
Currently Prometheus will always request gzip compression from the target when sending scrape requests.
HTTP compression does reduce the amount of bytes sent over the wire and so is often desirable.
The downside of compression is that it requires extra resources - cpu & memory.

This also affects the resource usage on the target since it has to compress the response
before sending it to Prometheus.

This change adds a new option to the scrape job configuration block: enable_compression.
The default is true so it remains the same as current Prometheus behaviour.

Setting this option to false allows users to disable compression between Prometheus
and the scraped target, which will require more bandwidth but it lowers the resource
usage of both Prometheus and the target.

Fixes #12319.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
 2023-11-20 12:02:55 +00:00
Julien Pivotto
84aadfc45b scrape: Added trackTimestampsStaleness configuration option 
Add the ability to track staleness when an explicit timestamp is set.
Useful for cAdvisor.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2023-10-31 16:58:42 -04:00
Bartlomiej Plotka
624b973ebf Added ability to specify scrape protocols to accept during HTTP content type negotiation. (#12738) 
* Added ability to specify scrape protocols to accept during HTTP content type negotiation.


This is done via new option in GlobalConfig and ScrapeConfig: "scrape_protocol"

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Fixed readability and log message.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
 2023-10-10 11:16:55 +01:00
Julien Pivotto
d659e672f1 Merge pull request #12816 from slashpai/lightsail 
LIGHTSAIL SD: Validate HTTP config
 2023-09-08 15:01:35 +02:00
Julien Pivotto
738e0dbaf0 Merge pull request #12814 from slashpai/puppetdb 
PUPPETDB SD: Validate HTTP config
 2023-09-08 15:00:21 +02:00
Julien Pivotto
a2208b2849 Merge pull request #12815 from slashpai/Uyuni 
UYUNI SD: Validate HTTP config
 2023-09-08 14:59:50 +02:00
Jayapriya Pai
57988fda75 LIGHTSAIL SD: Validate HTTP config 
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
 2023-09-08 17:54:58 +05:30
Jayapriya Pai
344f9d8d15 PUPPETDB SD: Validate HTTP config 
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
 2023-09-08 17:42:01 +05:30
Jayapriya Pai
34c8dcc73f UYUNI SD: Validate HTTP config 
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
 2023-09-08 17:34:41 +05:30
Julien Pivotto
75470ba815 Merge pull request #12812 from slashpai/azure 
AZURE SD: Validate HTTP config
 2023-09-08 14:03:19 +02:00
Jayapriya Pai
e44d5dfb3a AZURE SD: Validate HTTP config 
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
 2023-09-08 13:10:31 +05:30
Jayapriya Pai
7f085aeb6a EC2 SD: Validate HTTP config 
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
 2023-09-08 11:46:07 +05:30
Julien Pivotto
8498b96a2b HTTP SD: Validate HTTP config 
Fixes #12762

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2023-09-03 23:06:30 +02:00
TJ Hoplock
51d1d2cd96 feat: add AWS sigv4 support to alertmanager endpoints 
Addresses: #12536

This commit adds support for configuring sigv4 to an
`alertmanager_config`. Based heavily on the sigv4 work in the remote
write client.

Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>
 2023-08-31 21:47:25 -04:00
Bryan Boreham
1e3fef6ab0 scraping: limit detail on dropped targets, to save memory (#12647) 
It's possible (quite common on Kubernetes) to have a service discovery
return thousands of targets then drop most of them in relabel rules.
The main place this data is used is to display in the web UI, where
you don't want thousands of lines of display.

The new limit is `keep_dropped_targets`, which defaults to 0
for backwards-compatibility.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2023-08-14 15:39:25 +01:00
rakshith210
b1675e23af Add Azure AD package for remote write (#11944) 
* Add Azure AD package for remote write
* Made AzurePublic default and updated configuration.md
* Updated config structure and removed getToken at initialization
* Changed passing context from request

Signed-off-by: Rakshith Padmanabha <rapadman@microsoft.com>
Signed-off-by: rakshith210 <rakshith.me@gmail.com>
 2023-06-01 15:20:10 -06:00
Arianna Vespri
a16b876a05 Add limits to global config (#12126) 
* Add limits to global config

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Move changes into Validate func

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Make comments consistent wrt 0 meaning no limit

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Document global limits

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

---------

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>
 2023-05-30 09:22:23 +01:00
Julien Pivotto
0dc31ade41 Add support for consul path_prefix 
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2023-05-17 00:14:58 +02:00
beorn7
9e500345f3 textparse/scrape: Add option to scrape both classic and native histograms 
So far, if a target exposes a histogram with both classic and native
buckets, a native-histogram enabled Prometheus would ignore the
classic buckets. With the new scrape config option
`scrape_classic_histograms` set, both buckets will be ingested,
creating all the series of a classic histogram in parallel to the
native histogram series. For example, a histogram `foo` would create a
native histogram series `foo` and classic series called `foo_sum`,
`foo_count`, and `foo_bucket`.

This feature can be used in a migration strategy from classic to
native histograms, where it is desired to have a transition period
during which both native and classic histograms are present.

Note that two bugs in classic histogram parsing were found and fixed
as a byproduct of testing the new feature:

1. Series created from classic _gauge_ histograms didn't get the
   _sum/_count/_bucket prefix set.
2. Values of classic _float_ histograms weren't parsed properly.

Signed-off-by: beorn7 <beorn@grafana.com>
 2023-05-13 01:32:25 +02:00
Jeanette Tan
40240c9c1c Update according to code review 
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
 2023-05-05 02:33:00 +08:00
Jeanette Tan
2ad39baa72 Treat bucket limit like sample limit and make it fail the whole scrape and return an error 
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
 2023-04-22 03:25:07 +08:00
Jeanette Tan
d3ad158a66 Update docs and comments 
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
 2023-04-22 03:14:19 +08:00
Jeanette Tan
4d21ac23e6 Implement bucket limit for native histograms 
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
 2023-04-22 03:14:19 +08:00
Bryan Boreham
889ef998f4 remote-write: adjust MaxShards and Capacity for larger MaxSamplesPerSend 
Keep the target throughput and total pending samples the same.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2023-04-02 11:00:39 +01:00
Bryan Boreham
2fc2e233a6 remote-write: raise default samples per send to 2,000 
Larger messages cost less, because the per-message overheads at sender
and receiver are spread over more samples.

Example: scraping 1.5 million series every 15 seconds will send 50
messages per second.

Previous default was 500.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2023-03-31 08:47:06 +00:00
Julien Pivotto
599b70a05d Add include scrape configs 
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2023-03-06 23:35:39 +01:00
Julien Pivotto
6fc5305ce9 Merge pull request #11709 from jlevesy/allow-agent-no-remote-write 
feat(config): allow no remote write configs
 2022-12-22 23:14:40 +01:00
Bryan Boreham
623d306f91 Update package config for new labels.Labels type 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2022-12-19 15:22:09 +00:00
Julien Levesy
8c5da5e078 feat(config): allow no remote write configs 
Signed-off-by: Julien Levesy <jlevesy@gmail.com>
 2022-12-13 18:41:06 +01:00
Levi Harrison
f81fae2414 Add common HTTP client to AWS SDs (#11611) 
* Common client in EC2 and Lightsail

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Azure -> AWS

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2022-11-22 10:09:14 -05:00
Julien Pivotto
005ede70de relabel: add keepequal/dropequal relabel action 
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2022-11-10 14:41:02 +01:00
Marine Bal
16c3aa75c0 Add service discovery for OvhCloud (#10802) 
* feat(ovhcloud): add ovhcloud management

Signed-off-by: Marine Bal <marine.bal@corp.ovh.com>
Co-authored-by: Arnaud Sinays <sinaysarnaud@gmail.com>
 2022-11-03 10:20:09 +01:00
Jesus Vazquez
e934d0f011 Merge 'main' into sparsehistogram 
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>
 2022-10-05 22:14:49 +02:00
Ganesh Vernekar
f371d7f0fb Add docs for out of order ingestion (#11340) 
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
 2022-09-28 20:01:21 +05:30
Jesus Vazquez
c1b669bf9b Add out-of-order sample support to the TSDB (#11075) 
* Introduce out-of-order TSDB support

This implementation is based on this design doc:
https://docs.google.com/document/d/1Kppm7qL9C-BJB1j6yb6-9ObG3AbdZnFUBYPNNWwDBYM/edit?usp=sharing

This commit adds support to accept out-of-order ("OOO") sample into the TSDB
up to a configurable time allowance. If OOO is enabled, overlapping querying
are automatically enabled.

Most of the additions have been borrowed from
https://github.com/grafana/mimir-prometheus/
Here is the list ist of the original commits cherry picked
from mimir-prometheus into this branch:
- 4b2198d7ec
- 2836e5513f
- 00b379c3a5
- ff0dc75758
- a632c73352
- c6f3d4ab33
- 5e8406a1d4
- abde1e0ba1
- e70e769889
- df59320886

Co-authored-by: Jesus Vazquez <jesus.vazquez@grafana.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Dieter Plaetinck <dieter@grafana.com>
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* gofumpt files

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Add license header to missing files

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Fix OOO tests due to existing chunk disk mapper implementation

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Fix truncate int overflow

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Add Sync method to the WAL and update tests

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* remove useless sync

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Update minOOOTime after truncating Head

* Update minOOOTime after truncating Head

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add a unit test

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Load OutOfOrderTimeWindow only once per appender

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Fix OOO Head LabelValues and PostingsForMatchers

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Fix replay of OOO mmap chunks

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Remove unnecessary err check

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Prevent panic with ApplyConfig

Signed-off-by: Ganesh Vernekar 15064823+codesome@users.noreply.github.com
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Run OOO compaction after restart if there is OOO data from WBL

Signed-off-by: Ganesh Vernekar 15064823+codesome@users.noreply.github.com
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Apply Bartek's suggestions

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Refactor OOO compaction

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Address comments and TODOs

- Added a comment explaining why we need the allow overlapping
  compaction toggle
- Clarified TSDBConfig OutOfOrderTimeWindow doc
- Added an owner to all the TODOs in the code

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Run go format

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Fix remaining review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Change wbl reference when truncating ooo in TestHeadMinOOOTimeUpdate

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>

* Fix TestWBLAndMmapReplay test failure on windows

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Address most of the feedback

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Refactor the block meta for out of order

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix windows error

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Ganesh Vernekar 15064823+codesome@users.noreply.github.com
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Dieter Plaetinck <dieter@grafana.com>
Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2022-09-20 22:35:50 +05:30
Bryan Boreham
31bb281185 config: in tests use labels.FromStrings 
Replacing code which assumes the internal structure of `Labels`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2022-09-09 13:34:49 +02:00
Cosrider
bef6556ca5 delete redundant alias (#11180) 
Signed-off-by: Cosrider <cosrider7@gmail.com>

Signed-off-by: Cosrider <cosrider7@gmail.com>
 2022-08-31 15:50:38 +02:00
Levi Harrison
08f3ddb864 Sparse histogram remote-write support (#11001) 2022-07-14 09:13:12 -04:00
Bram Vogelaar
4456dcc26e feat(nomad): add nomad service discovery 
Signed-off-by: Bram Vogelaar <bram@attachmentgenie.com>
 2022-06-30 14:17:08 +02:00
Julien Pivotto
e4a09f2b4b uyuni: Use default HTTP client and set relative paths (#10814) 
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2022-06-08 11:29:44 +02:00
David Dymko
3ef153b00c vultr integration 
Signed-off-by: David Dymko <dymkod@gmail.com>
 2022-05-24 18:38:06 +02:00
Matthieu MOREL
8a01943abc refactor (package config): move from github.com/pkg/errors to 'errors' and 'fmt' packages (#10724) 
Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>
 2022-05-23 09:54:32 +02:00
Felix Ehrenpfort
ce3bc818a8 Add service discovery for IONOS Cloud (#10514) 
* Add service discovery for IONOS Cloud

Signed-off-by: Felix Ehrenpfort <felix@ehrenpfort.de>
 2022-05-20 14:57:23 +02:00
Julien Pivotto
71dbb4d091 Add lowercase and uppercase relabel action (#10641) 
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2022-05-03 10:09:53 +02:00
Matthieu MOREL
e2ede285a2 refactor: move from io/ioutil to io and os packages (#10528) 
* refactor: move from io/ioutil to io and os packages
* use fs.DirEntry instead of os.FileInfo after os.ReadDir

Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>
 2022-04-27 11:24:36 +02:00
Julien Pivotto
09da88114d Support overriding minimum TLS version 
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
 2022-04-20 23:41:30 +02:00
Julien Pivotto
98039cddfa Update Prometheus common (#10492) 
* Update Prometheus common

- Oauth2 supports proxy URL
- HTTP2 can be disabled

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2022-03-30 12:49:03 +02:00
David N Perkins
b13aec9167 Merge pull request #10365 from David-N-Perkins/azure-resource-group-filter 
Azure Service Discovery resource group filter
 2022-03-29 01:21:23 +02:00
Julien Pivotto
fb2da1f26a Followup on tracing (#10338) 
* Simplify code by letting common deal with empty TLS config
* Improve error message if we notice a user is putting an authorization
header into its configuration.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2022-02-22 21:44:36 +01:00
Matej Gera
0acbe5e3f5 Tracing: Add additional options to align with the upstream exporter (#10276) 
* Enhance configuration

Signed-off-by: Matej Gera <matejgera@gmail.com>
 2022-02-22 17:07:30 +01:00