github-mirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2026-04-30 23:11:34 +08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
8,137 Commits 358 Branches 529 Tags
fix-2.21-latency
Commit Graph

237 Commits

Author SHA1 Message Date
Bartlomiej Plotka
567e8dff8c Optimize chainSampleIterator for valid cases. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-09-04 15:02:19 +01:00
Bartlomiej Plotka
ba7233d33a Reused chunk iterators. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-09-04 13:04:29 +01:00
Bartlomiej Plotka
065432398b Added Realdata benchmark. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-09-04 11:45:18 +01:00
Bartlomiej Plotka
03d2b8df47 Added stub. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-09-03 22:02:20 +01:00
Bartlomiej Plotka
f8affc94fd Fix. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-09-03 19:28:09 +01:00
Bartlomiej Plotka
cacf1ccff2 Extended SelectQuerier benchmarks to reproduce regression. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-09-03 17:55:00 +01:00
Ganesh Vernekar
c806262206 Fix 'chunks.HeadReadWriter: maxt of the files are not set' error (#7856) 
* Fix chunks.HeadReadWriter: maxt of the files are not set

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2020-08-26 19:59:18 +02:00
Yukun Sun
cfd4e05c9e fix: return a corruption error when iterator function find a chunk that is out of sequence (#7855) 
Signed-off-by: sunyukun <sunyukun@didiglobal.com>

Co-authored-by: sunyukun <sunyukun@didiglobal.com>
 2020-08-26 20:36:27 +05:30
Max Neverov
bb5c6b38e2 Fix Possible Race Condition in TSDB (#7815) 
* Replace tsdb chunk mapper size with atomic; protect mmappedChunkFiles with read path mutex on DeleteCorrupted

Signed-off-by: Max Neverov <neverov.max@gmail.com>

* PR fixes

Signed-off-by: Max Neverov <neverov.max@gmail.com>
 2020-08-26 14:22:48 +05:30
johncming
3e7b463908 add testcase for rangeForTimestamp. (#6454) 
Signed-off-by: johncming <johncming@yahoo.com>
 2020-08-25 16:16:43 +01:00
Guangwen Feng
9439b2150d Fix golint warning caused by misspell (#7842) 
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
 2020-08-25 09:17:41 +01:00
johncming
b521612042 tsdb: simplify code. (#7792) 
Signed-off-by: johncming <johncming@yahoo.com>
 2020-08-14 15:15:08 +05:30
gastonqiu
5a7d398d19 Chore: Log segment number when segment read failed (#7687) 
* Chore: Log segment number when segment read failed

To manually fix the WAL files, it is good to know where the corrupt
happened so we should log the segment number when the read failed.

Related Issue #7506

Signed-off-by: gaston.qiu <gaston.qiu@umbocv.com>
 2020-08-13 11:01:21 -07:00
johncming
d19fc71903 tsdb: use NewRangeHead instead. (#7793) 
Signed-off-by: johncming <johncming@yahoo.com>
 2020-08-13 10:55:35 +01:00
Harold Dost
21a753c4e2 Make file permissions set to allow for wider umask options. (#7782) 
0644 -> 0666 on all non vendored code.

Fixes #7717

Signed-off-by: Harold Dost <harolddost@gmail.com>
 2020-08-12 23:23:17 +02:00
Bartlomiej Plotka
f16cbc20d6 tsdb: Bug fix for further continued deletions after crash deletions; added more tests. (#7777) 
* tsdb: Bug fix for further continued after crash deletions; added more tests.

Additionally: Added log line for block removal.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comment.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-08-11 15:53:23 +01:00
Bartlomiej Plotka
4ae2ef94e0 tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 
## Changes:

* Rename dir when deleting
* Ignoring blocks with broken meta.json on start (we do that on reload)
* Compactor writes <ulid>.tmp-for-creation blocks instead of just .tmp
* Delete tmp-for-creation and tmp-for-deletion blocks during DB open.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-08-11 06:56:08 +01:00
Zhou Hao
40ace418d1 fix misspell (#7764) 
Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
 2020-08-07 08:57:25 +01:00
Frederic Branczyk
e0cf219f0d tsdb: Save allocations on labels by re-using label array 
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
 2020-08-05 10:27:14 +02:00
Robert-André Mauchin
ed6ce7ac98 Convert int to string using rune() (#7707) 
See https://github.com/golang/go/issues/32479

Fix #7706.

Signed-off-by: Robert-André Mauchin <zebob.m@gmail.com>
 2020-08-03 15:10:04 +01:00
Bartlomiej Plotka
28c5cfaf0d tsdb: Moved code merge series and iterators to differen files; cleanup. No functional changes just move! (#7714) 
I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (:

## Changes

* Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code.
* Moved `fanout test` code from weird separate package to storage.
* Unskiped one test: TestFanout_SelectSorted/chunk_querier
* Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go`



Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-08-03 11:32:56 +01:00
johncming
ac677ed8b3 promql: delete redundant return value. (#7721) 
Signed-off-by: johncming <johncming@yahoo.com>
 2020-08-03 10:45:53 +01:00
Julien Pivotto
30e079bbd5 TSDB: Fix master tests (#7705) 
Now appenders take a context.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-07-31 17:33:54 +02:00
Bartlomiej Plotka
e6d7cc5fa4 tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 
* tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating.

Chained to https://github.com/prometheus/prometheus/pull/7059

* NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it.
* Added single SeriesEntry / ChunkEntry for all series implementations.
* Unified all vertical, and non vertical for compact and querying to single
merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before)
* Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples.
* Refactored endpoint tests and querier tests to include subtests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comments from Brian and Beorn.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed snapshot test and added chunk iterator support for DBReadOnly.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed race when iterating over Ats first.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed populate block tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed endpoints test.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed test.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added test & fixed case of head open chunk.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed DBReadOnly tests and bug producing 1 sample chunks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added cases for partial block overlap for multiple full chunks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added extra tests for chunk meta after compaction.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed small vertical merge bug and added more tests for that.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-07-31 16:03:02 +01:00
Annanay
48b9afd14b Address comments 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-30 17:25:51 +05:30
Annanay
0b4d448d29 Fix tests 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-30 17:18:47 +05:30
Annanay
263d2aa5f5 Fix failing tests 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-30 17:06:56 +05:30
Annanay
9bba8a6eae Merge branch 'master' into appender-context 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-30 16:43:18 +05:30
Annanay
89129cd39a Address comments 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-30 16:41:13 +05:30
Javier Palomo Almena
b58a613443 Replace sync/atomic with uber-go/atomic (#7683) 
* storage: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* web: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* notifier: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* cmd: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* scripts: Verify that we are not using restricted packages

It checks that we are not directly importing 'sync/atomic'.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* Reorganise imports in blocks

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* notifier/test: Apply PR suggestions

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* storage/remote: avoid storing references on newEntry

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* Revert "scripts: Verify that we are not using restricted packages"

This reverts commit 278d32748e.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* web: Group imports accordingly

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
 2020-07-30 13:15:42 +05:30
Javier Palomo Almena
348ff4285f tsdb: Replace sync/atomic with uber-go/atomic in tsdb (#7659) 
* tsdb/chunks: Replace sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb/heaad: Replace sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* vendor: Make go.uber.org/atomic a direct dependency

There is no modifications to go.sum and vendor/ because
it was already vendored.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb: Remove comments referring to the sync/atomic alignment bug

Related: https://golang.org/pkg/sync/atomic/#pkg-note-BUG

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
 2020-07-28 10:12:42 +05:30
Annanay
f40e4579b7 gofmt 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-24 20:40:19 +05:30
Annanay
7f98a744e5 Add context to Appender interface 
Signed-off-by: Annanay <annanayagarwal@gmail.com>
 2020-07-24 19:40:51 +05:30
Ben Ye
50c261502e add tsdb cmds into promtool (#6088) 
Signed-off-by: yeya24 <yb532204897@gmail.com>

update tsdb cli in makefile and promu

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove building tsdb bin

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove useless func

Signed-off-by: yeya24 <yb532204897@gmail.com>

refactor analyzeBlock

Signed-off-by: yeya24 <yb532204897@gmail.com>

Fix Makefile

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2020-07-23 19:35:50 +01:00
johncming
9801f52b0a tsdb/chunks: fix bug of data race(#7643). (#7646) 
Signed-off-by: johncming <johncming@yahoo.com>
 2020-07-23 18:05:19 +05:30
Ganesh Vernekar
4a8531a64b BlocksToDelete function in DB options (#7638) 
* Optional retention filter for DB

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Specify len for the map creation

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2020-07-22 20:49:33 +05:30
Julien Pivotto
ffc925dd21 TSDB: Error when we commit/rollback twice (#7593) 
* TSDB: Error when we commit/rollback twice

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-07-22 11:57:38 +02:00
Julien Pivotto
cfe30a7b62 TSDB: Use t.Cleanup to delete temporary files (#7620) 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-07-21 10:39:02 +02:00
Julien Pivotto
62805b2fe9 tsdb: test for leaks (#7566) 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-07-21 10:08:06 +02:00
Krasimir Georgiev
ccab2b30c9 Test no panic after a WAL corruption (#7625) 
* no panic the head memseries has chunks in it

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>

* fix a panic when querying after a wal corruption.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>

* review nits

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>

* Add test for reading the data after a wal corruption.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>

Update tsdb/db_test.go

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

Update tsdb/db_test.go

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>

* spellings

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
 2020-07-21 12:32:13 +05:30
Julien Pivotto
9b8cc663f7 Merge pull request #7623 from prometheus/release-2.20 
Release 2.20
 2020-07-20 19:16:06 +02:00
Krasi Georgiev
d30492cbb0 Avoid panic when the headChunk is nil during isolation. 
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2020-07-20 18:23:18 +03:00
Zhou Hao
ddedf454d0 add os.RemoveAll err verification (#7540) 
* add os.RemoveAll err verification for watcher_test

Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>

* add os.RemoveAll err verification for db_test

Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>

* add os.RemoveAll err verification for write_test

Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>

* add os.RemoveAll err verification for queue_manager_test

Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>

* tsdb/wal/watcher_test: add close operation before delete

Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
 2020-07-17 11:47:32 +05:30
Ganesh Vernekar
1760c7474c Replay m-map chunks irrespective of WAL (#7589) 
* Replay m-map chunks irrespective of WAL

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* More logs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2020-07-16 18:34:08 +05:30
Björn Rabenstein
e0067a7bd8 Merge pull request #7573 from codesome/mmap-empty-files 
Avoid empty mmap files by using .tmp files to write headers
 2020-07-16 12:13:34 +02:00
Ganesh Vernekar
b8a7e80f9b Fix review comments 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2020-07-16 12:43:27 +05:30
Ganesh Vernekar
ea013343ca Log when starting to create a checkpoint (#7581) 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2020-07-15 19:15:37 +05:30
Ganesh Vernekar
7a763ff61e Avoid empty mmap files by using .tmp files to write headers 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2020-07-14 14:59:28 +05:30
Bartlomiej Plotka
823b218e1b Fixed race between compact (gc, populate) and head append causing unknown symbol error. (#7560) 
* Fixed race between compact (gc, populate) and head append causing unknown symbol error.

Fixes https://github.com/prometheus/prometheus/issues/7373

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-07-14 09:36:22 +01:00
Bartlomiej Plotka
492061b24c Revert "Fix unknown symbol error during head compaction (#7526)" (#7556) 
This reverts commit 30505a202a.
 2020-07-11 22:37:16 +05:30