Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge master into features/rules-proxy #2623

Merged
merged 198 commits into from
May 20, 2020

Conversation

s-urbaniak
Copy link
Contributor

This is a merge commit to get all the changes from master into the feature branch so we can iterate.

bwplotka and others added 30 commits February 28, 2020 14:07
* Corrected all Prometheus possessives to read `Prometheus's`, this matches Prometheus's own documentation.
* Corrected `simple` to `simply` when describing compactor scanning behaviour

Signed-off-by: Peter Avdjian <[email protected]>
* Skip deleting files that we just deleted

We see this happening with Swift. Because the consistency of swift is eventual, swift sometimes didn't process the deletion of the meta file yet, and so it turns up in the bkt.Iter(). The second deletion then causes a 404 and compaction fails.

Signed-off-by: Wim Fournier <[email protected]>

* return, as this is a func. Add debug log and comment

Signed-off-by: Wim Fournier <[email protected]>

* fixing build: wrong parameter name

Signed-off-by: Wim Fournier <[email protected]>

* fix lint

Signed-off-by: Wim Fournier <[email protected]>

* Refactor deleteDir into deleteDirRec and add a parameter for a function that allows to keep certain files.

Signed-off-by: Wim Fournier <[email protected]>

* Fix lint

Signed-off-by: Wim Fournier <[email protected]>

* implementing suggested fixes

Signed-off-by: Wim Fournier <[email protected]>
This makes the handling of web.route-prefix more similar to the
behavior in Prometheus.  Correctly handles '/' and prefixes which
do not begin with a '/'.

Signed-off-by: Paul Gier <[email protected]>
* Create release v0.11.0-rc.0 (thanos-io#2156)

* Update version to v0.11.0-rc.0

* Update CHANGELOG with all PRs for v0.11

* Improve CHANGELOG by being more explicit

* Bumped minio-go library to v6.0.49, fixing an IAM bug in v6.0.45 (thanos-io#2189)

Signed-off-by: Kraig Amador <[email protected]>

* Create release candidate  v0.11.0-rc.1 (thanos-io#2192)

Signed-off-by: Matthias Loibl <[email protected]>

* Release v0.11.0 (thanos-io#2205)

Signed-off-by: Matthias Loibl <[email protected]>

* Update VERSION to 0.12.0-dev

Signed-off-by: Matthias Loibl <[email protected]>

* Resolve go.sum merge conflict and run go mod tidy

Signed-off-by: Matthias Loibl <[email protected]>

Co-authored-by: Kraig Amador <[email protected]>
* returns error messages when trigger reload with http

Signed-off-by: arthur yang <[email protected]>

* use simple reloadRules function instead of magic chan error error

Signed-off-by: yapo.yang <[email protected]>

* add tailing period for comment

Signed-off-by: yapo.yang <[email protected]>

* fix comment

Signed-off-by: arthur yang <[email protected]>

* add white space for better code reading

Signed-off-by: arthur yang <[email protected]>

* collect thanos rule metrics into one struct

Signed-off-by: arthur yang <[email protected]>

* remove termination logic and keep log only

Signed-off-by: arthur yang <[email protected]>

* update changelog for thanos-io#1848

Signed-off-by: arthur yang <[email protected]>

* add tailing period

Signed-off-by: arthur yang <[email protected]>

* check whether registry is nil

Signed-off-by: arthur yang <[email protected]>

* tailing period in metrics

Signed-off-by: arthur yang <[email protected]>

* cancel with context

Signed-off-by: arthur yang <[email protected]>

* return ctx.Err() instead of errors.New

Signed-off-by: arthur yang <[email protected]>

* register thanos rule metrics with promauto

Signed-off-by: arthur yang <[email protected]>

* return errs before set success related metrics

Signed-off-by: arthur yang <[email protected]>

* revert go.sum go.mod change

Signed-off-by: arthur yang <[email protected]>

* reload webhandler/sighup in one for loop

Signed-off-by: arthur yang <[email protected]>

* reload with chan chan error

Signed-off-by: yapo.yang <[email protected]>
Signed-off-by: mcsammac

Date:      Wed Mar 4 13:50:17 2020 -0500
On branch master
Changes to be committed:
	modified:   pkg/prober/intrumentation.go

Signed-off-by: s320009 <[email protected]>
…ed unnecessary printfs. (thanos-io#2228)

Fixes: thanos-io#2102

Also blocked them on CI side, thanks to fatih/faillint#8

Signed-off-by: Bartlomiej Plotka <[email protected]>
… full queue and dropped alerts (thanos-io#2238)

* Add test for alert queue Pop after multiple Push

Signed-off-by: Robin Clarke-Williams <[email protected]>

* Fix alert queue bug by resignal after Pop (thanos-io#2204)

Signed-off-by: Robin Clarke-Williams <[email protected]>

* Fix alert queue test and simplify

Signed-off-by: Robin Clarke-Williams <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Robin Clarke-Williams <[email protected]>

* Link to thanos-io/thanos PR in CHANGELOG.md

Signed-off-by: Robin Clarke-Williams <[email protected]>
* Fix pkg/errors error directive issues

Signed-off-by: Kemal Akkoyun <[email protected]>

* Fix misuse of Errorf

Signed-off-by: Kemal Akkoyun <[email protected]>
You would have to query `prometheus_tsdb_head_series` instead of `sum(prometheus_tsdb_head_series)` in order to get the 5 results when deduplicating.

Signed-off-by: John Chen <[email protected]>
* docs: Document Thanos Sharding

Signed-off-by: Xiang Dai <[email protected]>

* Add time partitioning

Signed-off-by: Xiang Dai <[email protected]>

* feedback

Signed-off-by: Xiang Dai <[email protected]>
…kgroud (thanos-io#2272)

* Sharding: document supported relabel action and add store gateway background

Signed-off-by: Xiang Dai <[email protected]>

* add hashmod

Signed-off-by: Xiang Dai <[email protected]>
…ls conversion. (thanos-io#2230)

## Changes

* method TranslateLables CPU Optimized (streamed sorting).
* All store GW label conversation to []storepb.Label are now alloc-less.

```
go test -bench=BenchmarkUnsafeVSSafeLabelsConversion -run=^$ -benchmem -timeout 2h -benchtime 10s ./pkg/store/storepb/...
 goos: linux
 goarch: amd64
 pkg: github.com/thanos-io/thanos/pkg/store/storepb
 BenchmarkUnsafeVSSafeLabelsConversion/safe-12         	   34822	    339076 ns/op	  655368 B/op	       2 allocs/op
 BenchmarkUnsafeVSSafeLabelsConversion/unsafe-12       	1000000000	         2.32 ns/op	       0 B/op	       0 allocs/op
PASS
```

TODO: Do the same on Querier.

Signed-off-by: Bartlomiej Plotka <[email protected]>
* scripts/genflagdocs.sh: Generate downsample flag

Signed-off-by: Xiang Dai <[email protected]>

* Document downsample component

Signed-off-by: Xiang Dai <[email protected]>

* Move downsample as bucket sub-command

Signed-off-by: Xiang Dai <[email protected]>

* update docs

Signed-off-by: Xiang Dai <[email protected]>

* feedback

Signed-off-by: Xiang Dai <[email protected]>
pstibrany and others added 21 commits May 7, 2020 17:39
* Added generic cache interface.

Signed-off-by: Peter Štibraný <[email protected]>

* Added memcached implementation of Cache.

Signed-off-by: Peter Štibraný <[email protected]>

* Chunks-caching bucket.

Signed-off-by: Peter Štibraný <[email protected]>

* Fix sentences

Signed-off-by: Peter Štibraný <[email protected]>

* Fix sentences

Signed-off-by: Peter Štibraný <[email protected]>

* Fix sentences

Signed-off-by: Peter Štibraný <[email protected]>

* Rename config objects.

Signed-off-by: Peter Štibraný <[email protected]>

* Review feedback.

Signed-off-by: Peter Štibraný <[email protected]>

* Review feedback.

Signed-off-by: Peter Štibraný <[email protected]>

* Added metrics for object size.

Signed-off-by: Peter Štibraný <[email protected]>

* Added requested chunk bytes metric.

Signed-off-by: Peter Štibraný <[email protected]>

* Caching bucket docs.

Signed-off-by: Peter Štibraný <[email protected]>

* Fixed tests.

Signed-off-by: Peter Štibraný <[email protected]>

* Fix test.

Signed-off-by: Peter Štibraný <[email protected]>

* Update docs/components/store.md
Update pkg/store/cache/caching_bucket.go

Co-authored-by: Marco Pracucci <[email protected]>
Signed-off-by: Peter Štibraný <[email protected]>

* Dots

Signed-off-by: Peter Štibraný <[email protected]>

* Always set lastBlockOffset.

Signed-off-by: Peter Štibraný <[email protected]>

* Merged cached metric into fetched metric, added labels.

Signed-off-by: Peter Štibraný <[email protected]>

* Added CHANGELOG.md entry

Signed-off-by: Peter Štibraný <[email protected]>

* Reworded help for thanos_store_bucket_cache_fetched_chunk_bytes_total

Signed-off-by: Peter Štibraný <[email protected]>

* Added tracing around getRangeChunkFile method.

Signed-off-by: Peter Štibraný <[email protected]>

* Updated CHANGELOG.md

Signed-off-by: Peter Štibraný <[email protected]>

* Options

Signed-off-by: Peter Štibraný <[email protected]>

* Fix parameter name. (store. got dropped by accident)

Signed-off-by: Peter Štibraný <[email protected]>

* Use embedded Bucket

Signed-off-by: Peter Štibraný <[email protected]>

* Added comments.

Signed-off-by: Peter Štibraný <[email protected]>

* Fixed comment.

Signed-off-by: Peter Štibraný <[email protected]>

* Hide store.caching-bucket.config flags.

Signed-off-by: Peter Štibraný <[email protected]>

* Renamed block to subrange.

Signed-off-by: Peter Štibraný <[email protected]>

* Renamed block to subrange.

Signed-off-by: Peter Štibraný <[email protected]>

* Header

Signed-off-by: Peter Štibraný <[email protected]>

* Added TODO

Signed-off-by: Peter Štibraný <[email protected]>

* Removed TODO, in favor of creating issue.

Signed-off-by: Peter Štibraný <[email protected]>

* Use NopCloser.

Signed-off-by: Peter Štibraný <[email protected]>

Co-authored-by: Marco Pracucci <[email protected]>
Changes mainly made for consistency, like section headers being in imperative tense: "do this thing" instead of "this is the thing"

Signed-off-by: Stephen Weber <[email protected]>
The subcommand is called now `tools bucket web` after the recent
changes.

Without this, the quickstart script outputs:
```
Error parsing commandline arguments: expected command but got "bucket"
thanos: error: expected command but got "bucket"
```

Signed-off-by: Giedrius Statkevičius <[email protected]>
* ui: add React UI from upstream Prometheus

Signed-off-by: Adrien Fillon <[email protected]>

* ui: incorporate new changes from Prometheus React UI

Signed-off-by: Prem Kumar <[email protected]>

* ui: adapted the React UI to Thanos

Signed-off-by: Prem Kumar <[email protected]>

Co-authored-by: Adrien Fillon <[email protected]>
Co-authored-by: Giedrius Statkevičius <[email protected]>
Signed-off-by: Pierre-Yves Aillet <[email protected]>
* react: graph/panel: revert changes temporarily

Signed-off-by: Giedrius Statkevičius <[email protected]>

* react-app: apply 'Update React vendoring'

Add the commit
prometheus/prometheus@65a1942
via a 3-way merge.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* ui/react-app: update yarn deps

Should fix security warnings. Ported from
prometheus/prometheus@24ecae9.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* ui: update bindata

Signed-off-by: Giedrius Statkevičius <[email protected]>
Found out that there is some weird interaction between `jest --coverage`
and `babel-plugin-istanbul`. Maybe related to:
jestjs/jest#6827.

From my testing, removing `--coverage` makes this work again. Probably
worth investigating in the future why that happens.

Also, this is really not needed during CI because we do not use the
coverage data anywhere anyway.

Signed-off-by: Giedrius Statkevičius <[email protected]>
* ci: test React UI using GitHub actions

Signed-off-by: Prem Kumar <[email protected]>

* ci: remove react-app-test from CircleCI as we now use GH Actions

Signed-off-by: Prem Kumar <[email protected]>
Unfortunately, all passes. ):

Signed-off-by: Bartlomiej Plotka <[email protected]>
* Added caching for Iter.

Signed-off-by: Peter Štibraný <[email protected]>

* Added cache for Exists call for meta-files.

Signed-off-by: Peter Štibraný <[email protected]>

* Added cache for reading block metadata files.

Signed-off-by: Peter Štibraný <[email protected]>

* Make caching bucket configurable with different caches for different type of objects.

Signed-off-by: Peter Štibraný <[email protected]>

* Fixed tests.

Signed-off-by: Peter Štibraný <[email protected]>

* Added caching for ObjectSize. Enabled caching of index.

Signed-off-by: Peter Štibraný <[email protected]>

* Lint feedback.

Signed-off-by: Peter Štibraný <[email protected]>

* Use single set of metrics for all operations.

Signed-off-by: Peter Štibraný <[email protected]>

* Constants.

Signed-off-by: Peter Štibraný <[email protected]>

* Use operation specific config. Generic configuration is only for user.

Signed-off-by: Peter Štibraný <[email protected]>

* Fix typo, make lint happy.

Signed-off-by: Peter Štibraný <[email protected]>

* Simplify constants.

Signed-off-by: Peter Štibraný <[email protected]>

* Simplify caching configuration.

Signed-off-by: Peter Štibraný <[email protected]>

* Refactor cache configuration.

Configuration is now passed to the cache when created.

Signed-off-by: Peter Štibraný <[email protected]>

* Review feedback.

Signed-off-by: Peter Štibraný <[email protected]>

* Fix operationRequests and operationHits for getRange.

Signed-off-by: Peter Štibraný <[email protected]>

* Make codec for Iter results configurable.

Signed-off-by: Peter Štibraný <[email protected]>

* Added header.

Signed-off-by: Peter Štibraný <[email protected]>

* Renamed "dir" config to "blocks-iter".

Signed-off-by: Peter Štibraný <[email protected]>

* Bump default values for meta exists/doesntExist ttls.

Signed-off-by: Peter Štibraný <[email protected]>

* Removed example how cache could be configured for index.

Signed-off-by: Peter Štibraný <[email protected]>

* Address review feedback.

Signed-off-by: Peter Štibraný <[email protected]>

* Get now implements streaming reader, and buffers object in memory.

Signed-off-by: Peter Štibraný <[email protected]>

* Added test for partial read.

Signed-off-by: Peter Štibraný <[email protected]>

* Removed unused function.

Signed-off-by: Peter Štibraný <[email protected]>
* Update store.go

Signed-off-by: Harshitha Chowdary <[email protected]>

* Update store.go

Signed-off-by: Harshitha Chowdary <[email protected]>

* Update cmd/thanos/store.go

Co-authored-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: Harshitha Chowdary <[email protected]>

* Updated the docs folder

Signed-off-by: Harshitha Chowdary <[email protected]>

Co-authored-by: Bartlomiej Plotka <[email protected]>
Co-authored-by: johncming <[email protected]>
Signed-off-by: Lili Cosic <[email protected]>

Co-authored-by: johncming <[email protected]>
* mixin/thanos: align mixin schema with other mixins

- promote default.libsonnet to main config file
- include config.libsonnet in mixin.libsonnet
- include jsonnetfile.json
- allow generating directly from mixin/thanos directory

Signed-off-by: paulfantom <[email protected]>

* mixin: move one directory higher

Signed-off-by: paulfantom <[email protected]>
)

* Documentation and CHANGELOG.md entry for thanos-io#2579.

Signed-off-by: Peter Štibraný <[email protected]>

* Document hidden option to configure caching bucket, to make it less hidden.

Signed-off-by: Peter Štibraný <[email protected]>

* Removed mention of caching_config.

Signed-off-by: Peter Štibraný <[email protected]>

* Added periods.

Signed-off-by: Peter Štibraný <[email protected]>
…ates. (thanos-io#2548)

* querier: Adjust deduplication for counters when querying for PromQL rates.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Added CHANGELOG.md

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Improved description as suggested by Frederic.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Moved err adjust to be per replica, not per already deduped value.

Signed-off-by: Bartlomiej Plotka <[email protected]>
@s-urbaniak
Copy link
Contributor Author

cc @bwplotka

@s-urbaniak s-urbaniak force-pushed the features/rules-proxy branch 2 times, most recently from 2df82c7 to c9f25f7 Compare May 18, 2020 13:41
Signed-off-by: Bartlomiej Plotka <[email protected]>
@bwplotka bwplotka merged commit d6a5cc1 into thanos-io:features/rules-proxy May 20, 2020
@s-urbaniak s-urbaniak deleted the features/rules-proxy branch May 20, 2020 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.