1188 Commits

Author SHA1 Message Date
beorn7
5d4db805ac Merge branch 'main' into sparsehistogram 2021-11-17 19:57:31 +01:00
beorn7
9de3ab60df promql: improve histogram support in engine.go
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-16 13:20:24 +01:00
Matthew
7ee52bfb86
feat: linreg => slope 0 for const values (#9728)
* feat: linreg => slope 0 for const values

Signed-off-by: mtfoley <mtfoley.mae@gmail.com>
2021-11-16 00:03:22 +01:00
beorn7
73858d7f82 storage: histogram support in memoized_iterator
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-15 21:55:58 +01:00
beorn7
9b30ca2598 promql: Support histogram in value string representation
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-15 20:36:44 +01:00
beorn7
4c28d9fac7 Move to histogram.Histogram pointers
This is to avoid copying the many fields of a histogram.Histogram all
the time.

This also fixes a bunch of formerly broken tests.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-12 23:17:35 +01:00
beorn7
f1065e44a4 model: String method for histogram.Histogram
This includes a regular bucket iterator and a string method for
histogram.Bucket.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-11 17:29:22 +01:00
Thomas Jackson
f0003bc0ba
Don't drop ParenExpr when creating StepInvariantExpr (#9591)
* Add test case to showcase the problem in #9590

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>

* Don't unwrap ParenExpr in newStepInvariantExpr

Fixes #9590

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2021-11-10 20:16:24 +05:30
beorn7
c954cd9d1d Move packages out of deprecated pkg directory
This creates a new `model` directory and moves all data-model related
packages over there:
  exemplar labels relabel rulefmt textparse timestamp value

All the others are more or less utilities and have been moved to `util`:
  gate logging modetimevfs pool runtime

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
beorn7
8f92c90897 Add TODOs and some minor tweaks
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-07 17:12:04 +01:00
Dieter Plaetinck
cda025b5b5
TSDB: demistify SeriesRefs and ChunkRefs (#9536)
* TSDB: demistify seriesRefs and ChunkRefs

The TSDB package contains many types of series and chunk references,
all shrouded in uint types.  Often the same uint value may
actually mean one of different types, in non-obvious ways.

This PR aims to clarify the code and help navigating to relevant docs,
usage, etc much quicker.

Concretely:

* Use appropriately named types and document their semantics and
  relations.
* Make multiplexing and demuxing of types explicit
  (on the boundaries between concrete implementations and generic
  interfaces).
* Casting between different types should be free.  None of the changes
  should have any impact on how the code runs.

TODO: Implement BlockSeriesRef where appropriate (for a future PR)

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* feedback

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* agent: demistify seriesRefs and ChunkRefs

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
2021-11-06 15:40:04 +05:30
Ganesh Vernekar
c8b267efd6
Get histograms from TSDB to the rate() function implementation
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-11-03 19:04:18 +05:30
Mateusz Gozdek
b7bdf6fab2 Fix imports formatting
According to
2829908806 (r58457095).

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2021-11-02 19:52:34 +01:00
Mateusz Gozdek
1a6c2283a3 Format Go source files using 'gofumpt -w -s -extra'
Part of #9557

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2021-11-02 19:52:34 +01:00
darshanime
42d786f1ac use kahan summation for aggregation functions
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-10-30 19:41:36 +05:30
darshanime
694b872dee address stylistic nits
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-10-30 19:08:23 +05:30
darshanime
a905354da3 use kahan for avg_over_time
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-10-30 19:04:18 +05:30
darshanime
0a9deb9597 use kahan summation for numerical stability
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-10-30 19:04:18 +05:30
Björn Rabenstein
c092a74be9
Merge pull request #9553 from prometheus/beorn7/histogram
promql: Add more test cases for `histogram_quantile`
2021-10-21 00:33:11 +02:00
beorn7
c00c6ed01f promql: Add more test cases for histogram_quantile
This is motivated by the doubts raised in #7970.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-20 16:15:44 +02:00
Bryan Boreham
a278ea4b58
promql: copy data when short-circuiting (#9552)
* promql: copy data when short-circuiting

Because the range query loop re-uses the output buffer each time round,
we must copy results into the buffer rather than using input as output.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-10-20 16:03:02 +02:00
beorn7
a9008f5423 Merge branch 'main' into sparsehistogram 2021-10-19 17:14:23 +02:00
Julien Pivotto
c812ee794e
PromQL: Comment flaky test (#9545)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-10-19 13:13:56 +02:00
Julien Pivotto
a18224d02d
make aggregations deterministic (#9459)
* Add deterministic test for aggregations

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Make aggregations deterministic

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Increase testing

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-10-17 15:16:38 +05:30
ziollek
55f9147b44
Add atan2 to scalar operators - issue #9485 (#9515)
* Add atan2 to scalar operators

Signed-off-by: Tomasz Ziolkowski <tomasz.ziolkowski@allegro.pl>
2021-10-15 16:03:11 +02:00
beorn7
7a8bb8222c Style cleanup of all the changes in sparsehistogram so far
A lot of this code was hacked together, literally during a
hackathon. This commit intends not to change the code substantially,
but just make the code obey the usual style practices.

A (possibly incomplete) list of areas:

* Generally address linter warnings.

* The `pgk` directory is deprecated as per dev-summit. No new packages should
  be added to it. I moved the new `pkg/histogram` package to `model`
  anticipating what's proposed in #9478.

* Make the naming of the Sparse Histogram more consistent. Including
  abbreviations, there were just too many names for it: SparseHistogram,
  Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in
  general. Only add "Sparse" if it is needed to avoid confusion with
  conventional Histograms (which is rare because the TSDB really has no notion
  of conventional Histograms). Use abbreviations only in local scope, and then
  really abbreviate (not just removing three out of seven letters like in
  "Histo"). This is in the spirit of
  https://github.com/golang/go/wiki/CodeReviewComments#variable-names

* Several other minor name changes.

* A lot of formatting of doc comments. For one, following
  https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences
  , but also layout question, anticipating how things will look like
  when rendered by `godoc` (even where `godoc` doesn't render them
  right now because they are for unexported types or not a doc comment
  at all but just a normal code comment - consistency is queen!).

* Re-enabled `TestQueryLog` and `TestEndopints` (they pass now,
  leaving them disabled was presumably an oversight).

* Bucket iterator for histogram.Histogram is now created with a
  method.

* HistogramChunk.iterator now allows iterator recycling. (I think
  @dieterbe only commented it out because he was confused by the
  question in the comment.)

* HistogramAppender.Append panics now because we decided to treat
  staleness marker differently.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-11 13:02:03 +02:00
beorn7
fd5ea4e0b5 Merge branch 'main' into sparsehistogram 2021-10-07 23:16:42 +02:00
Levi Harrison
587cb95093 Add operator tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-23 10:30:46 -04:00
Levi Harrison
f5c6ee4df6 Add lex test
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-23 10:30:46 -04:00
Levi Harrison
7471208b5c Add ATAN2 to maybe_label
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-23 10:30:46 -04:00
Levi Harrison
8547a2bd86 Add atan2 binary operator
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-23 10:30:46 -04:00
Bryan Boreham
5a754bc043
Short-circuit vector binary ops (#9362)
In degenerate cases we can save the effort of building a map.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-09-21 17:37:36 +05:30
Bryan Boreham
c4942ef3b7
Optimise query_range by computing join signatures just once (#9360)
* Add benchmark case for many-to-one join

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* query_range: compute join signatures just once

For an expression like `a + on(p,q) b`, extract the `p,q` part from each
series once, instead of re-computing at every step of the range.

Although there was a cache, computing the key by concatenating all
labels was expensive.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-09-21 15:58:39 +05:30
Bryan Boreham
7d105277fe
Optimise topk where k==1 (#9365)
* Add benchmark for query_range with topk

Modify sample data so values within a metric differ

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Optimise topk where k==1

In this case we don't need a heap to keep track of values; just a single
slot is fine.

Simplify the initialization of the heap: since all cases start off as a
single-item heap we can just assign the value directly.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Allow at least one slot in results for topk, quantile

k isn't set for quantile, but we need space to start collecting values

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-09-21 15:57:28 +05:30
Levi Harrison
4ac4e2ca4b Remove parenthesis
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
6faca22eec Add inverse hyperbolic functions
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
9c54ee0a6e Changed function comment
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
53d88fd147 Added hyperbolic trig functions
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
a8ad569db8 Change spaces to tabs
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
ecc29d38b8 Fixed pi definition
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
9fc7ba33aa Remove atan2()
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
f1d94cc5f6 Split and shortened tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
d5ecdc4597 Remove variadic
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:10 -04:00
Levi Harrison
814cd5844d Added tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:09 -04:00
Levi Harrison
309e020c0a Added functions
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 22:47:09 -04:00
Paweł Szulik
f5563bfe95
tests: Move from t.Errorf and others. (Part 2) (#9309)
* Refactor util tests.

Signed-off-by: Paweł Szulik <paul.szulik@gmail.com>
2021-09-13 21:19:20 +02:00
Oleg Zaytsev
0a43e788af
Allow VectorSelector.String() without matchers (#9282)
* Allow VectorSelector.String() without matchers

Previously this method was panicking because it was trying to allocate a
slice with capacity -1. There's nothing saying that VectorSelector
should have matchers, and it's actually prepared to have zero matcher
strings, so it's worth checking instead of panicking.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2021-09-01 09:48:18 +02:00
SuperQ
e167a45c65
Add new Go build tags.
Add new go:build comments based on 1.17 formatting[0].

[0]: https://golang.org/doc/go1.17#gofmt

Signed-off-by: SuperQ <superq@gmail.com>
2021-08-27 10:24:14 +02:00
Julien Pivotto
cab96a06ef
Merge release 2.29 in main (#9196)
* PromQL: Fix start and end keywords masking label and metric names

This commit fixes an issue with the "at modifier" that introduced two
new keywords: `start` and `end`. In grouping options and in metric
names, these keywords took precedence over metric or label names, so
that those metrics and labels could no longer be referenced.

Signed-off-by: Clayton Peters <clayton.peters@man.com>

* Add in additional tests for metrics and/or labels called start/end.

Signed-off-by: Clayton Peters <clayton.peters@man.com>

* *: Cut 2.29.0-rc.0

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* VERSION: bump to 2.29.0-rc.0

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* Remove experimental wording on size-based retention

Followup of #9004

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Fix PR reference in changelog

Signed-off-by: George Brighton <george@gebn.co.uk>

* Describe EC2 availability zone IDs at most once per refresh (#9142)

Signed-off-by: George Brighton <george@gebn.co.uk>

* Describe EC2 availability zones at most once per SD load

Closes #9142.

Signed-off-by: George Brighton <george@gebn.co.uk>

* Incorporate feedback

Signed-off-by: George Brighton <george@gebn.co.uk>

* Integrate feedback

Signed-off-by: George Brighton <george@gebn.co.uk>

* Add a compatibility note for macOS users.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* *: Cut v2.29.0-rc.1

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* Fix `kuma_sd` targetgroup reporting (#9157)

* Bundle all xDS targets into a single group

Signed-off-by: austin ce <austin.cawley@gmail.com>

* *: cut v2.29.0-rc.2

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* Rename links

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* bump codemirror-promql to 0.17.0

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* *: cut v2.29.0

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* tsdb: align atomically accessed int64 (#9192)

This prevents a panic in 32-bit archs:
https://pkg.go.dev/sync/atomic#pkg-note-BUG

Fixed #9190

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Release 2.29.1 (#9193)

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

Co-authored-by: Clayton Peters <clayton.peters@man.com>
Co-authored-by: Frederic Branczyk <fbranczyk@gmail.com>
Co-authored-by: George Brighton <george@gebn.co.uk>
Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Augustin Husson <husson.augustin@gmail.com>
2021-08-12 18:38:06 +02:00
Ganesh Vernekar
8b70e87ab9
Merge remote-tracking branch 'upstream/main' into sparse-refactor
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-08-05 12:16:08 +05:30