Native support for stddev and variance related window functions

[fulghum]

Fixed a bug where STDDEV_POP, STDDEV_SAMP, VAR_POP, VAR_SAMP, and their variance and stddev aliases would crash the server when used as window functions over integer columns. They now return correct, Postgres-compatible numeric/double precision results instead of panicking.

Fixes: #3038

[github-actions]

	Main	PR
covering_index_scan_postgres	2138.49/s	2124.61/s	-0.7%
groupby_scan_postgres	155.67/s	154.46/s	-0.8%
index_join_postgres	695.69/s	696.63/s	+0.1%
index_join_scan_postgres	868.33/s	861.55/s	-0.8%
index_scan_postgres	32.54/s	32.45/s	-0.3%
oltp_delete_insert_postgres	750.21/s	708.79/s	-5.6%
oltp_insert	681.25/s	661.02/s	-3.0%
oltp_point_select	3653.38/s	3658.77/s	+0.1%
oltp_read_only	3523.42/s	3550.97/s	+0.7%
oltp_read_write	2641.38/s	2635.73/s	-0.3%
oltp_update_index	695.61/s	682.71/s	-1.9%
oltp_update_non_index	727.11/s	714.86/s	-1.7%
oltp_write_only	1808.35/s	1784.55/s	-1.4%
select_random_points	2248.65/s	2246.46/s	-0.1%
select_random_ranges	1648.49/s	1673.74/s	+1.5%
table_scan_postgres	32.15/s	32.08/s	-0.3%
types_delete_insert_postgres	734.27/s	741.51/s	+0.9%
types_table_scan_postgres	14.47/s	14.32/s	-1.1%

[github-actions]

	Main	PR
Total	42090	42090
Successful	18949	18966
Failures	23141	23124
Partial Successes1	5340	5340

	Main	PR
Successful	45.0202%	45.0606%
Failures	54.9798%	54.9394%

${\color{red}Regressions (2)}$

aggregates

QUERY:          SELECT avg(x::float8), var_pop(x::float8)
FROM (VALUES (100000003), (100000004), (100000006), (100000007)) v(x);
RECEIVED ERROR: row sets differ:
    Postgres:
        {100000005.000000, 2.500000}
    Doltgres:
        {100000005.000000, 4.000000}

QUERY:          SELECT avg(x::float8), var_pop(x::float8)
FROM (VALUES (7000000000005), (7000000000007)) v(x);
RECEIVED ERROR: row sets differ:
    Postgres:
        {7000000000006.000000, 1.000000}
    Doltgres:
        {7000000000006.000000, 0.000000}

${\color{lightgreen}Progressions (19)}$

aggregates

QUERY: SELECT stddev_pop(b) FROM aggtest;

QUERY: SELECT stddev_samp(b) FROM aggtest;

QUERY: SELECT var_pop(b) FROM aggtest;

QUERY: SELECT var_samp(b) FROM aggtest;

QUERY: SELECT var_pop('inf'::float8), var_samp('inf'::float8);

QUERY: SELECT stddev_pop('inf'::float8), stddev_samp('inf'::float8);

QUERY: SELECT var_pop('nan'::float8), var_samp('nan'::float8);

QUERY: SELECT stddev_pop('nan'::float8), stddev_samp('nan'::float8);

QUERY: SELECT var_pop(1.0::float4), var_samp(2.0::float4);

QUERY: SELECT stddev_pop(3.0::float4), stddev_samp(4.0::float4);

QUERY: SELECT var_pop('inf'::float4), var_samp('inf'::float4);

QUERY: SELECT stddev_pop('inf'::float4), stddev_samp('inf'::float4);

QUERY: SELECT var_pop('nan'::float4), var_samp('nan'::float4);

QUERY: SELECT stddev_pop('nan'::float4), stddev_samp('nan'::float4);

QUERY: SELECT var_pop(1.0::numeric), var_samp(2.0::numeric);

QUERY: SELECT stddev_pop(3.0::numeric), stddev_samp(4.0::numeric);

QUERY: SELECT var_pop('nan'::numeric), var_samp('nan'::numeric);

QUERY: SELECT stddev_pop('nan'::numeric), stddev_samp('nan'::numeric);

random

QUERY: (SELECT unique1 AS random
  FROM onek ORDER BY random() LIMIT 1)
INTERSECT
(SELECT unique1 AS random
  FROM onek ORDER BY random() LIMIT 1)
INTERSECT
(SELECT unique1 AS random
  FROM onek ORDER BY random() LIMIT 1);

Footnotes

1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct. ↩

[itoqa]

Commit: ca32904: 19 test cases ran, 1 failed ❌, 18 passed ✅.

Summary

Coverage spans aggregate variance and standard-deviation behavior across numeric and floating-point inputs, grouped and moving-window calculations, null and empty-frame handling, duplicate ordering values, concurrent startup, precision limits, and recovery after extreme values. The broad results are healthy for ordinary use and many boundary conditions, but an extreme floating-point input exposes an incorrect result.

Not safe to merge yet — the PR introduces a medium-severity correctness defect in grouped and window aggregates for extreme real-valued inputs, where finite variance can become NaN. Ordinary and double-precision cases remain healthy, but this is a direct, user-visible data-result error in the changed functionality rather than a test-environment issue.

Tests run by Ito

Result	Severity	Type	Description
❌		Float	Running variance queries with an extreme REAL value returned NaN for the grouped and window population variance. The expected result is a finite value, with zero for the identical extreme values used by this check.
✅	—	Aggregate	A fresh local server registered all six variance and standard-deviation names. Grouped queries returned results for every test group without an undefined-function error or startup failure.
✅	—	Aggregate	Integer and numeric columns accepted the variance queries and returned numeric results. The checked values for 10 and 20 were correct: population variance 25 and sample standard deviation 7.07106781186547524.
✅	—	Aggregate	Real and double precision values are accepted by the variance functions, and the results are returned as double precision numbers.
✅	—	Aggregate	Four isolated workers started at the same time and all finished successfully. The variance functions stayed available, with no duplicate-registration errors or inconsistent catalog state.
✅	—	Aggregate	The aggregate functions keep the expected result type in grouped queries and window queries. Integer and decimal inputs return numeric values, while real and double inputs return double-precision values.
✅	—	Float	Grouped and window queries returned the expected population and sample variance and standard deviation values for floating-point data.
✅	—	Float	Grouped and window queries returned finite results for large, nearly equal values. The clamped partition returned zero variance and zero standard deviation, with no negative or NaN results.
✅	—	Null	Grouped calculations handled empty, one-value, and two-value groups correctly. Empty groups returned NULL, population results returned zero for one value, and sample results stayed NULL until two values were present.
✅	—	Null	Grouped and full-partition window calculations ignored empty values consistently. Empty partitions stayed NULL, one real value followed the population and sample rules, and two real values produced the expected results.
✅	—	Null	Sample variance and standard deviation stayed empty when only one real value was present, even when NULL rows were added before or after it. With two real values, padded and unpadded results matched.
✅	—	Precision	Variance and standard-deviation queries returned the expected values for ordinary numbers and numbers with a very large offset. Grouped and window results agreed, and the small spread was preserved.
✅	—	Precision	Very large decimal and BIGINT values completed without a crash or incorrect error. A normal query afterward returned variance 25 and standard deviation 5, and the database stayed available.
✅	—	Precision	A high-precision standard-deviation query returned the expected finite result. The next ordinary query returned variance 25 and standard deviation 5 in both the same and a fresh session.
✅	—	Precision	Large numeric and BIGINT values that differ by only a small amount still returned the correct positive variance in grouped and window queries.
✅	—	Window	Rows with the same ordering value receive the same cumulative variance and standard-deviation results.
✅	—	Window	Empty and backward-moving windows finish safely and return SQL NULL instead of an evaluation or division error.
✅	—	Window	Variance and standard deviation queries returned the expected values for each group, and equivalent window calculations matched those results.
✅	—	Window	Window calculations completed successfully as the frame moved between rows. No stale values or evaluation errors were observed.

Tip

Reply with @itoqa to send us feedback on this test run.

[itoqa]

Extreme real values return NaN

What failed: Running variance queries with an extreme REAL value returned NaN for the grouped and window population variance. The expected result is a finite value, with zero for the identical extreme values used by this check.

Impact · Steps · Stub / mock · Analysis · Why this is likely a bug

• Severity: Medium
• Impact: Queries using extreme REAL values can return NaN instead of the correct finite variance. Users may receive wrong results for these edge-case reports, while ordinary values continue to work.
• Steps to Reproduce:
  1. Create a table with REAL and DOUBLE PRECISION columns.
  2. Insert finite ordinary values and an extreme value such as 1.0e100 into both columns.
  3. Run grouped VAR_POP and STDDEV_POP queries for each column.
  4. Run the equivalent window VAR_POP and STDDEV_POP queries.
  5. Observe that the extreme REAL results contain NaN while the extreme DOUBLE PRECISION results are finite zero.
• Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
• Code Analysis: The PR adds server/functions/aggregate/variance_aggregates.go and registers its floating-point aggregate implementation through initVarianceAggs. In floatVariance at lines 233-245, the sum-of-squares expression (nf*sumX2 - sumX*sumX) / divisor can overflow to infinity and then produce NaN when finite extreme values are equal. The only guard at lines 242-244 checks variance < 0; IEEE NaN is not less than zero, so it is returned unchanged. The grouped path calls this function from floatVarianceBuffer.Eval at lines 269-281, and the window path rebuilds the same buffer and calls Eval at lines 324-335, matching both observed failure forms. The smallest practical fix is to handle non-finite variance before the negative clamp, using a numerically stable calculation or a targeted finite-result policy for the overflow case, and add an extreme REAL regression test for both grouped and window forms. The PR diff marks variance_aggregates.go as a newly added 335-line file, so the defective path is directly introduced by this PR.
• Why this is likely a bug: The local PostgreSQL client completed successfully with exit status 0, and the raw result showed NaN only for the extreme REAL fixture in both grouped and window forms; ordinary and close fixtures were finite, and the corresponding DOUBLE PRECISION results were zero. This is therefore a deterministic SQL result defect rather than a browser, transport, or setup failure. The source inspection identifies the direct cause: overflow-prone arithmetic creates NaN and the added implementation does not reject or normalize it. Since the PR adds the entire variance implementation, the PR directly introduces the affected code path and the smallest fix is local to its floating-point variance handling.

Relevant code

server/functions/aggregate/variance_aggregates.go:230-245

func floatVariance(n int64, sumX, sumX2 float64, sample bool) float64 {
	...
	variance := (nf*sumX2 - sumX*sumX) / divisor
	if variance < 0 {
		return 0
	}
	return variance
}

server/functions/aggregate/variance_aggregates.go:269-281

variance := floatVariance(b.count, b.sumX, b.sumX2, b.sample)
if b.sqrtResult {
	return math.Sqrt(variance), nil
}
return variance, nil

server/functions/aggregate/variance_aggregates.go:324-335

b := &floatVarianceBuffer[T]{expr: w.expr, sample: w.sample, sqrtResult: w.sqrtResult}
for i := interval.Start; i < interval.End; i++ {
	if err := b.Update(ctx, buf[i]); err != nil {
		return nil, err
	}
}
return b.Eval(ctx)

Evidence Package

• text/FLOAT-3/FLOAT-3-extreme-real-variance.txt — 3 KB

Copy prompt for an agent

Ito QA identified the following failure during automated PR testing. Please investigate and propose a fix.

**Medium severity — Extreme real values return NaN**

**What failed:** Running variance queries with an extreme REAL value returned NaN for the grouped and window population variance. The expected result is a finite value, with zero for the identical extreme values used by this check.

- **Impact:** Queries using extreme REAL values can return NaN instead of the correct finite variance. Users may receive wrong results for these edge-case reports, while ordinary values continue to work.
- **Steps to reproduce:**
  1. Create a table with REAL and DOUBLE PRECISION columns.
  2. Insert finite ordinary values and an extreme value such as 1.0e100 into both columns.
  3. Run grouped VAR_POP and STDDEV_POP queries for each column.
  4. Run the equivalent window VAR_POP and STDDEV_POP queries.
  5. Observe that the extreme REAL results contain NaN while the extreme DOUBLE PRECISION results are finite zero.
- **Stub / mock content:** No stubs, mocks, or bypasses were applied for this test in the recorded run.
- **Code analysis:** The PR adds server/functions/aggregate/variance_aggregates.go and registers its floating-point aggregate implementation through initVarianceAggs. In floatVariance at lines 233-245, the sum-of-squares expression `(nf*sumX2 - sumX*sumX) / divisor` can overflow to infinity and then produce NaN when finite extreme values are equal. The only guard at lines 242-244 checks `variance < 0`; IEEE NaN is not less than zero, so it is returned unchanged. The grouped path calls this function from floatVarianceBuffer.Eval at lines 269-281, and the window path rebuilds the same buffer and calls Eval at lines 324-335, matching both observed failure forms. The smallest practical fix is to handle non-finite variance before the negative clamp, using a numerically stable calculation or a targeted finite-result policy for the overflow case, and add an extreme REAL regression test for both grouped and window forms. The PR diff marks variance_aggregates.go as a newly added 335-line file, so the defective path is directly introduced by this PR.
- **Why this is likely a bug:** The local PostgreSQL client completed successfully with exit status 0, and the raw result showed NaN only for the extreme REAL fixture in both grouped and window forms; ordinary and close fixtures were finite, and the corresponding DOUBLE PRECISION results were zero. This is therefore a deterministic SQL result defect rather than a browser, transport, or setup failure. The source inspection identifies the direct cause: overflow-prone arithmetic creates NaN and the added implementation does not reject or normalize it. Since the PR adds the entire variance implementation, the PR directly introduces the affected code path and the smallest fix is local to its floating-point variance handling.

**Relevant code:**

`server/functions/aggregate/variance_aggregates.go:230-245`

~~~go
func floatVariance(n int64, sumX, sumX2 float64, sample bool) float64 {
	...
	variance := (nf*sumX2 - sumX*sumX) / divisor
	if variance < 0 {
		return 0
	}
	return variance
}
~~~

`server/functions/aggregate/variance_aggregates.go:269-281`

~~~go
variance := floatVariance(b.count, b.sumX, b.sumX2, b.sample)
if b.sqrtResult {
	return math.Sqrt(variance), nil
}
return variance, nil
~~~

`server/functions/aggregate/variance_aggregates.go:324-335`

~~~go
b := &floatVarianceBuffer[T]{expr: w.expr, sample: w.sample, sqrtResult: w.sqrtResult}
for i := interval.Start; i < interval.End; i++ {
	if err := b.Update(ctx, buf[i]); err != nil {
		return nil, err
	}
}
return b.Eval(ctx)
~~~