Skip to content

Conversion from int64 to float64 prometheus metrics not ideal for counters #300

Open
@gsstark

Description

@gsstark

Currently all int64 metrics are converted to prometheus metrics using:

	case int64:
		return float64(v), true

That's unfortunate because it loses precision, any value over 2^53 will be imprecise.

For gauges this is still the best we can do, and the loss of precision may not matter much anyways. But for counters we can do better since resets are handled. And moreover for counters the loss of precision could be a major issue since it's the difference between two values which matters.

I would suggest we convert bigints to float64 for prometheus by just taking them mod 2^53. That means we effectively always use the full precision of the mantissa and never lose precision. For values larger than 2^53 the counter will just wrap around to 0 and but that's ok for counters, rate() et al will still work correctly.

So if you just made that something like return float64(v % uint64(1<<53)), at least for counters, then there would be no loss in precision for larger values.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions