Perfetto shows GPU events as bars after the timestamp, should they not be before the timestamp

[dfriederich]

GPU events are captured on the device as gpu_counter_event's. That event has a timestamp and a list of counter ids/values

packet {
  timestamp_clock_id: 5
  timestamp: 179681168735066
  gpu_counter_event {
    counters {
      counter_id: 0
      double_value: 183495.000000
    }
....
    counters {
      counter_id: 65673
      double_value: 0.000000
    }
  }
  trusted_uid: 2000
  trusted_packet_sequence_id: 6
  trusted_pid: 8586
}

What it does not have is an explicit duration, or end time. Instead the duration has to be derived from two sequential gpu_counter_event's.

The issue I see in the perfetto UI is that it takes the value at the first event, and draws a bar until the time of the next event. That is conceptually simple, but has the issue that all the counted events the bar represents did not happen in the time period perfetto shows the bar, all of them did happen in the time period since the prior (and not the next) gpu_counter_event.
This can be seen in the display, for example on a Pixel, the gpu events have a minimal configurable duration of about 1 ms.
That is long in CPU/GPU time, for example the bitmap below, I did enable some vertex events. Vertex events are happening on the GPU in the 'non-fragment' time slices, the quite small green bars in the first track in the bitmap.
However all the counts for those vertex events are shown in perfetto in the 1ms after the event was reported, hence after all the vertex (non-fragment) work did end.

compare this to how AGI displays the same data

it also uses rectangles to display counter values, but it picks the value of the end event for the box height (not the start as perfetto does). This results in that all the visualized events are shown in the time span during which the non-fragment GPU code did run.

PS: I do work on sokatoa which uses perfetto as its base for displaying the timeline. So for us displaying the counter values at the right time is important.

[LalitMaganti]

There was a very long internal thread about this in Feb about this very issue where the conclusion was some internal team wanted to send a patch to fix this issue in trace processor by changing how these events are parsed

However that was conditioned on "when they get the time to do this". I think that never ended up happening. Let me send the patch to fix this given I basically know exactly what needs to be done.

[LalitMaganti]

Send a CL at #1631. @dfriederich can you PTAL?

Also while I have you, can I ask: is it guaranteed that all non-percentage counter vales are delta encoded? Should we also undo the delta encoding in trace processor also since I'm here?

[LalitMaganti]

Seems like even AGI does not undo deltafication from reading its source code: it just displays the counter values directly.

I think we could change the counter mode to be a "rate" as well in the Perfetto UI if we knew it was useful for a given counter track type but given we don't I guess we'll just keep it as deltas in both the trace processor and the UI.

[dfriederich]

That's a lot for the quick fix. It was merged before I had the time to look at the PR :-).

Now the counts are below the non-fragment vertex queue entries :-)

Should we also undo the delta encoding in trace processor also since I'm here?

not sure what 'undo the delta encoding in trace processor' references, can you provide a link?

I don't think there is a reliable way to know if a specific counter has the number of events which happen since the last event, vs the counter represents a 'full' value say a ratio, or number used memory in bytes. For some counters we could know, say if they have a unit like a ratio (events/second).

[LalitMaganti]

Now the counts are below the non-fragment vertex queue entries :-)

Great!

not sure what 'undo the delta encoding in trace processor' references, can you provide a link?

Never mind, see below.

specific counter has the number of events which happen since the last event, vs the counter represents a 'full' value say a ratio, or number used memory in bytes

Right yes exactly this is what I was asking about! I know there are numerator_unit and denominator_unit in the protos but a) they don't seem to be filled very often and b) for number of events since last event (what I called "delta" encoding), there's no difference in units vs a full value from what I can tell.