For ScrapeType is 'object' extract Labels not from Child object but from Parent

[SelAnt]

Proposal - when ScrapeType is 'object' extract Labels not from Child object but from Parent as for ScrapeType is 'value'

[rustycl0ck]

This use case is valid. A large example of a json object, where this is required, is mentioned in #85 as well as in #132.

TL;DR -

An example json:

{
  "name": "global-counter",
  "category": "living-things",
  "beings":
  [
    {
      "kind": "animals",
      "specifics":
      [
        {
          "noun": "lion",
          "population": 123
        },
        {
          "noun": "deer",
          "population": 456
        }
      ]
    },
    {
      "kind": "fish",
      "specifics":
      [
        {
          "noun": "shark",
          "population": 789
        },
        {
          "noun": "tuna",
          "population": 890
        }
      ]
    }
  ]
}

Right now, for the population metrics, it is possible to only attach the noun label, but not possible to attach any labels from the parent objects (like kind or category).

Currently, this PR only partially allows to attach the labels from the parents. For example, it would be possible to attach the category label, but not the kind or the noun label anyhow still.
If you can make changes to address both the use cases somehow, I'd be happy to review this PR.

[iceman91176]

The general problem ist that the jsonpath implementation does not allow to access parent nodes (https://kubernetes.io/docs/reference/kubectl/jsonpath/) This seems to be the case with all jsonpath implementations i've been looking at.

IMHO i think that this problem can't be solved easily, BUT....

A possible solution would be transforming the JSON before extracting labels and values. This approch has been described in #132 and implemented in https://github.com/fredr/data-exporter. It would be very powerful to add a feature like transformation-steps to json_exporter.

To keep it simple start with JQ for transformations, and add more transformation-types over time

Based on your data above it would look like this

modules:
  default:
    metrics:
    - name: example_value
      type: object
      transformations:
      - type: jq
        query:  |-
           [.category as $category | .beings[] | (.kind as $kind | .specifics[] | {category: $category,kind:$kind,noun: .noun, val: .population} )]
      help: Example of sub-level value scrapes from a json
      path: '{[*]}'
      labels:
        category: '{.category}'
        kind: '{.kind}'
        noun: '{.noun}'
      values:
        count: '{.val}'

The jq-query turns your example-data into

[
  {
    "category": "living-things",
    "kind": "animals",
    "noun": "lion",
    "val": 123
  },
  {
    "category": "living-things",
    "kind": "animals",
    "noun": "deer",
    "val": 456
  },
  {
    "category": "living-things",
    "kind": "fish",
    "noun": "shark",
    "val": 789
  },
  {
    "category": "living-things",
    "kind": "fish",
    "noun": "tuna",
    "val": 890
  }
]

And the scrape returns

# HELP example_value_count Example of sub-level value scrapes from a json
# TYPE example_value_count untyped
example_value_count{category="living-things",kind="animals",noun="deer"} 456
example_value_count{category="living-things",kind="animals",noun="lion"} 123
example_value_count{category="living-things",kind="fish",noun="shark"} 789
example_value_count{category="living-things",kind="fish",noun="tuna"} 890

Unfortunately i don't know anything about golang, so this would either take ages and/or or be a complete failure if i tried to implement it. I think the transformatiosn should be added before here (

json_exporter/cmd/main.go

Line 123 in 1c1ae57

jsonMetricCollector.Data = data

)

@rustycl0ck - what do you think ? Maybe @SelAnt is willing to add this as described ?

[corpocott]

any movement on this? currently running into a problem this would solve.

[MattK360]

I created a PR to add transformations: #333