-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Description
Is your feature request related to a problem? Please describe.
Currently, Loki allows filtering logs by labels or by message content using operators like |=, !=, |, and !. However, it is difficult to apply conditional filtering based on multiple label combinations, where inclusion/exclusion of log lines depends on multiple labels and their relation to log content.
In practice, this leads to situations like:
Only include messages containing "Failed login" when logger="auth-service" and severity="high".
Exclude "heartbeat" debug messages when logger="monitoring-service" and severity="debug".
Keep "ERROR_[0-9]{3}" messages for logger="payment-service" but not for other services.
Currently, these queries require multiple separate queries or external post-processing, which is inefficient and error-prone.
Describe the solution you'd like
I propose a new operator called labasearch (short for “label-based search”), which allows filtering log lines based on label combinations. The key features would be:
-
Filtering logs for specific labels or label combinations.
-
Applying line filters with &|, &!, &
, &!that always apply only to the preceding labasearch. -
Joining multiple labasearch expressions in a single pipeline to define complex conditional rules.
Example using multiple joined labasearch calls:
# Include only failed logins from auth-service with high severity
{job="production"}
| labasearch {logger="auth-service", severity="high"} &| "Failed login"
# Exclude heartbeat messages from monitoring-service with debug severity
| labasearch {logger="monitoring-service", severity="debug"} &! "heartbeat"
# Include error codes for payment-service only
| labasearch {logger="payment-service"} &~ "ERROR_[0-9]{3}"
Behavior:
labasearch stands for label-based search, i.e., it filters logs based on the labels specified inside {…}.
Each &|, &!, &, &! applies only to the preceding labasearch.
Multiple labasearch steps can be combined to express complex conditional filters per label set.
Lines not matching a labasearch are passed through the pipeline unaffected, allowing selective inclusion/exclusion per label set.
Describe alternatives you've considered
Using multiple separate queries in Grafana and combining results manually – inefficient and harder to maintain.
Post-processing logs externally – adds operational complexity, latency, and maintenance overhead.
Additional context
This feature would extend LogQL in a natural way, staying consistent with current pipeline semantics. Users could define fine-grained, label-aware inclusion/exclusion rules in a single query, which is especially useful in large, multi-service environments with structured logs.