Skip to content

Commit 3086ef0

Browse files
committed
update text
1 parent 9e9afb8 commit 3086ef0

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

docs/design/4940-reliable-loki-pipelines.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@
88

99
Alloy's Loki pipelines currently use channels, which limits throughput due to head-of-line blocking and can cause silent log drops during config reloads or shutdowns.
1010

11-
This proposal introduces a function-based pipeline using a `Consumer` interface, replacing the channel-based design. Source components will call `Consume()` directly on downstream components, enabling parallel processing and returning errors that sources can use for retry logic or proper HTTP error responses.
11+
This proposal introduces a function-based pipeline using a `Consumer` or `Appender` interface, replacing the channel-based design.
12+
13+
Source components will call functions directly on downstream components, enabling parallel processing and returning errors that sources can use for retry logic or proper HTTP error responses.
1214

1315
## Problem
1416

@@ -34,6 +36,8 @@ loki.write "loki" {}
3436

3537
`loki.source.file` will tail all files from targets and compete to send on the channel exposed by `loki.process`. Only one entry will be processed by each stage configured in `loki.process`. If a reload happens or if Alloy is shutting down, logs could be silently dropped.
3638

39+
There is also no way to abort entries in the pipeline. This is problematic when using components such as `loki.source.api` where caller could cancel request due to e.g. timeouts.
40+
3741
## Proposal 0: Do nothing
3842
This architecture works in most cases, it will be hard to use slow components such as `secretfilter` because a lot of the time it's too slow.
3943
It's also hard to use Alloy as a gateway for loki pipelines with e.g. `loki.source.api` due to the limitations listed above.

0 commit comments

Comments
 (0)