Skip to content

Commit a2a6978

Browse files
merge with upstream
2 parents 6fca1c0 + d4a660b commit a2a6978

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+2016
-472
lines changed

ADOPTERS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ We're proud to be used by the most innovative teams around the world! If you're
1717
| [BforeAI](https://bfore.ai) | [User Story](https://metalbear.com/mirrord/stories/bfore-ai/) |
1818
| [Utila](https://utila.io) | [User Story](https://metalbear.com/mirrord/stories/utila/) |
1919
| [Savvy Security](https://www.savvy.security) | [User Story](https://metalbear.com/mirrord/stories/savvy/) |
20+
| [HM Courts & Tribunal Service](https://www.gov.uk/government/organisations/hm-courts-and-tribunals-service) | [Use Case](https://github.com/hmcts/civil-citizen-ui/blob/master/README.md#development--debugging-environment---preview-with-mirrord) |
2021

2122
\* Optional: link to a blog post, tweet, internal write-up, or leave blank.
2223

CHANGELOG.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,70 @@ This project uses [*towncrier*](https://towncrier.readthedocs.io/) and the chang
88

99
<!-- towncrier release notes start -->
1010

11+
## [3.174.0](https://github.com/metalbear-co/mirrord/tree/3.174.0) - 2025-11-20
12+
13+
14+
### Added
15+
16+
- Added HM Courts & Tribunal Service to ADOPTERS.md.
17+
- Added a new option under `mirrord ci start` to run the binary and wait for it
18+
to complete.
19+
20+
21+
### Changed
22+
23+
- Allow an overlap in the configuration for `incoming.ports` and
24+
`incoming.http_filter.ports`.
25+
26+
27+
### Fixed
28+
29+
- Fixed minor edge case where intproxy might hit code that's supposed to be
30+
unreachable and panic
31+
[#intproxy-panic](https://github.com/metalbear-co/mirrord/issues/intproxy-panic)
32+
- Fixed a DNS regression that happens when user's DNS library bind their
33+
outgoing
34+
socket to unspecified address before send UDP message to the nameserver.
35+
36+
## [3.173.2](https://github.com/metalbear-co/mirrord/tree/3.173.2) - 2025-11-18
37+
38+
39+
### Changed
40+
41+
- Exclude `GOPATH` and `GOMODCACHE` from env fetched
42+
43+
## [3.173.1](https://github.com/metalbear-co/mirrord/tree/3.173.1) - 2025-11-16
44+
45+
46+
### Internal
47+
48+
- Add cleanup of unnecessary items in CI builder for docker builds.
49+
[#add-cleanup-before-docker-build](https://github.com/metalbear-co/mirrord/issues/add-cleanup-before-docker-build)
50+
51+
## [3.173.0](https://github.com/metalbear-co/mirrord/tree/3.173.0) - 2025-11-16
52+
53+
54+
### Changed
55+
56+
- Update `experimental.non_blocking_tcp_connect` config default to `false`.
57+
58+
59+
### Fixed
60+
61+
- Fixed the security advisories of the config's template engine by updating it
62+
to the newest version
63+
[#3657](https://github.com/metalbear-co/mirrord/issues/3657)
64+
65+
66+
### Internal
67+
68+
- Fix the way release script gets workspace version.
69+
- Kubernetes-related packages have been removed from the devshell, they should
70+
be installed system-wide instead.
71+
- Use the pip from the setup action, not the one preinstalled on the runner.
72+
- make CI workflow trigger for PRs targeted to `windows-support` branch
73+
(windows main)
74+
1175
## [3.172.0](https://github.com/metalbear-co/mirrord/tree/3.172.0) - 2025-11-06
1276

1377

CONTRIBUTING.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Make sure to take a look at the project's [style guide](STYLE.md).
1717
- [Adding new target types](#adding-new-target-types)
1818
- [Testing the release workflow](#testing-the-release-workflow)
1919
- [Architecture](#architecture)
20+
- [Release mirrord](#release-mirrord)
2021

2122
# Getting Started
2223

@@ -660,7 +661,7 @@ Add a changelog file in `changelog.d/` named `<identifier>.<category>.md`
660661
**Category:**
661662
Check `towncrier.toml` for available categories (`added`, `changed`, `fixed`, etc.) and choose the one that fits your change.
662663
663-
## Architecture
664+
# Architecture
664665
665666
A high level view of mirrord.
666667
@@ -755,3 +756,27 @@ flowchart TB
755756
style Pod fill:#e6ffe6,stroke:#006600,stroke-width:2px
756757
style Binary fill:#f9f9f9,stroke:#333,stroke-width:2px
757758
```
759+
760+
# Release mirrord
761+
762+
## Release PR
763+
764+
1. Create a new branch named after the new version, e.g. `3.333.0`. This will trigger additional CI jobs.
765+
2. On the new branch, bump the workspace version in `Cargo.toml` and run `cargo update -w` to update `Cargo.lock`.
766+
3. Generate the changelog with: `towncrier build --version <new-version>`.
767+
4. Review the generated changelog and fix any issues or typos.
768+
5. Push the release branch and open a PR.
769+
770+
**Note:** All the steps above can also be completed by running: `./scripts/release.sh 3.333.0`.
771+
Before running the script, ensure there are no uncommitted changes in your repository.
772+
773+
## Create a new GitHub release
774+
775+
1. After the release PR is merged, create a new GitHub release with a new tag. Use the new version for both
776+
the tag name and the release title. Use the changelog from the release PR as the release description,
777+
excluding the `Internal` section if present.
778+
779+
**Note**: Ensure the tag is attached to the release commit.
780+
781+
2. Creating the release will trigger the `Release` workflow, which builds and publishes all artifacts, including images.
782+
3. When the `Release` workflow completes successfully, update the relevant environment variables in the analytics server.

Cargo.toml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ resolver = "2"
3636
# latest commits on rustls suppress certificate verification
3737
[workspace.package]
3838
edition = "2024"
39-
version = "3.173.1"
39+
version = "3.174.0"
4040
license = "MIT"
4141
readme = "README.md"
4242
repository = "https://github.com/metalbear/mirrord"
@@ -116,6 +116,10 @@ fancy-regex = { version = "0.14" }
116116
enum_dispatch = "0.3"
117117
dotenvy = "0.15"
118118

119+
# Used by `agent`, `protocol`
120+
serde_json_path = "0.7.2"
121+
serde_json_path_macros = "0.1.6"
122+
119123
# If you update `hyper`, check that `h2` version is compatible in `intproxy/Cargo.toml`.
120124
# There is a test for this: `cargo test -p mirrord-intproxy hyper_and_h2_versions_in_sync`
121125
hyper = { version = "1", features = ["full"] }
@@ -203,7 +207,7 @@ fs4 = { version = "0.13", features = ["tokio"], default-features = false }
203207
# Used by `cli`, `sip`
204208
hex = "0.4"
205209

206-
# Used by `config`, `protocol`.
210+
# Used by `config`, `protocol`, `agent`
207211
strum = "0.27.1"
208212
strum_macros = "0.27.1"
209213

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Change tracing to not emit ansi format logs in the terminal
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added support for filtering incoming HTTP requests by running JSONPath queries on the body and matching the results against a regex.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added release guide in CONTRIBUTING.md.

mirrord-schema.json

Lines changed: 70 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -306,7 +306,7 @@
306306
"additionalProperties": false
307307
},
308308
"AgentFileConfig": {
309-
"description": "Configuration for the mirrord-agent pod that is spawned in the Kubernetes cluster.\n\n**Note:** this configuration is ignored when using the mirrord Operator. Agent configuration is done by the cluster admin.\n\nWe provide sane defaults for this option, so you don't have to set up anything here.\n\n```json { \"agent\": { \"log_level\": \"info\", \"json_log\": false, \"namespace\": \"default\", \"image\": \"ghcr.io/metalbear-co/mirrord:latest\", \"image_pull_policy\": \"IfNotPresent\", \"image_pull_secrets\": [ { \"secret-key\": \"secret\" } ], \"ttl\": 30, \"ephemeral\": false, \"communication_timeout\": 30, \"startup_timeout\": 360, \"network_interface\": \"eth0\", \"flush_connections\": false, \"exclude_from_mesh\": false \"inject_headers\": false, } } ```",
309+
"description": "Configuration for the mirrord-agent pod that is spawned in the Kubernetes cluster.\n\n**Note:** this configuration is ignored when using the mirrord Operator. Agent configuration is done by the cluster admin.\n\nWe provide sane defaults for this option, so you don't have to set up anything here.\n\n```json { \"agent\": { \"log_level\": \"info\", \"json_log\": false, \"namespace\": \"default\", \"image\": \"ghcr.io/metalbear-co/mirrord:latest\", \"image_pull_policy\": \"IfNotPresent\", \"image_pull_secrets\": [ { \"secret-key\": \"secret\" } ], \"ttl\": 30, \"ephemeral\": false, \"communication_timeout\": 30, \"startup_timeout\": 360, \"network_interface\": \"eth0\", \"flush_connections\": false, \"exclude_from_mesh\": false \"inject_headers\": false, \"max_body_buffer_size\": 65535, \"max_body_buffer_timeout\": 1000 } } ```",
310310
"type": "object",
311311
"properties": {
312312
"annotations": {
@@ -450,6 +450,26 @@
450450
"null"
451451
]
452452
},
453+
"max_body_buffer_size": {
454+
"title": "agent.max_body_buffer_size {#agent-max_body_buffer_size}",
455+
"description": "Maximum size, in bytes, of HTTP request body buffers. Used for temporarily storing bodies of incoming HTTP requests to run body filters. HTTP body filters will not match any requests with bodies larger than this.",
456+
"type": [
457+
"integer",
458+
"null"
459+
],
460+
"format": "uint32",
461+
"minimum": 0.0
462+
},
463+
"max_body_buffer_timeout": {
464+
"title": "agent.max_body_buffer_timeout {#agent-max_body_buffer_timeout}",
465+
"description": "Maximum timeout, in milliseconds, for receiving HTTP request bodies. HTTP body filters will not match any requests whose bodies do not arrive within this timeout.",
466+
"type": [
467+
"integer",
468+
"null"
469+
],
470+
"format": "uint32",
471+
"minimum": 0.0
472+
},
453473
"metrics": {
454474
"title": "agent.metrics {#agent-metrics}",
455475
"description": "Enables prometheus metrics for the agent pod.\n\nYou might need to add annotations to the agent pod depending on how prometheus is configured to scrape for metrics.\n\n```json { \"agent\": { \"metrics\": \"0.0.0.0:9000\" } } ```",
@@ -615,6 +635,34 @@
615635
}
616636
}
617637
},
638+
"BodyFilter": {
639+
"oneOf": [
640+
{
641+
"title": "feature.network.incoming.inner_filter.body_filter.json {#feature-network-incoming-inner-body-filter-json}",
642+
"description": "Tries to parse the body as a JSON object and find (a) matching subobjects(s).\n\n`query` should be a valid JSONPath (RFC 9535) query string. `matches` should be a regex. Supports regexes validated by the [`fancy-regex`](https://docs.rs/fancy-regex/latest/fancy_regex/) crate\n\nExample: ```json \"http_filter\": { \"body_filter\": { \"body\": \"json\", \"query\": \"$.library.books[*]\", \"matches\": \"^\\\\d{3,5}$\" } } ``` will match ```json { \"library\": { \"books\": [ 34555, 1233, 234 23432 ] } } ```\n\nThe filter will match if there is at least one query result.\n\nNon-string matches are stringified before being compared to the regex. To filter query results by type, the `typeof` [function extension](https://www.rfc-editor.org/rfc/rfc9535.html#name-function-extensions) is provided. It takes in a single `NodesType` parameter and returns `\"null\" | \"bool\" | \"number\" | \"string\" | \"array\" | \"object\"`, depending on the type of the argument. If not all nodes in the argument have the same type, it returns `nothing`.\n\nExample:\n\n```json \"body_filter\": { \"body\": \"json\", \"query\": \"$.books[?(typeof(@) == 'number')]\", \"matches\": \"4$\" } ``` will match\n\n```json { \"books\": [ 1111, 2222, 4444 ] } ```\n\nbut not\n\n```json { \"books\": [ \"1111\", \"2222\", \"4444\" ] } ```\n\nTo use with with `all_of` or `any_of`, use the following syntax: ```json \"http_filter\": { \"all_of\": [ { \"path\": \"/buildings\" }, { \"body\": \"json\", \"query\": \"$.library.books[*]\", \"matches\": \"^\\\\d{3,5}$\" } ] } ```",
643+
"type": "object",
644+
"required": [
645+
"body",
646+
"matches",
647+
"query"
648+
],
649+
"properties": {
650+
"body": {
651+
"type": "string",
652+
"enum": [
653+
"json"
654+
]
655+
},
656+
"matches": {
657+
"type": "string"
658+
},
659+
"query": {
660+
"type": "string"
661+
}
662+
}
663+
}
664+
]
665+
},
618666
"CiFileConfig": {
619667
"description": "Configuration for mirrord for CI.\n\n```json { \"ci\": { \"output_dir\": \"/tmp/mirrord/\", } } ```",
620668
"type": "object",
@@ -1493,6 +1541,18 @@
14931541
"$ref": "#/definitions/InnerFilter"
14941542
}
14951543
},
1544+
"body_filter": {
1545+
"title": "feature.network.incoming.http_filter.body_filter {#feature-network-incoming-http-body-filter}",
1546+
"description": "Matches the request based on the contents of its body. Currently only JSON body filtering is supported.",
1547+
"anyOf": [
1548+
{
1549+
"$ref": "#/definitions/BodyFilter"
1550+
},
1551+
{
1552+
"type": "null"
1553+
}
1554+
]
1555+
},
14961556
"header_filter": {
14971557
"title": "feature.network.incoming.http_filter.header_filter {#feature-network-incoming-http-header-filter}",
14981558
"description": "Supports regexes validated by the [`fancy-regex`](https://docs.rs/fancy-regex/latest/fancy_regex/) crate.\n\nThe HTTP traffic feature converts the HTTP headers to `HeaderKey: HeaderValue`, case-insensitive.",
@@ -1767,6 +1827,15 @@
17671827
"type": "string"
17681828
}
17691829
}
1830+
},
1831+
{
1832+
"title": "feature.network.incoming.inner_filter.body_filter {#feature-network-incoming-inner-body-filter}",
1833+
"description": "Matches the request based on the contents of its body. Currently only JSON body filtering is supported.",
1834+
"allOf": [
1835+
{
1836+
"$ref": "#/definitions/BodyFilter"
1837+
}
1838+
]
17701839
}
17711840
]
17721841
},

mirrord/agent/Cargo.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,10 +74,14 @@ axum = { version = "0.7", features = ["macros"] }
7474
rawsocket = { git = "https://github.com/metalbear-co/rawsocket.git", rev = "86bba7dbe971e166d5153227dd0099fe47da4489" }
7575
procfs = "0.17.0"
7676
rcgen.workspace = true
77+
serde_json_path.workspace = true
78+
serde_json_path_macros.workspace = true
79+
strum_macros.workspace = true
7780

7881
[target.'cfg(target_os = "linux")'.dev-dependencies]
7982
pem.workspace = true
8083
rcgen.workspace = true
8184
reqwest.workspace = true
8285
rstest.workspace = true
8386
tempfile.workspace = true
87+
rand.workspace = true

mirrord/agent/env/src/envs.rs

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,3 +83,10 @@ pub const IDDLE_TTL: CheckedEnv<u64> = CheckedEnv::new("MIRRORD_AGENT_IDLE_TTL")
8383
/// Sets whether `Mirrord-Agent` headers are injected into HTTP
8484
/// responses that went through the agent.
8585
pub const INJECT_HEADERS: CheckedEnv<bool> = CheckedEnv::new("MIRRORD_AGENT_INJECT_HEADERS");
86+
87+
/// Sets the max size (in bytes) for bodies buffered for body filters.
88+
pub const MAX_BODY_BUFFER_SIZE: CheckedEnv<u32> = CheckedEnv::new("MIRRORD_MAX_BODY_BUFFER_SIZE");
89+
90+
/// Sets how long to wait (in milliseconds) to receive the entire body for body filters.
91+
pub const MAX_BODY_BUFFER_TIMEOUT: CheckedEnv<u32> =
92+
CheckedEnv::new("MIRRORD_MAX_BODY_BUFFER_TIMEOUT");

0 commit comments

Comments
 (0)