Skip to content

Decode map keys into keywords for [:map schemas in json-transformer #1135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Dec 8, 2024
Merged

Decode map keys into keywords for [:map schemas in json-transformer #1135

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
15b56d0
Failing test
dzhus Nov 25, 2024
1abc8e3
Direct fix
dzhus Nov 25, 2024
7d44b1c
CHANGELOG
dzhus Nov 25, 2024
cb8e7f7
Unnest changelog entry
dzhus Nov 26, 2024
761d62f
Update test
dzhus Nov 26, 2024
5b1dbaf
Opt-in, do not touch non-keyword schema keys
dzhus Nov 26, 2024
6a1f515
This is no longer a breaking change
dzhus Nov 26, 2024
3c9ab67
Merge remote-tracking branch 'origin/master'
dzhus Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Malli is in well matured [alpha](README.md#alpha).
* **BREAKING**: `:gen/fmap` property requires its schema to create a generator.
* previous behavior defaulted to a `nil`-returning generator, even if the schema doesn't accept `nil`
* use `:gen/return nil` property to restore this behavior
* Decode map keys into keywords for `[:map` schemas in `json-transformer` [#1135](https://github.com/metosin/malli/issues/1135)
Copy link
Member

@opqdonut opqdonut Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is now nested under the :gen/fmap change, even though it's not related. Please add it as a top-level bullet point at the top of the list.

Copy link
Contributor Author

@dzhus dzhus Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to top-level + no longer labeled as breaking.

* FIX: `malli.registry/{mode,type}` not respected in Babashka [#1124](https://github.com/metosin/malli/issues/1124)
* Updated dependencies:

Expand Down
2 changes: 2 additions & 0 deletions src/malli/transform.cljc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,8 @@
(-transform-if-valid key-schema)
(-transform-map-keys))
(-transform-map-keys m/-keyword->string))))})
(assoc :map {:compile (fn [_ _]
(-transform-map-keys -string->keyword))})
Copy link
Member

@opqdonut opqdonut Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this might need similar map-of-key-decoders logic as map-of to work in the general case. Alternatively, this might need to be an optional feature like json-vectors below.

Copy link
Member

@ikitommi ikitommi Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this should not be the default as it blows the perf (your JSON decoder most likely already emits keyword keys) + this will break maps like:

[:map
 ["kikka" :string]
 ["kukka" :string]]

Copy link
Contributor Author

@dzhus dzhus Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will break maps like:

@ikitommi Yeah that's a good point – I've made this transformation schema-aware.

this should not be the default as it blows the perf (your JSON decoder most likely already emits keyword keys)

True, I've now made this functionality opt-in via ::keywordize-map-keys option of the transformer.

It's a bit sad that we have to opt in in order to have "decoded value validates against schema" property, but I guess there's no other way (other than creating JSON deserialiser from Malli schemas directly to fuse away this step).

(cond-> json-vectors (assoc :vector -sequential->vector)))
:encoders (-json-encoders)})))

Expand Down
12 changes: 12 additions & 0 deletions test/malli/core_test.cljc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -997,6 +997,18 @@
(mt/key-transformer
{:decode #(-> % name (str "_key") keyword)}))))

(testing "JSON transformer decodes map schema keys"
(let [schema [:map
[:a :uuid]
Copy link
Member

@opqdonut opqdonut Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a breaking change for users who want to keep keys as strings. Could you add a test case that demonstrates what happens with a schema with string keys?

Copy link
Contributor Author

@dzhus dzhus Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an example with "string" key in [:map in 761d62f.

[:b [:enum :x :y :z]]]
Copy link
Contributor Author

@dzhus dzhus Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

used non-trivial schemas for leaves here just to convince myself that a layer of (-transform-map-keys -string->keyword) doesn't mess with recursive decoding

value {"a" "b699671c-d34d-b33f-1337-dbdbfd337e73"
"b" "x"}
decoded-value (m/decode schema value mt/json-transformer)]
(is (= {:a #uuid "b699671c-d34d-b33f-1337-dbdbfd337e73"
:b :x}
decoded-value))
(is (m/validate schema decoded-value))))

(is (= {:x 32}
(m/decode
[:map {:decode/string '{:enter #(update % :x inc), :leave #(update % :x (partial * 2))}}
Expand Down