Repository navigation
Logging fails when non JSON-serialisable arguments or return values are present #515
Description
Using @log_call fails and generates errors completely unrelated to the actual execution when any arguments or return types are not JSON-serialisable. I imagine the same is also the case for any other logging function, but @log_call where I ran into it, since it's much more probable that arbitrary types will be logged in that situation.
Small repro case:
import subprocess
import eliot
@eliot.log_call
def foo(x):
return x
eliot.to_file(open("/tmp/the-log.json", "wb"))
foo(subprocess.Popen("date"))Results:
$ eliot-tree /tmp/the-log.json
3a6afc51-3f52-4e3f-9bf1-9de3943ab9c4
└── eliot:destination_failure/1 2025-04-13 16:03:33Z
├── exception: builtins.TypeError
├── message: {"'x'": "<Popen: returncode: None args: 'date'>", "'action_status'": "'started'", "'timestamp'": '17…
└── reason: Type is not JSON serializable: Popen
1857c350-f0d4-4c84-8063-8c8426f24b08
└── eliot:destination_failure/1 2025-04-13 16:03:33Z
├── exception: builtins.TypeError
├── message: {"'result'": "<Popen: returncode: None args: 'date'>", "'action_status'": "'succeeded'", "'timestamp…
└── reason: Type is not JSON serializable: Popen
This is, IMHO, unacceptable for a general-purpose logging library, especially since there's not a word anywhere in the documentation about the requirement for items being logged to be fully JSON-serialisable. If a value doesn't have a predefined serialiser, it should just fall back to repr(), rather than logging errors unrelated to the code being executed and completely dropping the actual logs. Slightly lower-fidelity logs that never fail are far, far superior to no logs at all and confusing errors.
Activity
[-]Logging causes errors when non JSON-serialisable arguments or return values are present[/-][+]Logging fails when non JSON-serialisable arguments or return values are present[/+]itamarst commentedon Apr 13, 2025
That is probably a more reasonable fallback. It should probably be done in the JSON serializer, rather than the higher-level infrastructure, since JSON output is not the only possible kind of output.
Would you be interested in submitting a PR?
mathrick commentedon Apr 13, 2025
I would, though I very much don't have the time to do it right now (next weekend might be doable), and would need some pointers for where to put the code.
I do agree about this being the job of the serialiser; Eliot supplies its own serialiser when using JSON output, right? If so, that should be fairly straightforward.
Don't error out when logging non-JSON serialisable values
mathrick commentedon Apr 13, 2025
Actually, turns out it was very easy to make the PR after I looked at the code enough to fix it for myself, so here you go :)
Don't error out when logging non-JSON serialisable values