pacemaker-remoted load_env_var refactors, and assorted minor changes

[nrwahl2]

I'm trying to fix the Coverity issues that are popping up for #3832 and these are also appearing on my machine. There are some more I haven't reached yet.

[nrwahl2]

Used a define per review (https://github.com/ClusterLabs/pacemaker/compare/17bb059c39e3ee0b75a8fc26c259a26c5241c719..3ebf24e3100e21a2900009ebcb5c5c7edafa92a1)

[clumens]

Were you able to test the changes to load_env_vars? That, or some unit tests, would make me feel a whole lot better about all that small batch string processing code.

[nrwahl2]

Were you able to test the changes to load_env_vars? That, or some unit tests, would make me feel a whole lot better about all that small batch string processing code.

Not really. That's gonna take me a little while to set up and figure out (won't be ready today during business hours). But it is a good idea.

Also interested in whether there's a better way than rolling our own parser, but haven't found anything yet.

[nrwahl2]

Not really. That's gonna take me a little while to set up and figure out (won't be ready today during business hours). But it is a good idea.

Also interested in whether there's a better way than rolling our own parser, but haven't found anything yet.

I'm planning to use pcmk__scan_nvpair() here (now that I'm aware it exists) and then process the key and value separately. It will hopefully make things easier to reason about, even though there will still be corner cases involving escapes and nested quotes for now. See also #3834.

I think the authors made the load_env_vars() function way too complicated in our habitual obsession with "try to never allocate heap memory for a string." Which makes sense in the kernel, but IMO it's pretty silly in MOST of Pacemaker, where we're allocating large data structures frequently. Way too complicated without even achieving totally correct behavior, because it's so easy to get bogged down in the character-by-character parsing details.

[clumens]

I'm planning to use pcmk__scan_nvpair() here (now that I'm aware it exists) and then process the key and value separately. It will hopefully make things easier to reason about, even though there will still be corner cases involving escapes and nested quotes for now. See also #3834.

Okay, I'll wait to do final review on this until after you've moved to using pcmk__scan_nvpair().

I think the authors made the load_env_vars() function way too complicated in our habitual obsession with "try to never allocate heap memory for a string." Which makes sense in the kernel, but IMO it's pretty silly in MOST of Pacemaker, where we're allocating large data structures frequently. Way too complicated without even achieving totally correct behavior, because it's so easy to get bogged down in the character-by-character parsing details.

I absolutely agree. If I were writing this function from scratch, I would (1) have it take file contents as a parameter instead of a file name, (2) put it in a library somewhere as private API to allow us to unit test it, and (3) find something in glib that could be used to do the parsing instead of writing it by hand. Now we're stuck with having to support whatever bugs there might be in processing since otherwise we'll break people's remote nodes.

[nrwahl2]

Now we're stuck with having to support whatever bugs there might be in processing since otherwise we'll break people's remote nodes.

Not necessarily. Obviously we've run into many situations where users are relying on behavior they shouldn't be... but these corner cases seem like pretty clear bugs that shouldn't be guaranteed to work, and that it's unlikely anyone is using. I'd be willing to gamble, if we actually feel like fixing them.

That's not something I want to spend time on right now though. If we find a good parser in a library, fine. But setting flags for quote nesting levels, whether we just saw a backslash and are escaping the next character, etc., sounds really unappealing.

[nrwahl2]

Rebased on current main, which led to dropping the final commit. Otherwise no changes.

[nrwahl2]

I added a commit to fix the rand() suppression. I also split up the load_env_vars() stuff into several more incremental commits and then used pcmk__scan_nvpair() in the final one. It doesn't really make that function any shorter, but IMO it's a lot more readable now.

I have not tested yet.

[nrwahl2]

Still need to test the load_env_vars() stuff and maybe functionize the value validation part, and run cts-lab for a bit to maybe find any obvious issues with the remote message stuff.

I can't easily test the localized_remote_header() involving byte swapping for endianness, because I don't have a big-endian machine. The last time I tried to set up a big-endian VM, if I ever succeeded at all, it was an enormous pain. I don't see much benefit from unit testing it either: we can trust that the GLib functions do what they're supposed to do, as long as the surrounding logic is correct.

But this actually resolves all the Coverity issues on my laptop. There is a suppression for rand(), and there are two suppressions in crm_resource.c that will go away as part of my crm_resource overhaul PRs. And that's it.

[clumens]

@nrwahl2 Is there anything else remaining to be done on this PR?

[nrwahl2]

@nrwahl2 Is there anything else remaining to be done on this PR?

Only testing, hopefully.

There is no unit testing yet, which may be overkill now that we rely on a handful of GLib function calls... but more importantly I haven't tested this by creating a bundle and making sure it loads environment variables appropriately. I need to refresh on how to do that.

[nrwahl2]

Updated the "Standardize remote environment variable parsing" commit substantially, to resolve an issue where leading whitespace in an unquoted value was allowed. Also added a commit to a document a couple of limitations that I don't feel like dealing with. No changes to earlier commits.

About to rebase on current main to resolve conflict.

[nrwahl2]

I tested successfully with the following environment file, making sure the values were as expected. It's not pretty, just a quick-and-dirty thing that hits all the important cases I could think of. The comment at the top is the variables that should get set.

# 1-3, 5, 6, 10-16, 21-24
#
VAR1=value1
   VAR2='value2'

VAR3='value3  3'
VAR4=value4 4
# starts with comment
VAR5=value5 #5
VAR6=value6 #6 also 6
1VAR7=value7
'VAR8'='value8'
#VAR9=value9
VAR10=
VAR11=''
VAR12=""
VAR13='""'
VAR14="''"
VAR15="  "
VAR16='  '
VAR17 =17
VAR18 = 18
VAR19= 19
VAR20= 20 # comment
VAR21="'nested single then double \"quotes\"' after-text"
VAR22="pre-text \"nested double then 'quotes'\""
VAR23='pre-text '\''nested single then double '\"quotes\"\'
VAR24='"pre-text nested double then single '\''quotes'\''" after-text'