Difference in (-d) parsing between curlie and curl

[bored-engineer]

It looks like curlie gets a little confused when passing -d:

$ curlie https://httpbin.org/post -d 'foo=bar'
...
{
    "args": {

    },
    "data": "foo=bar&{\"foo\":\"bar\"}",
    "files": {

    },
    "form": {

    },
    "headers": {
        "Accept": "application/json, */*",
        "Content-Length": "21",
        "Content-Type": "application/json",
        "Host": "httpbin.org",
        "User-Agent": "curl/7.64.1"
    },
    "json": null,
    ...
}

$  curl https://httpbin.org/post -d 'foo=bar'
{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "foo": "bar"
  },
  "headers": {
    "Accept": "*/*",
    "Content-Length": "7",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.64.1"
  },
  "json": null,
  ...
}

I'm assuming this is a bug where it's doing both JSON parsing and passing data?

[bored-engineer]

Note: The issue only occurs when -d has a trailing space before the data begins and using --data instead results in the correct behavior so the issue is specific to the short args parsing

[rs]

I guess it's due to this: https://github.com/rs/curlie/blob/master/main.go#L74

[bored-engineer]

@rs Do you think the correct behavior here is to handle -d when it has a trailing space and disable the JSON/HTTPie arg parsing? Or should it just fix the duplicate post body bug but still parse the -d "somedata" arguments differently then curl does?

[rs]

I don't remember why I added this exception. I would tend to restore the default curl behavior.

[bored-engineer]

@rs Would you be opposed to a pull request introducing a library like github.com/spf13/pflag to do the argument parsing/handling instead of rolling a custom one? That could help make argument parsing/usage easier and allow the addition of new args/opts in the future easier (plus some nice things like adding the curlie options to --help)

[rs]

The advantage of the current parsing is that it is a lazy parsing: it adapts to whatever arguments the installed curl version supports.

[bored-engineer]

Hmm I thought pflag had support for that but looking more into it it doesn't exactly. It has a UnknownFlags attribute that will skip over unknown/undefined flags, but it doesn't offer a way to later recover those flags: https://godoc.org/github.com/spf13/pflag#ParseErrorsWhitelist
Looks like there's an open pull request for it that hasn't been accepted: spf13/pflag#199

[tom-on-the-internet]

Loving Curlie. Just got bit by this. --data works as expected.