[configuration] make the config file optional and fully usable.

[jbygdell]

Please describe the feature

All options specified in the config file should be possible to set as flags, making the config file optional.

At the same time it should be possible to include all required fields in the config file, also note the updated format from auth service:

access_token = eyJraWQ...
url = download.example.org
public_key = base64enc.string
dataset_id = some-identifier (possibly)

Adding viper for config parsing in sda-cli would make it easy to populate configuration for the subcommand before invoking it, similar to how it's done in sensitive-data-archive.

sda-cli --config myconfig download --some-option

Strive for consistency:

cli: --config-option
config-file: config_option
env: CONFIG_OPTION

Acceptance criteria

• Access key can be set on the command line when downloading data
• It should be possible to specify only the config file for downloading given it contains all information.
• New version of sda-cli should work with both old and new format of config file. sda-cli should just silently ignore options it does not recognize (for example host_base should be ignored).
• Tests verifying the changes are added

Additional context

For download it's currently only the access-token that is not possible to specify without config file. Add --access-token and/or through an ENV ACCESS_TOKEN.

Estimation of size

medium

Estimation of priority

medium

[KarlG-nbis]

Something like

package cmd
...
func PreRun() func(*cobra.Command, []string) {
	return func(cmd *cobra.Command, _ []string) {
		if err := viper.BindPFlags(cmd.Flags()); err != nil {
			log.Fatalf("failed to bind flags: %v", err)
		}

		if viper.IsSet("config") {
			viper.SetConfigFile(filepath.Clean(viper.GetString("config")))
		}
		viper.SetConfigType("yaml")

		if err := viper.ReadInConfig(); err != nil {
			var configFileNotFoundError viper.ConfigFileNotFoundError
			if !errors.As(err, &configFileNotFoundError) {
				log.Fatalf("failed to read config file: %v", err)
			}
		}

		for _, key := range viper.AllKeys() {
			if err := cmd.Flags().Set(key, viper.GetString(key)); err != nil {
				log.Fatalf("failed to set %s: %v", key, err)
			}
		}
	}
}

in root.go

And then in each cmd file, eg decrypt.go

var decryptCmd = &cobra.Command{
	...
	PreRun: cmd.PreRun(),
	...
}

Should suffice to allow the --config flag to populate the cobra options for each sub command