More Error messages needed

[christian-rli]

OMI can benefit from useful error messages.

For example:

• File doesn't match metadata description
  • column doesn't exist / has different data type / has different name
  • table has more columns than described in metadata
• JSON seems to be invalid at location X
• key name does not exist in standards, typo?
• Warning: no license has been chosen, this field will be necessary when uploading to the OEP

I would recommend creating a list of possible errors first and to then write useful messages and tests for them.

Also, maybe have a look a the license check performed on OEP (implemented by @jh-RLI ) and if it makes sense to put it into OMI or leave it there.

See #99

[christian-rli]

@henhuy I thought up a list of possible error types:

in the metadata:
- wrong structure
- other version than declared
- JSON syntax error (like missing ,}]:'")
- incorrect nesting
- key name not in schema (typo?)
- additional field (this should just show a WARNING. tolerating additional fields has been requested)
- wrong data type in a JSON field
- duplicate key/value
- null values that mess with the structure of the JSON
- missing field (maybe as WARNING? When missing ID or licence)

I have provided a metadata string for each of the errors above in a separate branch. Maybe it's useful to you. I wasn't sure what a minimal metadata string had to include, so maybe they're unnecessarily big.

Checking against data:
- metadata don't match data structure
- missing column
- extra column
- wrong data type in a column (Warning or error?)
- duplicate or null values in ID column

I haven't created samples for these, because I saw that you had a test data table. So I would like use that for any error sample files.

[henhuy]

Tried to implement metadata checks from above. Metadata files were VERY helpful, thx!
Following findings ("x" means a test is included which checks for given issue):

• wrong structure
  is covered by either JSON decoder (if structure is no valid JSON format) or JSONschema if structure does not fit
• other version than declared
  Not yet possible, as only versions >1.5.2 are implemented and there are no differences compared to 1.6.0
• JSON syntax error (like missing ,}]:'")
  handled by JSON decoder
• incorrect nesting
  handled by JSON decoder
• key name not in schema (typo?)
  as we don't have required fields yet, I can not test against it. Typos are out of scope IMO. (wrong_key.json will only throw error due to additional key "titel", but will not know that this is a typo)
• additional field (this should just show a WARNING. tolerating additional fields has been requested)
  must be allowed in Metadata schema first, otherwise JSONschema will always throw an error
• wrong data type in a JSON field
  should work due to JSONschema, but not yet tested as types are all "string" or "null"
• duplicate key/value
  added check to JSON decoder which checks for duplicate keys
• null values that mess with the structure of the JSON
  handled by JSON decoder
• missing field (maybe as WARNING? When missing ID or licence)
  this actually raises an error, due to JSONschema

[henhuy]

Checking against data:

• metadata don't match data structure
• missing column
• extra column
• wrong data type in a column (Warning or error?)
• duplicate or null values in ID column

Added tests for all cases.
And: Wrong data type raises an error

[nesnoj]

Great, big thanks for these efforts!
(I guess this will also fix #40)