Check null to avoid null pointer exception

[bact]

Check if the string input is null in getFirstLicenseToken and isSingleTokenString

To fix #333

[goneall]

Since we're now using a Java version that support the @Nullable annotations, we should add these to the calls.

Do we want to open a separate PR to fix the empty optional strings returning true when it should be false?

[bact]

Do we want to open a separate PR to fix the empty optional strings returning true when it should be false?

We can fix it here as well.
I also have to update the unit test to expect false when the string is empty.

[goneall]

Do we want to open a separate PR to fix the empty optional strings returning true when it should be false?

We can fix it here as well. I also have to update the unit test to expect false when the string is empty.

It looks like there are quite a few unit test failures. It looks like I was wrong about returning false - @bact - try returning true for empty and null and see if it passes the CI.

[bact]

Thanks @goneall. I'm trying to understand the rationale behind isSingleTokenString() method.

In general, I think we like to return true if the text contains zero or one token (and what constitute token does not include whitespaces and certain punctuations) -- is this correct?

Another thing I'm not sure about is this line, on why we treat newline differently from other whitespaces:

Spdx-Java-Library/src/main/java/org/spdx/utility/compare/LicenseCompareHelper.java

Lines 215 to 217 in 3ec6b34

	if (text.contains("\n")) {
	return false;
	}

Do you mind to help me confirm this truth table please?

text	tokens (m.group())	tokens (m.group(1))	isSingleTokenString(text)
"token"	token	token	true
" token"	, token	(empty), token	true
"\ntoken"	\n, token	(empty), token	true ?
"token\n"	token\n	token	true ?
"'''token"	''', token	(empty), token	true
"token'''"	token'''	token	true
"a and"	a , and	a, and	false
"a\nand"	a\n, and	a, and	false
"Apache-2.0"	Apache-2., 0	Apache-2, 0	false
" " (two spaces)	(two spaces)	(empty)	true
" " (one space)	(one space)	(empty)	true
"\n"	\n	(empty)	true ?
""	(empty)	(empty)	true
null	n/a	n/a	true

[goneall]

@bact - the table above looks correct

This code was originally added to fix a corner case in the matching. I believe the actual check is to see if a text field contains more than one token. If it contains more than one token, we need to do some extra processing.

[goneall]

Thans @bact for the analysis and fix.

This seems to pass the unit tests.

We could add more documentation that the isSingleToken is really returning 0 or 1 tokens, but that can be done in a separate PR.

I'll go ahead and merge this so we can move forward with a working solution.