Advanced Public Suffix List testing (registrable domain, same site)

[annevk]

Today we have not-web-platform.test and web-platform.test.

As discussed in whatwg/url#391 to test corner cases of the Public Suffix List that have manifested in differences between browsers, we need more.

In particular, we need something like psl-web-platform.test, browsers to recognize that as a public suffix (perhaps through some flag), we need that to be reachable on its own, as well as multiple subdomains of one and two domain labels.

cc @sleevi

[sleevi]

I am not clear what your objective is or how this would be used. I identified two sources of divergence - one in the application of wildcard rules, one in the shipping of versions.

Can you explain what value there is over integration tests, compared to unit tests as part of PSL, which already has a number of them (and designed to work with various test harnesses).

// @weppos @dnsguru

[annevk]

@sleevi I want to test document.domain and such do the right thing, since you mentioned it's possible for these to diverge depending on the consumer.

[sleevi]

I think it’s worth unpacking that goal. You are potentially testing a few things:

1. The implementation of the PSL algorithm
2. The interaction with the PSL of the various features (e.g. document.domain and whether or not it includes things like the PRIVATE section)
3. The version of the PSL being used

All three of those can cause differences. The first should be tested in the PSL, not WPT. The third should not be tested because there’s zero guarantees around it (and that’s assuming all browsers use the PSL - my understanding is that’s only been true in the last year or two). And with the second test case, you aren’t really able to iterate through the scenarios because the PSL will not accept test-only PRs or changes to the ICANN/IANA section that aren’t delegated as such.

Is there another scenario not covered? Is the intent less about testing differences and simply about testing whether they use the PSL for a given feature?

[sleevi]

I suppose put differently: Anything using the PSL is expected to diverge per-consumer. If this is not desired, then more support and effort is needed to migrate away from the PSL, or on the coordination and management of the PSL, by browsers. In my mind, this is no difference than divergence in the HSTS preload lists or in the trusted set of CA certificates - they are imperfect but necessarily solutions.

I totally understand and appreciate the predictability goal, but I think it’s worth unpacking when that predictability matters. Your applications should not be built based around the observable properties, and should assume inclusion on the PSL is “best effort” at reducing scope, not “guaranteed”. Mitigations include changing actual domains, rather than relying on expressed split behaviours. Neither the ICANN nor PRIVATE sections should be assumed to be in a given state at any times. It is merely a hint.

[mikewest]

@sleevi: I think the second goal you've outlined is an important one. Regardless of the PSL's contents for a given version of a given client, document.domain should behave in a well-understood way. As should cookies. As should anything else that's web-visible and depends on the notion of a "site".

If the PSL won't accept test-only PRs (though both test.ru and test.tjmade it in...), perhaps browser vendors could agree on a certain name to add to their imported list?

[annevk]

In the second test of html/browsers/origin/relaxing-the-same-origin-restriction/document_domain_setter.html Safari throws "Attempted to use a non-registrable domain." so PSL already seems to be leaking. (Filed https://bugs.webkit.org/show_bug.cgi?id=186388.)

cc @cdumez

[foolip]

@annevk, if testing this requires infra changes, can you condense those out? I guess it'd require having domains like web-platform-test.co.uk, but that is at odds with #11075.

[annevk]

I don't think that is at odds, necessarily. We'd need to update the certificates to include the added names, but otherwise I don't see a conflict.

As for infrastructure changes, that depends on how we want to go about this. If we cannot modify the PSL, we'd need to instruct browsers somehow that something is a public suffix. Either via an API or by having browsers agree to add something locally. Not sure.

[foolip]

Can you point to some of the mentioned differences between browsers? Is it just that their PSLs are not in perfect sync, or are there differences around processing of it that also results in problems?

[annevk]

whatwg/url#391 (comment) was the reason I filed this. The above comment about Safari is an example I spotted while working on something related, but can be manifested without additional infrastructure or domains it seems. (It's unclear if it could still be reproduced if "test" was added to the PSL however.)

[sleevi]

If the PSL won't accept test-only PRs (though both test.ru and test.tj made it in...)

Both of these are expressions of the ccTLDs reserved registration policies - that is, probably the worst examples to pick, since IANA and the ccTLDs handle those allocations ;) I suppose if you can find a ccTLD or gTLD that will update its policies (and potentially contracts with ICANN) to set aside a domain namespace, that's.... one way ;)

For what it's worth, this policy is the same reason that the PSL doesn't contain IANA-reserved names like example.com or example.net (or, for that matter, .example) 😃

perhaps browser vendors could agree on a certain name to add to their imported list?

I think this is actually a rather sensible thing for the goal of testing that second item. This aligns with the effective requirement of needing to modify the TLS stack to trust WPT certificates - of which the 'how' a given platform does this will vary (in means and difficulty), but the result is consistency.

Much like browsers do not ship with the WPT test certificates trusted to all of their end users, one would hope that they wouldn't ship this PSL name to all of their end users, although I suspect that 'ease' will lead them to doing exactly that (that is, shipping forked copies of the PSL even though it's for testing purposes). At the same time, it's hopefully understandable why including a name for testing in a database that will be shipped in a vast array of non-browser clients (including a number of programming language runtimes) is probably not the best use of a shared resource like the PSL.

So I suppose the question, for implementors, is whether or not they could support local-only modifications to their PSL algorithm, for purposes of WPT changes (i.e. can they commit to not forking the PSL for WPT)? Understanding that would help understand more of the tradeoffs.