`include` directive doesn't work as expected

[flihp]

We've been trying to get away from textual / tabular descriptions of PKI & certificate structures, preferring a KDL format understood by the pki-playground tool instead. I'm pulling the KDL into RFD 387 using the asciidoc include directive to make running the files through the pki-playground command line tool easier. I never thought to look till now but the RFD site doesn't render the KDL. It could be that I'm holding it wrong, but the site shows the include:file/path.kdl text instead of the contents of the file. I've been using asciidoctor-pdf to build locally and that works as expected. include looks like it might cause problems if we allow folks to pull in arbitrary URLs but files local to the repo would be sufficient for my needs.

[benjaminleonard]

So essentially what we're doing on the RFD site is returning the .adoc contents from the API and then running that through the AsciiDoc loader on the server. The issue then being that it's completely isolated from the context of the filesystem you have when running it locally. In this instance it has no reference to platform-identity-csr.kdl. The RFDs themselves exist in a database, so it's not working off a file system at all.

The image references we transform from a relative link to a signed URL from the GCP storage where they are kept. In theory I wonder if we might be able to chuck other non-image assets that sit in the RFD folder in there too. But it might not be worth the work to get it working, and whilst I understand it's a bit inconvenient to update manually it might still be better than the alternative.

Will defer to @augustuswm on what he thinks is best here.

[david-crespo]

A starting point: We could make it work locally, and then maybe the API would handle it by resolving references and inlining the content.

[benjaminleonard]

Inlining the content would be easier with this I think: oxidecomputer/react-asciidoc#24

We could handle includes by post-processing the document object or we might not need to since it'd be running with the correct context. As it stands we'd have to do some sort of regex to find includes in the adoc file.

[augustuswm]

I think we can probably make this work. It would be simple enough to process the contents and handle files that match a known allowlist of file types.

Note though that I don't think GitHub supports includes either from what I can tell.

[inickles]

Just noting that I ran into this myself. I have new tooling to generate much of the content in RFD 536 that I'd like to just import.

[benjaminleonard]

Perhaps we move this issue to rfd-api @augustuswm – not sure if you're planning to support? This isn't really something we can support at the site level but you might be able to pre-process the documents.

[augustuswm]

Yeah, makes sense to me to move it to the rfd-api repo. I don't think we can or should support this on the frontend side when operating in remote mode. I think we could support it for local mode though. It would be similar parsing logic to what the API does.

I definitely want to support this on the API side, but I need to read up on how exactly it is supposed to operate.

[augustuswm]

I've added initial support for includes to the RFD API processing. We only support the simplest form which has the following restrictions:

1. Includes exist entirely on their own line (as opposed to embedded in a sentence or other content)
2. Included content must be UTF-8 encoded text
3. No options may be passed to the include macro * `include::path/to/file.adoc[] will work
   • include::path/to/file.adoc[indent=2] will not work
4. Paths must be relative to the location of the README.adoc
   • include::path/to/file.adoc[] will work
   • include::rfd/0001/path/to/file.adoc[] will not work
   • include::../path/to/file.adoc[] will not work
   • include::/rfd/0001/path/to/file.adoc[] will not work
5. Only the top level README file will be processed for includes. Include statements in included files will not be processed.
6. Paths must be written out, and not used attribute resolution to fill in variables. (This may or may not work depending on the RFD, so I'm marking as unsupported for now)
7. Local editing support is not yet implemented, but I'll work on that next