-xsl flag does not work as intended

[zdefne-usgs]

I am trying to do this on my desktop with the -custom and -xls the flags:

 java -Xms1024m -Xmx1024m -jar ncISO-2.3.5.jar -ts https://geoport.usgs.esipfed.org/thredds/sand/usgs/Projects/BBLEH/run071tRX/catalog.xml -num 1 -depth 1 -iso true -custom true -xslt UnidataDD2MI_dev.xsl

but I get this error:

ERROR [main] (DatasetTreeService.java:229) - _id=1
thredds.server.metadata.exception.ThreddsUtilitiesException: Configuration problem: xsl/null TransformerConfigurationException. D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO\xsl\null (The system cannot find the file specified)
        at thredds.server.metadata.util.ThreddsTranslatorUtil.doTransform(ThreddsTranslatorUtil.java:307)
        at thredds.server.metadata.util.ThreddsTranslatorUtil.transform(ThreddsTranslatorUtil.java:199)
        at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:213)
...

ncISO is trying to locate the style sheet here:

D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO\xsl\null

I tried with placing the XSL in the same folder and with/without the folder name in the command line but no luck.

This works without the flags and the custom XSL. Also ncISO-2.3.5 throws this error but ncISO-2.3.7 completely ignores the flag and uses the default style sheet therefore no errors generated...

[zdefne-usgs]

Closing this because it is related to the standalone ncISO (posted here by mistake). But the error message is generated by the following line in thredssIso I believe...
Any comments @lesserwhirls ?

threddsIso/nciso-common/src/main/java/thredds/server/metadata/nciso/util/ThreddsTranslatorUtil.java

Line 261 in d19c063

String err = "Configuration problem: "+xsltFile+" TransformerConfigurationException. "+

[lesserwhirls]

Hello @zdefne-usgs - I think I see what is happening. Two things.

1. The commandline argument is -xsl, not -xslt
2. The xsl file needs to be live in a directory called xsl, and that directory must live in the same directory as the jar file.

I coordination with NOAA-PMEL, I had made an attempt at combining the standalone codebase with the plugin codebase so that there could be a one-stop shot, and did a few updates in terms of dependencies (like using the latest netCDF-Java), etc. The current release of that jar can be found at https://artifacts.unidata.ucar.edu/repository/unidata-releases/EDS/nciso/2.4.3/nciso-2.4.3-commandline.jar if you are interested in giving it a spin.

Cheers,

Sean

[zdefne-usgs]

Thank you @lesserwhirls It has been a few years since I las t ran this so I have been following the instructions on NOAA-PMEL github here which says use -xslt

I tried again following your instructions 1. and 2. and I still get an error message

Warning: Could not get charToByteConverterClass!
ERROR [main] (DatasetTreeService.java:229) - _id=1
thredds.server.metadata.exception.ThreddsUtilitiesException: Configuration problem: xsl/UnidataDD2MI.xsl TransformerConfigurationException. javax.xml.transform.TransformerException: org.xml.sax.SAXException: javax.xml.transform.TransformerException: "as" attribute is not allowed on the xsl:variable element!
javax.xml.transform.TransformerException: "as" attribute is not allowed on the xsl:variable element!

I get the following messages with 2.4.3-commandline.jar. It seems to read the custom template but doesn't apply the changes in the file. Any thoughts?

PS D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO> java -Xms1024m -Xmx1024m -jar .\nciso-2.4.3-commandline.jar -ts https://geoport.usgs.esipfed.org/thredds/vortexfs1/usgs/Projects/breach/breach_matanzas/catalog.xml -num 1 -depth 1 -iso true -custom true -xsl UnidataDD2MI_cur.xsl
10:56:12,721 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
10:56:12,722 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
10:56:12,723 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/D:/Documents/DATA_RELEASE/ISOcompare/standalone_ISO/nciso-2.4.3-commandline.jar!/logback.xml]
10:56:12,738 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@6895a785 - URL [jar:file:/D:/Documents/DATA_RELEASE/ISOcompare/standalone_ISO/nciso-2.4.3-commandline.jar!/logback.xml] is not of type file
10:56:12,803 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
10:56:12,803 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
10:56:12,811 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [F]
10:56:12,820 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:56:12,838 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@184f6be2 - There is no conversion class registered for conversion word [x]
10:56:12,838 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@184f6be2 - [x] is not a valid conversion word
10:56:12,878 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@56aac163 - No compression will be used
10:56:12,880 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[F] - Active log file name: ncISO.log
10:56:12,880 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[F] - File property is set to [ncISO.log]
10:56:12,881 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
10:56:12,882 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [stdout]
10:56:12,882 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:56:12,883 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
10:56:12,883 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [F] to Logger[ROOT]
10:56:12,884 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [stdout] to Logger[ROOT]
10:56:12,884 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
10:56:12,884 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1f7030a6 - Registering current configuration as safe fallback point

 INFO [main] (ServiceController.java:61 INFO [main] (ServiceController.java:65 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (DatasetTreeService.java:254 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:98 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (CatalogCrawlerImpl.java:81 INFO [main] (CatalogCrawlerImpl.java:77 INFO [main] (DatasetTreeService.java:264 INFO [main] (DatasetTreeService.java:268 INFO [main] (ThreddsDatasetTree.java:78 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsDatasetTree.java:26 INFO [main] (ThreddsExtentUtil.java:459 INFO [main] (ThreddsExtentUtil.java:344 INFO [main] (ThreddsExtentUtil.java:345 INFO [main] (ThreddsExtentUtil.java:469 INFO [main] (ThreddsExtentUtil.java:470 INFO [main] (ThreddsExtentUtil.java:356

[lesserwhirls]

Do you see anything in the ncIso.log file (should be in the same directory as the nciso-2.4.3-commandline.jar file)?

[zdefne-usgs]

There is some of this...

...
INFO  [main] g.n.e.c.ServiceController.[%PARSER_ERROR[x]] Jan/24 22:07:33 - Running service controller...
INFO  [main] g.n.e.c.ServiceController.[%PARSER_ERROR[x]] Jan/24 22:07:33 - Launch start time: 24 Jan 2022 10:07
INFO  [main] g.n.e.t.b.ThreddsDatasetTree.[%PARSER_ERROR[x]] Jan/24 22:07:33 - displayName=ROOT
INFO  [main] g.n.e.s.DatasetTreeService.[%PARSER_ERROR[x]] Jan/24 22:07:33 - rootNode=breach_matanzas
INFO  [main] g.n.e.t.s.i.CatalogCrawlerImpl.[%PARSER_ERROR[x]] Jan/24 22:07:34 - maxDepth: 1 depth: 0 dataset.getFullName():00_dir_roms.ncml; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 0
INFO  [main] g.n.e.t.s.i.CatalogCrawlerImpl.[%PARSER_ERROR[x]] Jan/24 22:07:34 - allowable service
INFO  [main] g.n.e.t.s.i.CatalogCrawlerImpl.[%PARSER_ERROR[x]] Jan/24 22:07:34 - adding mdc
INFO  [main] g.n.e.t.s.i.CatalogCrawlerImpl.[%PARSER_ERROR[x]] Jan/24 22:07:34 - maxDepth: 1 depth: 0 dataset.getFullName():00_dir_roms_ISO.xml; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 1
INFO  [main] g.n.e.t.s.i.CatalogCrawlerImpl.[%PARSER_ERROR[x]] Jan/24 22:07:34 - allowable service
INFO  [main] g.n.e.t.s.i.CatalogCrawlerImpl.[%PARSER_ERROR[x]] Jan/24 22:07:34 - adding mdc
...

and some of that...

...
INFO  [main] g.n.e.t.b.ThreddsDatasetTree.[%PARSER_ERROR[x]] Jan/24 22:07:35 - found: /geoport.usgs.esipfed.org/thredds/dodsC/vortexfs1/usgs/Projects/breach/breach_matanzas/
INFO  [main] g.n.e.t.b.ThreddsDatasetTree.[%PARSER_ERROR[x]] Jan/24 22:07:35 - displayName=coamps_wind_7318.nc
INFO  [main] t.s.m.n.u.ThreddsExtentUtil.[%PARSER_ERROR[x]] Jan/24 22:07:38 - FeatureType is null or a GRID getting extent from axes: GRID
INFO  [main] t.s.m.n.u.ThreddsExtentUtil.[%PARSER_ERROR[x]] Jan/24 22:07:38 - numTimeElems=722
INFO  [main] t.s.m.n.u.ThreddsExtentUtil.[%PARSER_ERROR[x]] Jan/24 22:07:38 - axisName=ocean_time
INFO  [main] t.s.m.n.u.ThreddsExtentUtil.[%PARSER_ERROR[x]] Jan/24 22:07:38 - Total Memory: 1024
INFO  [main] t.s.m.n.u.ThreddsExtentUtil.[%PARSER_ERROR[x]] Jan/24 22:07:38 - Free Memory: 995
INFO  [main] t.s.m.n.u.ThreddsExtentUtil.[%PARSER_ERROR[x]] Jan/24 22:07:38 - udunits string = 4.9825152E9 seconds since 1858-11-17 00:00:00
ERROR [main] t.s.m.n.u.ThreddsTranslatorUtil.[%PARSER_ERROR[x]] Jan/24 22:07:42 - Could not load NETCDF file: https://geoport.usgs.esipfed.org/thredds/dodsC/vortexfs1/usgs/Projects/breach/breach_matanzas/00_dir_roms_ISO.xml because of Exception. https://geoport.usgs.esipfed.org/thredds/dodsC/vortexfs1/usgs/Projects/breach/breach_matanzas/00_dir_roms_ISO.xml is not a valid URL, return status=400
java.io.IOException: https://geoport.usgs.esipfed.org/thredds/dodsC/vortexfs1/usgs/Projects/breach/breach_matanzas/00_dir_roms_ISO.xml is not a valid URL, return status=400
	at ucar.nc2.dataset.DatasetUrl.checkIfCdmr(DatasetUrl.java:402)
	at ucar.nc2.dataset.DatasetUrl.disambiguateHttp(DatasetUrl.java:388)
	at ucar.nc2.dataset.DatasetUrl.findDatasetUrl(DatasetUrl.java:154)
	at ucar.nc2.dataset.NetcdfDatasets.openDataset(NetcdfDatasets.java:134)
	at ucar.nc2.dataset.NetcdfDatasets.openDataset(NetcdfDatasets.java:118)
	at ucar.nc2.dataset.NetcdfDatasets.openDataset(NetcdfDatasets.java:104)
	at thredds.server.metadata.nciso.util.ThreddsTranslatorUtil.doGetNcml(ThreddsTranslatorUtil.java:86)
	at thredds.server.metadata.nciso.util.ThreddsTranslatorUtil.getNcml(ThreddsTranslatorUtil.java:78)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:159)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:234)
	at gov.noaa.eds.service.DatasetTreeService.generateTree(DatasetTreeService.java:318)
	at gov.noaa.eds.controller.ServiceController.launch(ServiceController.java:69)
	at gov.noaa.eds.controller.ServiceController.main(ServiceController.java:125)
...

Any thoughts? Thanks for looking into it...

[zdefne-usgs]

@lesserwhirls to follow up with my previous post with more information. More has been discussed on the ncISO repo
NOAA-PMEL/uafnciso#21

To summarize, the version of Java seems to matter and users may have different experience when they run the command on their system. The error I had above when running nciso-2.4.3-commandline.jar was with the following set up:

openjdk version "16.0.2" 2021-07-20
OpenJDK Runtime Environment Temurin-16.0.2+7 (build 16.0.2+7)
OpenJDK 64-Bit Server VM Temurin-16.0.2+7 (build 16.0.2+7, mixed mode, sharing)

When I downgraded to Java8 (which worked with ncISO_v.2.3.7.jar) I got an error message saying that nciso-2.4.3-commandline.jar was compiled with a newer version.

ncISO has updated to v.2.3.8 to fix this issue and I can run it either with java8 or java16 and get the same behavior. But now I get errors that complain about "as=string" tags in the XSL. It looks like this is an issue with XSLT1.0 but not with XSLT2.0 (see here for more info).

UPDATE: switching to a newer version of Xalan has fixed this issue. See here.
So issues we had with the standalone ncISO seem to be resolved. Hopefully threddsIso will be solved too... Thanks a lot for all the help @lesserwhirls !

-what version of java was use to compile nciso-2.4.3-commandline.jar?
-was the XSLT version downgraded from 2.0 to 1.0 sometime in the past? I see a similar discussion here:
#19 (comment)
-when I replace the UnidataDD2MI.xsl on our Thredds server the customizations are applied correctly. How can I compare the standalone ncISO to what I have on the server (apple to apple comparison)?

[lesserwhirls]

I'm guessing JDK 11 was used to compile nciso-2.4.3-commandline.jar since that is what the TDS requires. The xsl file used by nciso and the tds iso plugin have diverged, and that was the only remaining task to complete to finalize merging the code bases (which would allow an apples to apples comparison). Glad to hear the issue is fixed over on the uafnciso repository. I guess there are no longer plans to merge the two codebases? Tagging @ethanrd, as he may have more up-to-date information on where that effort stands.