Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
d3ed8c9
fix: revert back to surpressing pull_request triggers and append debu…
Pfeil Mar 14, 2025
8029da8
ci: debug information and remove trigger limitation for now
Pfeil Mar 14, 2025
3450e38
ci: remove full event information as it breaks something
Pfeil Mar 14, 2025
49fe1da
chore(deps): update plugin io.freefair.maven-publish-java to v8.13.1
renovate[bot] Mar 25, 2025
a92b19f
chore(deps): update dependency com.github.fslev:json-compare to v7
renovate[bot] Mar 28, 2025
79c65b4
Added zip stream writer, minor cleanup
ThomasJejkal Apr 1, 2025
9d86237
Updated version to 2.0.3, fixed performance tests, fixed some documen…
ThomasJejkal Apr 2, 2025
2f140ce
Merge pull request #236 from kit-data-manager/renovate/com.github.fsl…
ThomasJejkal Apr 2, 2025
8aace84
Merge pull request #235 from kit-data-manager/renovate/io.freefair.ma…
ThomasJejkal Apr 2, 2025
8d95e20
Added tests for ZipStreamWriter, started adding integrated preview (t…
ThomasJejkal Apr 3, 2025
91dc34d
Merge branch 'development' of github.com:kit-data-manager/ro-crate-ja…
ThomasJejkal Apr 3, 2025
ad72cc4
Merge with origin
ThomasJejkal Apr 3, 2025
f3aef3f
Added json-ld expander to prepare crate metadata for template renderi…
ThomasJejkal Apr 6, 2025
9e41a50
Finished template and preview generation, still needs to be integrate…
ThomasJejkal Apr 7, 2025
0e80875
Integrated custom preview, renamed previously CustomPreview to Static…
ThomasJejkal Apr 8, 2025
f83386a
Fixed all test issues, added some checks to preview generators for mo…
ThomasJejkal Apr 9, 2025
3098d13
Fixed issues with stale writers
ThomasJejkal Apr 10, 2025
4cd37db
chore(deps): update dependency font-awesome to v6
renovate[bot] Apr 10, 2025
51610ff
Added streamReader, fixed an issue with zipReader adding crate metada…
ThomasJejkal Apr 10, 2025
0b8f32a
Merge pull request #239 from kit-data-manager/renovate/font-awesome-6.x
ThomasJejkal Apr 10, 2025
6cd8d8a
Fixed broken tests caused by setting default preview generator
ThomasJejkal Apr 10, 2025
5cefe53
Merge branch 'development' of https://github.com/kit-data-manager/ro-…
ThomasJejkal Apr 10, 2025
2c8dcb5
Manual dependency updates
ThomasJejkal Apr 10, 2025
2c81192
chore(deps): update dependency font-awesome to v6
renovate[bot] Apr 10, 2025
476adec
Merge pull request #240 from kit-data-manager/renovate/font-awesome-6.x
ThomasJejkal Apr 10, 2025
4511404
chore(deps): update dependency org.junit:junit-bom to v5.12.2
renovate[bot] Apr 11, 2025
b2dbc48
chore(deps): update dependency commons-io:commons-io to v2.19.0
renovate[bot] Apr 12, 2025
bd277ad
Merge pull request #243 from kit-data-manager/renovate/org.junit-juni…
ThomasJejkal Apr 15, 2025
67480bd
Merge pull request #244 from kit-data-manager/renovate/commons-io-com…
ThomasJejkal Apr 15, 2025
03bfc08
refactor: rename internal field 'url' to 'urls'
Pfeil Apr 15, 2025
83f5d51
feat: add getter for context pair values
Pfeil Apr 15, 2025
1786cdb
feat: add methods to retrieve immutable keys and pairs from metadata …
Pfeil Apr 15, 2025
6a940e8
refactor: rename metadata context accessors to use "read" prefix to i…
Pfeil Apr 15, 2025
8322b18
refactor: avoid repeated test preparation (reading crate with complex…
Pfeil Apr 15, 2025
be0aaeb
refactor: satisfy linter warnings
Pfeil Apr 15, 2025
d5af790
test: verify immutability of views on the context
Pfeil Apr 15, 2025
597568d
refactor: rename context accessors to use "get" prefix for consistency
Pfeil Apr 15, 2025
3274fa3
Merge pull request #245 from kit-data-manager/241-getting-key-value-p…
Pfeil Apr 15, 2025
cb08ff0
refactor: rename ROCratePreviewModel to CustomPreviewModel and update…
Pfeil Apr 16, 2025
470e5d4
preview: enable auto-escaping for html
Pfeil Apr 16, 2025
c698a2f
chore: remove debug echo statements from gradle.yml
Pfeil Apr 16, 2025
350ad00
refactor: enhance JsonLdExpander with improved null handling and docu…
Pfeil Apr 16, 2025
28f86c0
refactor: improve logging and error handling in PreviewGenerator
Pfeil Apr 16, 2025
8fb9342
refactor: improve consistency in ZipStreamWriterTest
Pfeil Apr 16, 2025
72b5187
refactor: remove unused import in RoCrateReader
Pfeil Apr 16, 2025
bef3db7
refactor: remove redundant save call in PreviewCrateTest and generate…
Pfeil Apr 16, 2025
5fdb5f0
refactor: add null check for otherFiles in saveAllToStream method
Pfeil Apr 16, 2025
95fbe25
cleanup: fix typo in test string
Pfeil Apr 16, 2025
5a9555f
fix: correct logger reference in StreamReaderStrategy interface
Pfeil Apr 16, 2025
578ad33
docs: add interface description for StreamReaderStrategy
Pfeil Apr 16, 2025
c151ad3
fix: update error message for StreamReaderStrategy implementation
Pfeil Apr 16, 2025
bb4796f
feat: add footer to custom preview with attribution message
Pfeil Apr 16, 2025
7024885
fix: add null checks for metadata JSON and content files in rebuildCr…
Pfeil Apr 16, 2025
9779b0b
fix: remove redundant save call in PreviewCrateTest
Pfeil Apr 16, 2025
6bde4de
docs: add class-level documentation for ZipStreamReader
Pfeil Apr 16, 2025
222a000
docs: correct typo in method documentation for readCrateMetadata
Pfeil Apr 16, 2025
f76db9d
fix: validate extracted file paths to prevent directory traversal vul…
Pfeil Apr 16, 2025
5f16590
fix: add missing parentheses in README example
Pfeil Apr 16, 2025
4736e28
fix: replace get() with path() for safer JSON node access in CustomPr…
Pfeil Apr 16, 2025
fe1bb10
fix: add null and existence checks for JSON-LD file in JsonLdExpander
Pfeil Apr 16, 2025
d5eb384
feat: introduce generic readers instead of the StreamReaderStrategy.
Pfeil Apr 22, 2025
cd8aa1c
refactor: replace deprecated readers in tests with new ones
Pfeil Apr 22, 2025
2d669d5
fix(test): add missing assertions about profile presence
Pfeil Apr 22, 2025
06d2ce6
test: add tests for convenience APIs
Pfeil Apr 22, 2025
d44defa
feat: introduce generic readers instead of the StreamReaderStrategy
Pfeil Apr 22, 2025
2ba9b01
feat: rename strategies from the scheme $(format)Writer to $(format)S…
Pfeil Apr 23, 2025
4ee8c9c
refactor: apply smaller linter suggestions about best practices
Pfeil Apr 23, 2025
b341acf
feat: add new writer factory methods for folder, zip stream, and zip …
Pfeil Apr 23, 2025
37ca46d
Fix #101, updated ror provider to v2, adapted tests
ThomasJejkal Apr 23, 2025
0ab3f3c
Merged with remote dev
ThomasJejkal Apr 23, 2025
4d572f0
feat: rename newZipFileWriter to newZipPathWriter for consistency wit…
Pfeil Apr 24, 2025
bef5f67
refactor(test): Unify reader tests and use new API
Pfeil Apr 25, 2025
a4e0ad7
refactor(test): rename ZipStreamStrategyTest to ZipStreamReaderTest f…
Pfeil Apr 28, 2025
6fd11da
refactor(test): unify writer tests
Pfeil Apr 28, 2025
532880c
cleanup: remove whitespace
Pfeil Apr 28, 2025
45b2be4
refactor(test): replace deprecated writer use in specific writer test
Pfeil Apr 28, 2025
d1accd8
refactor(test): enhance CrateWriterTest readability and output
Pfeil Apr 29, 2025
3cfda3a
refactor(test): make testFilesBeingAdjusted understandable
Pfeil Apr 29, 2025
2cbe30e
refactor(ZipUtil): improve method documentation and formatting
Pfeil Apr 29, 2025
378b81e
test(CrateWriterTest): verify presence of subdirectory in extraction …
Pfeil Apr 29, 2025
a81f8f8
fix(zip writers): ensure the dataset also sets its name properly in t…
Pfeil Apr 29, 2025
c14350c
test(CrateWriterTest): test and document subdirectory behavior
Pfeil Apr 29, 2025
1ce985c
refactor(DataEntity): improve Javadoc formatting and remove unused im…
Pfeil Apr 29, 2025
5f59145
Merge pull request #247 from kit-data-manager/generalize-reading-and-…
Pfeil Apr 29, 2025
14ebbd3
refactor(RorProvider): replace get with path for safer JSON access
Pfeil Apr 29, 2025
8bffc90
cleanup: formatting and smaller linter proposals
Pfeil Apr 29, 2025
2f54be8
refactor(ZipStrategy): use try-with-resources for InputStream management
Pfeil Apr 29, 2025
0f30a14
cleanup(ZipStrategy): do not put everything into the try block
Pfeil Apr 29, 2025
cafe696
refactor(CrateReader): improve file resolution with path normalizatio…
Pfeil Apr 29, 2025
3f2c1cb
refactor(CrateReader): enforce non-null checks for metadataJson and f…
Pfeil Apr 29, 2025
e1644dd
Merge branch 'development' into fix-renaming-of-data-set-entities
Pfeil Apr 29, 2025
d966a15
Merge pull request #253 from kit-data-manager/fix-renaming-of-data-se…
Pfeil Apr 29, 2025
e71f132
cleanup: delete JsonLdExpander.java
Pfeil Apr 29, 2025
05d1edf
feat: add Javadoc generation step to CI pipeline
Pfeil Apr 29, 2025
d7bc033
refactor: use new writer APIs in all remaining tests
Pfeil Apr 29, 2025
fae0067
fix: change cfg to local variable in CustomPreview constructor
Pfeil Apr 29, 2025
8fe411d
fix: update test cases to use more complex crates
Pfeil Apr 29, 2025
8b4e4b4
fix: wrong indentation in CI yaml
Pfeil Apr 29, 2025
880df80
fix: add dependency on build step for Javadoc generation in CI
Pfeil Apr 29, 2025
b6614ad
cleanup: remove unused json test files
Pfeil Apr 29, 2025
bda49bc
fix: add newline at end of gradle.yml for consistency
Pfeil Apr 29, 2025
521ca55
fix: remove unnecessary Javadoc build step from CI configuration
Pfeil Apr 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,7 @@ env:
COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}

jobs:
# avoid double CI runs on push and PR, from https://github.com/orgs/community/discussions/57827
build:
if: github.event_name != 'push' || github.event.push.head.repo.full_name != github.event.push.base.repo.full_name
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
Expand Down Expand Up @@ -49,3 +47,6 @@ jobs:
- name: Do one Coveralls test report
if: matrix.os == 'ubuntu-latest' && matrix.jdk == 21
run: ./gradlew -Dprofile=release jacocoTestReport coveralls
- name: Compile Javadoc
if: matrix.os == 'ubuntu-latest' && matrix.jdk == 21
run: ./gradlew javadoc
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -169,4 +169,7 @@ gradle-app.setting
# gradle/wrapper/gradle-wrapper.properties

### Gradle Patch ###
.DS_Store
**/build/
**/jte-classes
**/temp
49 changes: 37 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,18 +117,24 @@ PersonEntity person = ORCIDProvider.getPerson("https://orcid.org/*")
OrganizationEntity organization = RORProvider.getOrganization("https://ror.org/*");
```

### Writing Crate to folder or zip file
### Writing Crate to folder, zip file, or zip stream

Writing to folder:
```java
RoCrateWriter folderRoCrateWriter = new RoCrateWriter(new FolderWriter());
folderRoCrateWriter.save(roCrate, "destination");
folderRoCrateWriter.save(roCrate, "destinationFolder");
```

Writing to zip file:
```java
RoCrateWriter roCrateZipWriter = new RoCrateWriter(new ZipWriter());
roCrateZipWriter.save(roCrate, "destination");
roCrateZipWriter.save(roCrate, "destinationFolder");
```

Writing to zip stream:
```java
RoCrateWriter roCrateZipStreamWriter = new RoCrateWriter(new ZipStreamWriter());
roCrateZipStreamWriter.save(roCrate, outputStream);
```

More writing strategies can be implemented, if required.
Expand All @@ -138,25 +144,48 @@ More writing strategies can be implemented, if required.
Reading from folder:
```java
RoCrateReader roCrateFolderReader = new RoCrateReader(new FolderReader());
RoCrate res = roCrateFolderReader.readCrate("source");
RoCrate res = roCrateFolderReader.readCrate("destinationFolder");
```

Reading from zip file:
```java
RoCrateReader roCrateFolderReader = new RoCrateReader(new ZipReader());
RoCrate crate = roCrateFolderReader.readCrate("source");
RoCrate crate = roCrateFolderReader.readCrate("sourceZipFile");
```

Reading from zip stream:
```java
RoCrateReader roCrateFolderReader = new RoCrateReader(new ZipStreamReader());
RoCrate crate = roCrateFolderReader.readCrate(inputStream);
```

### RO-Crate Website (HTML preview file)
By setting the preview to an `AutomaticPreview`, the library will automatically create a preview using the [ro-crate-html-js](https://www.npmjs.com/package/ro-crate-html-js) tool.
It has to be installed using `npm install --global ro-crate-html-js` in order to use it.
If you want to use a custom-made preview, you can set it using the `CustomPreview` class. `AutomaticPreview` is currently **not** set by default.
ro-crate-java offers tree different kinds of previews:

* AutomaticPreview: Uses third-party library [ro-crate-html-js](https://www.npmjs.com/package/ro-crate-html-js), which must be installed separately.
* CustomPreview: Pure Java-based preview using an included template processed by the FreeMarker template engine. At the same time, CustomPreview is the fallback for AutomaticPreview if ro-crate-html-js is not installed.
* StaticPreview: Allows to provide a static HTML page (including additional dependencies, e.g., CSS, JS) which is then shipped with the RO-Crate.

When creating a new RO-Crate using the builder, the default setting is to use CustomPreview. If you want to change this behaviour, thr preview method is set as follows:

```java
RoCrate roCrate = new RoCrateBuilder("name", "description", "datePublished", "licenseIdentifier")
.setPreview(new AutomaticPreview())
.build();
```

Keep in mind that, if you want to use AutomaticPreview, you have to install ro-crate-html-js via `npm install --global ro-crate-html-js` first.

For StaticPreview, the constuctor is a bit different, such that it looks as follows:

```java
File pathToMainPreviewHtml = new File("localPath");
File pathToAdditionalFiles = new File("localFolder");
RoCrate roCrate = new RoCrateBuilder("name", "description", "datePublished", "licenseIdentifier")
.setPreview(new StaticPreview(pathToMainPreviewHtml, pathToAdditionalFiles))
.build();
```

### RO-Crate validation (machine-readable crate profiles)
Right now, the only implemented way of validating a RO-crate is to use a [JSON-Schema](https://json-schema.org/) that the crates metadata JSON file should match. JSON-Schema is an established standard and therefore a good choice for a crate profile. Example:

Expand Down Expand Up @@ -395,8 +424,6 @@ The web resource does not use `.setSource()`, but uses the ID to indicate the fi
{"@id": "data2.txt"}
]
},


{
"@id": "data1.txt",
"@type": "File",
Expand Down Expand Up @@ -637,7 +664,6 @@ If there is no special method for including relative entities (ID properties) on
.addIdProperty("additionalType", nucSec)
.addIdProperty("encodingFormat", fasta)
.build();

ContextualEntity clnParam = new ContextualEntity.ContextualEntityBuilder()
.addType("FormalParameter")
.setId("#6c703fee-6af7-4fdb-a57d-9e8bc4486044")
Expand All @@ -646,7 +672,6 @@ If there is no special method for including relative entities (ID properties) on
.addIdProperty("additionalType", nucSec)
.addIdProperty("encodingFormat", ban)
.build();

ContextualEntity alignParam = new ContextualEntity.ContextualEntityBuilder()
.addType("FormalParameter")
.setId("#2f32b861-e43c-401f-8c42-04fd84273bdf")
Expand Down
20 changes: 14 additions & 6 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
plugins {
id 'java'
id 'jvm-test-suite'

id 'application'
id 'jacoco'
// Adds coveralls task for CI to send results to the coveralls service.
id "com.github.kt3k.coveralls" version "2.12.2"
Expand All @@ -13,7 +13,7 @@ plugins {
// Publishing of JAR to Nexus instances (e.g., OSSRH)
// https://github.com/gradle-nexus/publish-plugin
id "io.github.gradle-nexus.publish-plugin" version "2.0.0"
id "io.freefair.maven-publish-java" version "8.13"
id "io.freefair.maven-publish-java" version "8.13.1"
}

group 'edu.kit.datamanager'
Expand All @@ -24,6 +24,9 @@ println "Building ${name} version: ${version}"
println "JDK version: ${JavaVersion.current()}"
println "Profile (system property): ${System.getProperty('profile')}"

sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17

if (JavaVersion.current() == JavaVersion.VERSION_17) {
println "Setting encoding to UTF-8 manually"
compileJava.options.encoding = "UTF-8"
Expand All @@ -40,7 +43,7 @@ ext {

dependencies {
// JUnit setup for testing
testImplementation(platform("org.junit:junit-bom:5.12.1"))
testImplementation(platform("org.junit:junit-bom:5.12.2"))
testImplementation('org.junit.jupiter:junit-jupiter')
testRuntimeOnly('org.junit.platform:junit-platform-launcher')
// JSON object mapping / (de-)serialization
Expand All @@ -49,11 +52,11 @@ dependencies {
// http client
implementation group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.14'
// common file system operations
implementation group: 'commons-io', name: 'commons-io', version: '2.18.0'
implementation group: 'commons-io', name: 'commons-io', version: '2.19.0'
// read from and write to zip files
implementation group: 'net.lingala.zip4j', name: 'zip4j', version: '2.11.5'
// compare json documents in tests
implementation 'com.github.fslev:json-compare:6.18'
implementation 'com.github.fslev:json-compare:7.0'
// url validator
implementation group: 'commons-validator', name: 'commons-validator', version: '1.9.0'
// logging
Expand All @@ -63,8 +66,13 @@ dependencies {
// metadata validation, profiles based on JSON schema
implementation group: "com.networknt", name: "json-schema-validator", version: "1.5.6"
implementation 'org.glassfish:jakarta.json:2.0.1'
//JTE for template processing
implementation('gg.jte:jte:3.2.0')
implementation("org.freemarker:freemarker:2.3.34")
}

logging.captureStandardOutput LogLevel.INFO

def signingTasks = tasks.withType(Sign)
tasks.withType(AbstractPublishToMaven).configureEach{
mustRunAfter(signingTasks)
Expand Down Expand Up @@ -155,7 +163,7 @@ jacocoTestReport {
}

jacoco {
toolVersion = "0.8.12"
toolVersion = "0.8.13"
}

// maxParallelForks(2)
Expand Down
8 changes: 7 additions & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
version=2.0.3
version=2.0.3
action.custom-1=install
action.custom-1.args=--configure-on-demand -w -x check clean publishToMavenLocal
action.custom-2=jacoco
action.custom-2.args=--configure-on-demand -w clean check jacocoTestReport
action.custom-3=releaseNewVersion
action.custom-3.args=--configure-on-demand -w -x check -Prelease release
22 changes: 22 additions & 0 deletions ro-crate-metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"@context" : "https://w3id.org/ro/crate/1.1/context",
"@graph" : [ {
"name" : "name",
"description" : "description",
"@id" : "./",
"@type" : "Dataset",
"license" : {
"@id" : "https://creativecommons.org/licenses/by-nc-sa/3.0/au/"
},
"datePublished" : "2024"
}, {
"about" : {
"@id" : "./"
},
"conformsTo" : {
"@id" : "https://w3id.org/ro/crate/1.1"
},
"@id" : "ro-crate-metadata.json",
"@type" : "CreativeWork"
} ]
}
20 changes: 20 additions & 0 deletions src/main/java/edu/kit/datamanager/ro_crate/Crate.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.io.File;
import java.util.Collection;
import java.util.Map;
import java.util.Optional;
import java.util.Set;

Expand Down Expand Up @@ -51,6 +52,25 @@ public interface Crate {

void setMetadataContext(CrateMetadataContext metadataContext);

/**
* Get the value of a key from the metadata context.
* @param key the key to be searched
* @return the value of the key, null if not found
*/
String getMetadataContextValueOf(String key);

/**
* Get an immutable collection of the keys in the metadata context.
* @return the keys in the metadata context
*/
Set<String> getMetadataContextKeys();

/**
* Get an immutable map of the context.
* @return an immutable map containing the context key-value pairs
*/
Map<String, String> getMetadataContextPairs();

RootDataEntity getRootDataEntity();

void setRootDataEntity(RootDataEntity rootDataEntity);
Expand Down
Loading