Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ ENV APP_CONTEXT="mir" \
MCR_OVERRIDE_DIR="/mcr/override/" \
SOLR_CORE="mir" \
SOLR_CLASSIFICATION_CORE="mir-classifications" \
SOLR_MAIN_CONFIGSET="mycore_main" \
SOLR_CLASSIFICATION_CONFIGSET="mycore_classification" \
XMX="1g" \
XMS="1g"
COPY --from=regreb/bibutils --chown=mcr:mcr /usr/local/bin/* /usr/local/bin/
Expand Down
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,17 +67,21 @@ The docker container has its own install script which uses the environment varia
### Environment Variables
| Property | Default, required | Description |
|--------------------------------|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ENABLE_SOLR_CLOUD | false | If true the Solr Cloud mode is enabled. (solr cores will be created on install) |
| SOLR_ADMIN_USER | none | The username for the Solr Admin. (will be used for admin commands like creating cores) |
| ENABLE_SOLR_CLOUD | false | If true the Solr Cloud mode is enabled. (solr cores/collections will be created on install). Sets `MCR.Solr.IndexRegistry.Index.*.Class` to `MCRConfigurableSolrCloudCollection` instead of `MCRConfigurableSolrCore`. |
| SOLR_URL | none | The HTTP URL to the Solr server. In standalone mode sets `MCR.Solr.IndexRegistry.Index.*.SolrUrl`. In cloud mode sets `MCR.Solr.IndexRegistry.Index.*.SolrUrls`. Not required if `SOLR_ZK_HOST` is set in cloud mode. |
| SOLR_ZK_HOST | none | ZooKeeper URL(s) for SolrCloud (comma-separated). Sets `MCR.Solr.IndexRegistry.Index.*.ZkUrls`. Only used when `ENABLE_SOLR_CLOUD=true`. Takes precedence over `SOLR_URL` for the connection when both are set. |
| SOLR_ZK_CHROOT | none | Optional ZooKeeper chroot path for SolrCloud. Sets `MCR.Solr.IndexRegistry.Index.*.ZkChroot`. Only used when `SOLR_ZK_HOST` is set. |
| SOLR_CORE | mir | In standalone mode: the core name (`MCR.Solr.IndexRegistry.Index.main.CoreName`). In cloud mode: the collection name (`MCR.Solr.IndexRegistry.Index.main.CollectionName`). |
| SOLR_CLASSIFICATION_CORE | mir-classifications | In standalone mode: the classification core name (`MCR.Solr.IndexRegistry.Index.classification.CoreName`). In cloud mode: the classification collection name (`MCR.Solr.IndexRegistry.Index.classification.CollectionName`). |
| SOLR_MAIN_CONFIGSET | mycore_main | The Solr configset template for the main collection. Sets `MCR.Solr.IndexRegistry.Index.main.ConfigSetTemplate`. Only used in cloud mode. |
| SOLR_CLASSIFICATION_CONFIGSET | mycore_classification | The Solr configset template for the classification collection. Sets `MCR.Solr.IndexRegistry.Index.classification.ConfigSetTemplate`. Only used in cloud mode. |
| SOLR_ADMIN_USER | none | The username for the Solr Admin. (will be used for admin commands like creating cores). Sets `MCR.Solr.Server.Auth.Admin.*`. |
| SOLR_ADMIN_PASSWORD | none | The password for the Solr Admin. |
| SOLR_INDEX_USER | none | The username for the Solr Indexer. (will be used for indexing) |
| SOLR_INDEX_USER | none | The username for the Solr Indexer. (will be used for indexing). Sets `MCR.Solr.Server.Auth.Index.*`. |
| SOLR_INDEX_PASSWORD | none | The password for the Solr Indexer. |
| SOLR_SEARCH_USER | none | The username for the Solr Searcher. (will be used for searching) |
| SOLR_SEARCH_USER | none | The username for the Solr Searcher. (will be used for searching). Sets `MCR.Solr.Server.Auth.Search.*`. |
| SOLR_SEARCH_PASSWORD | none | The password for the Solr Searcher. |
| TIKASERVER_URL | none | The URL to the Tika Server. Same as `MCR.Solr.Tika.ServerURL` in `mycore.properties`. (also sets `MCR.Solr.FileIndexStrategy` to `org.mycore.solr.index.file.tika.MCRTikaSolrFileStrategy`) |
| SOLR_URL | none, required | The URL to the SOLR Server. Same as `MCR.Solr.ServerURL` in `mycore.properties`. |
| SOLR_CORE | mir | The name of the Solr main core. Same as `MCR.Solr.Core.main.Name` in `mycore.properties`. |
| SOLR_CLASSIFICATION_CORE | mir-classifications | The name of the Solr classification core. Same as `MCR.Solr.Core.classification.Name` in `mycore.properties`. |
| JDBC_NAME | none, required | The username for the Database authentication. Same as `javax.persistence.jdbc.user` in `persistence.xml`. |
| JDBC_PASSWORD | none, required | The password for the Database authentication. Same as `javax.persistence.jdbc.password` in `persistence.xml`. |
| JDBC_DRIVER | none, required | The driver for the Database. Same as `javax.persistence.jdbc.driver` in `persistence.xml`. If you use `org.postgresql.Driver`, `org.mariadb.jdbc.Driver`, `org.hsqldb.jdbcDriver`, `org.h2.Driver` or `com.mysql.jdbc.Driver`, the right database drivers get downloaded by the installer script. |
Expand Down
75 changes: 65 additions & 10 deletions docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,71 @@ function setOrAddProperty() {
function setDockerValues() {
echo "Set Docker Values to Config!"

if [ -n "${SOLR_URL}" ]; then
setOrAddProperty "MCR.Solr.ServerURL" "${SOLR_URL}"
fi

if [ -n "${SOLR_CORE}" ]; then
setOrAddProperty "MCR.Solr.Core.main.Name" "${SOLR_CORE}"
fi

if [ -n "${SOLR_CLASSIFICATION_CORE}" ]; then
setOrAddProperty "MCR.Solr.Core.classification.Name" "${SOLR_CLASSIFICATION_CORE}"
if [[ "$ENABLE_SOLR_CLOUD" == "true" ]]; then
# SolrCloud mode
if [ -n "${SOLR_ZK_HOST}" ]; then
# Connect via ZooKeeper
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.Class" "org.mycore.solr.cloud.collection.MCRConfigurableSolrCloudCollection"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.ZkUrls" "${SOLR_ZK_HOST}"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.Class" "org.mycore.solr.cloud.collection.MCRConfigurableSolrCloudCollection"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.ZkUrls" "${SOLR_ZK_HOST}"
if [ -n "${SOLR_ZK_CHROOT}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.ZkChroot" "${SOLR_ZK_CHROOT}"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.ZkChroot" "${SOLR_ZK_CHROOT}"
fi
elif [ -n "${SOLR_URL}" ]; then
# Connect via Solr HTTP URL
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.Class" "org.mycore.solr.cloud.collection.MCRConfigurableSolrCloudCollection"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.SolrUrls" "${SOLR_URL}"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.Class" "org.mycore.solr.cloud.collection.MCRConfigurableSolrCloudCollection"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.SolrUrls" "${SOLR_URL}"
fi
if [ -n "${SOLR_CORE}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.CollectionName" "${SOLR_CORE}"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.ConfigSetTemplate" "${SOLR_MAIN_CONFIGSET:-mycore_main}"
fi
if [ -n "${SOLR_CLASSIFICATION_CORE}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.CollectionName" "${SOLR_CLASSIFICATION_CORE}"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.ConfigSetTemplate" "${SOLR_CLASSIFICATION_CONFIGSET:-mycore_classification}"
fi
if [ -n "${SOLR_CORE_NUM_SHARDS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.NumShards" "${SOLR_CORE_NUM_SHARDS}"
fi
if [ -n "${SOLR_CORE_NUM_NRT_REPLICAS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.NumNrtReplicas" "${SOLR_CORE_NUM_NRT_REPLICAS}"
fi
if [ -n "${SOLR_CORE_NUM_TLOG_REPLICAS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.NumTlogReplicas" "${SOLR_CORE_NUM_TLOG_REPLICAS}"
fi
if [ -n "${SOLR_CORE_NUM_PULL_REPLICAS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.NumPullReplicas" "${SOLR_CORE_NUM_PULL_REPLICAS}"
fi
if [ -n "${SOLR_CLASSIFICATION_CORE_NUM_SHARDS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.NumShards" "${SOLR_CLASSIFICATION_CORE_NUM_SHARDS}"
fi
if [ -n "${SOLR_CLASSIFICATION_CORE_NUM_NRT_REPLICAS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.NumNrtReplicas" "${SOLR_CLASSIFICATION_CORE_NUM_NRT_REPLICAS}"
fi
if [ -n "${SOLR_CLASSIFICATION_CORE_NUM_TLOG_REPLICAS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.NumTlogReplicas" "${SOLR_CLASSIFICATION_CORE_NUM_TLOG_REPLICAS}"
fi
if [ -n "${SOLR_CLASSIFICATION_CORE_NUM_PULL_REPLICAS}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.NumPullReplicas" "${SOLR_CLASSIFICATION_CORE_NUM_PULL_REPLICAS}"
fi
else
# Standalone mode
if [ -n "${SOLR_URL}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.Class" "org.mycore.solr.standalone.core.MCRConfigurableSolrCore"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.SolrUrl" "${SOLR_URL}"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.Class" "org.mycore.solr.standalone.core.MCRConfigurableSolrCore"
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.SolrUrl" "${SOLR_URL}"
fi
if [ -n "${SOLR_CORE}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.main.CoreName" "${SOLR_CORE}"
fi
if [ -n "${SOLR_CLASSIFICATION_CORE}" ]; then
setOrAddProperty "MCR.Solr.IndexRegistry.Index.classification.CoreName" "${SOLR_CLASSIFICATION_CORE}"
fi
fi

if [ -n "${JDBC_NAME}" ]; then
Expand Down
16 changes: 14 additions & 2 deletions mir-it/src/test/integration/mycore.properties
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
MCR.Solr.ServerURL=http\://localhost\:${solr.port}/
MCR.Solr.DelayIndexing_inMS=200
MCR.Solr.SolrClient.SocketTimeout=70000

MCR.Solr.Default.Client.RequestTimeout=70
MCR.Solr.Default.Client.RequestTimeout.Unit=SECONDS

MCR.Solr.Default.Client.IdleTimeout=70
MCR.Solr.Default.Client.IdleTimeout.Unit=SECONDS

MCR.Solr.IndexRegistry.Index.classification.Class=org.mycore.solr.cloud.collection.MCRConfigurableSolrCloudCollection
MCR.Solr.IndexRegistry.Index.classification.CollectionName=mir-classifications
MCR.Solr.IndexRegistry.Index.classification.SolrUrls=http\://localhost\:${solr.port}/

MCR.Solr.IndexRegistry.Index.main.Class=org.mycore.solr.cloud.collection.MCRConfigurableSolrCloudCollection
MCR.Solr.IndexRegistry.Index.main.CollectionName=mir
MCR.Solr.IndexRegistry.Index.main.SolrUrls=http\://localhost\:${solr.port}/

MCR.Solr.Server.Auth.Admin.Class=org.mycore.solr.auth.MCRSolrPropertyBasicAuthenticator
MCR.Solr.Server.Auth.Admin.Password=alleswirdgut
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@
package org.mycore.mir.impexp;

import java.io.IOException;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.request.QueryRequest;
import org.apache.solr.client.solrj.request.SolrQuery;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.jdom2.Element;
import org.jdom2.filter.Filters;
import org.jdom2.input.DOMBuilder;
import org.jdom2.xpath.XPathExpression;
import org.jdom2.xpath.XPathFactory;
import org.mycore.common.MCRConstants;
import org.mycore.solr.MCRXMLFunctions;
import org.mycore.solr.MCRSolrIndexRegistryManager;
import org.mycore.solr.auth.MCRSolrAuthenticationLevel;
import org.mycore.solr.auth.MCRSolrAuthenticationManager;
import org.w3c.dom.NodeList;

/**
Expand All @@ -18,6 +25,26 @@ public class MIRRelatedItemFinderUtils {

private static final Logger LOGGER = LogManager.getLogger();

private static String getIdentifierOfFirst(String q) throws SolrServerException, IOException {
if (q == null || q.isEmpty()) {
throw new IllegalArgumentException("The query string must not be null");
}
SolrQuery solrQuery = new SolrQuery(q);
solrQuery.set("rows", 1);
QueryResponse queryResponse;
QueryRequest queryRequest = new QueryRequest(solrQuery);
MCRSolrAuthenticationManager.obtainInstance().applyAuthentication(queryRequest,
MCRSolrAuthenticationLevel.SEARCH);
queryResponse = queryRequest.process(MCRSolrIndexRegistryManager.obtainRegistry()
.requireMainIndex().getClient());

if (queryResponse.getResults().getNumFound() == 0) {
return null;
}

return queryResponse.getResults().getFirst().get("id").toString();
}

public static String findRelatedItem(final NodeList sources) {
if (sources.getLength() == 0) {
LOGGER.warn("Cannot get first element of node list 'sources'.");
Expand All @@ -32,14 +59,14 @@ public static String findRelatedItem(final NodeList sources) {
String mcrID = "";
try {
if (identifierElement != null) {
mcrID = MCRXMLFunctions.getIdentifierOfFirst("mods.identifier:\"" + identifierElement.getText() + "\"");
mcrID = getIdentifierOfFirst("mods.identifier:\"" + identifierElement.getText() + "\"");
}
if (identifierElement == null || mcrID == null) {
XPathExpression<Element> xpathTitle = XPathFactory.instance().compile("mods:titleInfo/mods:title",
Filters.element(), null, MCRConstants.MODS_NAMESPACE, MCRConstants.XLINK_NAMESPACE);
Element titleElement = xpathTitle.evaluateFirst(relatedItem);
if (titleElement != null) {
mcrID = MCRXMLFunctions.getIdentifierOfFirst("mods.title.main:\"" + titleElement.getText() + "\"");
mcrID = getIdentifierOfFirst("mods.title.main:\"" + titleElement.getText() + "\"");
}
}
} catch (Exception e) {
Expand Down
7 changes: 5 additions & 2 deletions mir-module/src/main/resources/config/mir/mycore.properties
Original file line number Diff line number Diff line change
Expand Up @@ -253,8 +253,11 @@ MCR.mir-module.sendEditorMailToCurrentAuthor=false
##############################################################################
# SOLR #
##############################################################################
MCR.Solr.Core.main.Name=%MIR.projectid.default%
MCR.Solr.Core.classification.Name=%MCR.Solr.Core.main.Name%-classifications
MCR.Solr.IndexRegistry.Index.main.CoreName=%MIR.projectid.default%
MCR.Solr.IndexRegistry.Index.classification.CoreName=%MCR.Solr.Core.main.Name%-classifications
MCR.Solr.IndexRegistry.Index.main.CollectionName=%MIR.projectid.default%
MCR.Solr.IndexRegistry.Index.classification.CollectionName=%MCR.Solr.Core.main.Name%-classifications

MCR.Category.LinkService=org.mycore.solr.classification.MCRSolrCategLinkService
MCR.Category.DAO=org.mycore.solr.classification.MCRSolrCategoryDAO
MCR.Solr.SolrInputDocument.Path.Factory=org.mycore.mir.index.MirPathDocumentFactory
Expand Down
Loading
Loading