support central marketplace solution

Author: wistefan

charts/data-space-connector/Chart.yaml

@@ -62,7 +62,7 @@ dependencies:
     repository: https://fiware.github.io/helm-charts
   - name: contract-management
     condition: contract-management.enabled
-    version: 3.5.4
+    version: 3.5.6
     repository: https://fiware.github.io/helm-charts
   # marketplace
   - name: business-api-ecosystem

charts/data-space-connector/templates/did-ingress.yaml

@@ -5,11 +5,19 @@ metadata:
   name: did-json
   namespace: {{ $.Release.Namespace | quote }}
   annotations:
-    traefik.ingress.kubernetes.io/router.tls: "true"
+    {{- with .Values.didJson.ingress.annotations }}
+      {{- toYaml . | nindent 4 }}
+    {{- end }}
     traefik.ingress.kubernetes.io/service.passhostheader: "true"
   labels:
     {{ include "dsc.labels" . | nindent 4 }}
 spec:
+  {{- if .Values.didJson.ingress.tlsSecret }}
+  tls:
+    - hosts:
+      - {{ .Values.didJson.ingress.host }}
+      secretName: {{ .Values.didJson.ingress.tlsSecret }}
+  {{- end }}
   rules:
     - host: {{ .Values.didJson.ingress.host }}
       http:

charts/data-space-connector/templates/realm.yaml

@@ -11,8 +11,8 @@ data:
     {
       "id": "{{ .Values.keycloak.realm.name }}",
       "realm": "{{ .Values.keycloak.realm.name }}",
-      "displayName": "Keycloak",
-      "displayNameHtml": "<div class=\"kc-logo-text\"><span>Keycloak</span></div>",
+      "displayName": "{{ .Values.keycloak.realm.name }}",
+      "displayNameHtml": "<div class=\"kc-logo-text\"><span>{{ .Values.keycloak.realm.name }}</span></div>",
       "verifiableCredentialsEnabled": true,
       "enabled": true,
       "attributes": {

doc/CENTRAL_MARKETPLACE.md

@@ -1,88 +1,119 @@
-![CENTRAL MARKETPLACE](./img/central_market.png)
-
+# Support for a Central Marketplace
 
-1. A Provider employee registers the marketplace in the Provider:
+Various Data Space Use-Cases require a solution for offering services through a central Marketplace, in order to lower requirements for participants and improve visibility of their offerings.
 
-    1.1. As a trusted-issuer of credentials("MarketplaceCredential") required to send notifications
+## Architecture
 
-    * check if Gaia-X credential for that purpose exists
 
-    1.2. Register policies that allow the marketplace to send order notifications to the Contract-Management
+![CENTRAL MARKETPLACE](./img/central_market.png)
 
-    * should f.e. check the offer-id
+Integration of Participants into a Central Marketplace happens through the same mechanisms and components as the normal [marketplace integration](MARKETPLACE_INTEGRATION.md).
+Particpants offering their services on the Central Marketplace are required to provide an instance of the [ContractManagement](https://github.com/FIWARE/contract-management). The Central Marketplace also runs the Contract Management Component and manages Organizations, Products and Offerings through the TMForum API. 
 
-2. Provider employee creates the product-offering at the BAE-Marketplace
+The basic flow of integration is as following:
 
-* needs to contain the Providers Contract-Management address -> Options: "place"/"channel" in offering, "productSpecCharacteristic" in spec
+1. The Data Provider has to prepare its integration by:
 
-* needs to contain accessible endpoints for the customer(same options)
+    1.1. Register the Marketplace as a trusted-issuer of credentials used by the Contract Management
+    1.2. Register policies that allow the marketplace to send order notifications to the Contract-Management(see [allowContractManagement.json](../it/src/test/resources/policies/allowContractManagement.json) as an example) 
+    1.3 Provider has to register itself at the Central Marketplace as an Oranization, containing access information to the Contract Management
 
-3. Customer buys access at the BAE-Marketplace
+2. Create Product Offering
 
-* BAE interacts with TMForum
-* Contract-Management(Marketplace) receives notifications from TMForum
+3. Customer buys access at the Marketplace
 
 4. Contract Management sends notifcation to the Providers Contract-Management
 
-    4.1. Authenticates with VC at the verifier
-
-    * Credential could either be pre-issued, created via library or a helper component -> to be decided
-    
+    4.1. Authenticates with the Verifiable Credential configured in 1.1 at the verifier
     4.2. Send order notifications to the Contract-Management(through PEP)
 
 5. Contract-Management activates the service:
 
     5.1. Adds the customer to the trusted-issuers-list(according to the order)
-
     5.2. Creates the policies from the order
 
-Open:
+## Demo
+
+In order to run the Demo flows, the [Local Deployment](./deployment-integration/local-deployment/LOCAL.MD) has to be prepared.
+In the Demo-Scenario, the Consumer-Organization("fancy-marketplace.biz") also acts as the provider of the centralized marketplace, while the Provider-Organization("mp-operations.org") offers its services through that market. 
+
+### Prepare the marketplace
+
+Register the policies required to restrict access to the TMForum. Only Users in the Role "REPRESENTATIVE" should be allowed to interact with it.
+
+```shell
+  ./doc/scripts/prepare-central-market-policies.sh
+```
+
+### Prepare the provider [(Step 1)](#architecture)
+
+Create a policy, restricting access to the Contract Management to requests authenticated with a "MarketplaceCredential"
 
-* notify product to the provider -> handling of cancellation etc.?
+```shell
+# Allow contract management access at the provider side
+curl -X 'POST' http://pap-provider.127.0.0.1.nip.io:8080/policy \
+    -H 'Content-Type: application/json' \
+    -d "$(cat ./it/src/test/resources/policies/allowContractManagement.json)"
+```
 
-1. Register the Provider at the Marketplace:
+Get a "LegalPersonCredential" for the Provider, containing the role "REPRESENTATIVE".(see [Provider` ClientScopes config](../k3s/provider.yaml#957))
 
 ```shell
-  export FANCY_MARKETPLACE_ID=$(curl -X POST http://mp-tmf-api.127.0.0.1.nip.io:8080/tmf-api/party/v4/organization \
+export PROVIDER_USER_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-provider.127.0.0.1.nip.io user-credential employee); echo ${PROVIDER_USER_CREDENTIAL}
+```
+
+Register the Provider at the Marketplace, containing the address of the Contract Management and the required clientId/scope for authentication:
+
+```shell
+  # set the Providers DID
+  export PROVIDER_DID="did:web:mp-operations.org"
+  # get an AccessToken for the Credential
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $PROVIDER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  # create the organization at the providers TMForum
+  export MP_OPERATIONS_ID=$(curl -X POST http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/party/v4/organization \
     -H 'Accept: */*' \
     -H 'Content-Type: application/json' \
     -H "Authorization: Bearer ${ACCESS_TOKEN}" \
     -d "{
-      \"name\": \"Fancy Marketplace Inc.\",
+      \"name\": \"M&P Operations Org.\",
       \"partyCharacteristic\": [
         {
             \"name\": \"did\",
-            \"value\": \"${CONSUMER_DID}\" 
+            \"value\": \"${PROVIDER_DID}\" 
         },
         {
-            \"name\": \"contract-management\",
+            \"name\": \"contractManagement\",
             \"value\": {
-                \"address\": \"https://contract-management.data-provider.io\",
+                \"address\": \"http://contract-management.127.0.0.1.nip.io:8080\",
                 \"clientId\":\"contract-management\",
-                // maybe
-                \"scope\": [\"openid\", \"external-marketplace\"]  
-            },
-            \"@schemaLocation\": \"to-be-created\"
+                \"scope\": [\"external-marketplace\"]  
+            }
         }
       ]
-    }" | jq '.id' -r); echo ${FANCY_MARKETPLACE_ID} 
+    }" | jq '.id' -r); echo ${MP_OPERATIONS_ID} 
 ```
 
-2. Create specification, referencing the provider:
+### Create the Offering [(Step 2)](#architecture)
+
+Create product specification, referencing the provider:
+
 ```shell
-   export PRODUCT_SPEC_FULL_ID=$(curl -X 'POST' http://tm-forum-api.127.0.0.1.nip.io:8080/tmf-api/productCatalogManagement/v4/productSpecification \
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $PROVIDER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  export PRODUCT_SPEC=$(curl -X 'POST' http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/productCatalogManagement/v4/productSpecification \
+  -H 'Accept: */*' \
   -H 'Content-Type: application/json;charset=utf-8' \
+  -H "Authorization: Bearer ${ACCESS_TOKEN}" \
   -d "{
     \"brand\": \"M&P Operations\",
     \"version\": \"1.0.0\",
     \"lifecycleStatus\": \"ACTIVE\",
     \"name\": \"M&P K8S\",
     \"relatedParty\": [
         {
-            \"id\": \"${FANCY_MARKETPLACE_ID}\",
+            \"id\": \"${MP_OPERATIONS_ID}\",
             \"role\": \"provider\"
         }
-    ]
+    ],
     \"productSpecCharacteristic\": [
       {
         \"id\": \"credentialsConfig\",
@@ -170,7 +201,128 @@ Open:
         ]
       }
     ]
-  }" | jq '.id' -r ); echo ${PRODUCT_SPEC_FULL_ID}
+  }" | jq '.id' -r ); echo ${PRODUCT_SPEC}
+```
+
+Create product offering, referencing the product spec:
+
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $PROVIDER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  export PRODUCT_OFFERING_ID=$(curl -X 'POST' http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/productCatalogManagement/v4/productOffering \
+    -H 'Accept: */*' \
+    -H 'Content-Type: application/json;charset=utf-8' \
+    -H "Authorization: Bearer ${ACCESS_TOKEN}" \
+    -d "{
+      \"version\": \"1.0.0\",
+      \"lifecycleStatus\": \"ACTIVE\",
+      \"name\": \"M&P K8S Offering\",
+      \"productSpecification\": {
+        \"id\": \"${PRODUCT_SPEC}\"
+      }
+    }" | jq '.id' -r ); echo ${PRODUCT_OFFERING_ID}
+```
+
+
+### Buy access to the offering [(Step 2)](#architecture)
+
+In order to keep the demo environment manageable, the Consumer-Organization will buy access to the Providers offering: 
+
+Create a UserCredential for the consumer:
+
+```shell
+export CONSUMER_USER_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io user-sd employee vc+sd-jwt); echo ${CONSUMER_USER_CREDENTIAL}
+```
+Create an OperatorCredential for the consumer: 
+
+```shell
+export CONSUMER_OPERATOR_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io operator-credential operator); echo ${CONSUMER_OPERATOR_CREDENTIAL}
+```
+
+Assert that access is not yet possible(e.g. the Consumer cannot get an AccessToken for the OperatorCredential):
+
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://mp-data-service.127.0.0.1.nip.io:8080 $CONSUMER_OPERATOR_CREDENTIAL operator); echo ${ACCESS_TOKEN}
+```
+
+Now register "fancy-marketplace.biz" as a Consumer at the Marketplace:
+
+Register Fancy Marketplace as a Consumer
+```shell
+  # set the Consumer DID
+  export CONSUMER_DID="did:web:fancy-marketplace.biz"
+  # get an access token for the UserCredential
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $CONSUMER_USER_CREDENTIAL default); echo ${ACCESS_TOKEN}
+  # register fancy-marketplace.biz as an organization(no need for a contract-management address in the consumer)
+  export FANCY_MARKETPLACE_ID=$(curl -X POST http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/party/v4/organization \
+    -H 'Accept: */*' \
+    -H 'Content-Type: application/json' \
+    -H "Authorization: Bearer ${ACCESS_TOKEN}" \
+    -d "{
+      \"name\": \"Fancy Marketplace Inc.\",
+      \"partyCharacteristic\": [
+        {
+          \"name\": \"did\",
+          \"value\": \"${CONSUMER_DID}\" 
+        }
+      ]
+    }" | jq '.id' -r); echo ${FANCY_MARKETPLACE_ID} 
 ```
 
--> CM extracts spec and provider from the order, extract CM information from org
+List the offerings of the marketplace, only one should be registered at the moment:
+
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $CONSUMER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  curl -X GET http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/productCatalogManagement/v4/productOffering -H "Authorization: Bearer ${ACCESS_TOKEN}" | jq .
+```
+
+
+Extract the OfferId from the TMForum API:
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $CONSUMER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  export OFFER_ID=$(curl -X GET http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/productCatalogManagement/v4/productOffering -H "Authorization: Bearer ${ACCESS_TOKEN}" | jq '.[0].id' -r); echo ${OFFER_ID}
+```
+
+Create an order for the offering:
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $CONSUMER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  export ORDER_ID=$(curl -X POST http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/productOrderingManagement/v4/productOrder \
+  -H 'Accept: */*' \
+  -H 'Content-Type: application/json' \
+  -H "Authorization: Bearer ${ACCESS_TOKEN}" \
+  -d "{
+      \"productOrderItem\": [
+        {
+          \"id\": \"random-order-id\",
+          \"action\": \"add\",
+          \"productOffering\": {
+            \"id\" :  \"${OFFER_ID}\"
+          }
+        }  
+      ],
+      \"relatedParty\": [
+        {
+          \"id\": \"${FANCY_MARKETPLACE_ID}\"
+        }
+      ]}" | jq '.id' -r); echo ${ORDER_ID}
+```
+
+Complete the order:
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://fancy-marketplace.127.0.0.1.nip.io:8080 $CONSUMER_USER_CREDENTIAL default); echo $ACCESS_TOKEN
+  curl -X 'PATCH' \
+      -H "Authorization: Bearer ${ACCESS_TOKEN}" \
+      http://fancy-marketplace.127.0.0.1.nip.io:8080/tmf-api/productOrderingManagement/v4/productOrder/${ORDER_ID} \
+      -H 'accept: application/json;charset=utf-8' \
+      -H 'Content-Type: application/json;charset=utf-8' \
+      -d "{
+              \"state\": \"completed\"
+          }" | jq .
+```
+
+Once the order is completed, the Contract Management of the Marketplace will send a notification to the Providers Contract Management, where the credential configuration and policies from the product will be applied to the Trusted Issuers List and the PAP. 
+
+Now it should be possible to get an AccessToken for the Consumer's OperatorCredential:
+
+```shell
+  export ACCESS_TOKEN=$(./doc/scripts/get_access_token_oid4vp.sh http://mp-data-service.127.0.0.1.nip.io:8080 $CONSUMER_OPERATOR_CREDENTIAL operator); echo ${ACCESS_TOKEN}
+```

doc/CONTRACT_NEGOTIATION.md

@@ -48,7 +48,7 @@ In order to access a Service by using the Transfer Process Protocol, the followi
 
 * Get credentials:
 ```shell
-export USER_CREDENTIAL=$(./doc/scripts/get_credential_for_consumer.sh https://keycloak-consumer.127.0.0.1.nip.io user-credential employee); echo ${USER_CREDENTIAL}
+export USER_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io user-credential employee); echo ${USER_CREDENTIAL}
 ```
 
 * Prepare holder and its did:

doc/DSP_INTEGRATION.md

@@ -250,15 +250,15 @@ In order to use the endpoints, the Consumer needs to issue a User and an Operato
 
 1. Get the UserCredential:
 ```shell
-export USER_CREDENTIAL=$(./doc/scripts/get_credential_for_consumer.sh https://keycloak-consumer.127.0.0.1.nip.io user-credential employee); echo ${USER_CREDENTIAL}
+export USER_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io user-credential employee); echo ${USER_CREDENTIAL}
 ```
 2. Get the OperatorCredential:
 ```shell
-export OPERATOR_CREDENTIAL=$(./doc/scripts/get_credential_for_consumer.sh https://keycloak-consumer.127.0.0.1.nip.io operator-credential operator); echo ${OPERATOR_CREDENTIAL}
+export OPERATOR_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io operator-credential operator); echo ${OPERATOR_CREDENTIAL}
 ```
 3. Get the LegalPersonCredential for the representative:
 ```shell
-export REP_CREDENTIAL=$(./doc/scripts/get_credential_for_consumer.sh https://keycloak-consumer.127.0.0.1.nip.io user-credential representative); echo ${REP_CREDENTIAL}
+export REP_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io user-credential representative); echo ${REP_CREDENTIAL}
 ```
 
 To prepare usage of the credentials, also create the keymaterial for the holder:

doc/MARKETPLACE_INTEGRATION.md

@@ -126,7 +126,7 @@ The following flow will enable a customer to use a service offered by the provid
 
 * Get the "OperatorCredential":
 ```shell
-  export OPERATOR_CREDENTIAL=$(./doc/scripts/get_credential_for_consumer.sh https://keycloak-consumer.127.0.0.1.nip.io operator-credential operator); echo ${OPERATOR_CREDENTIAL}
+  export OPERATOR_CREDENTIAL=$(./doc/scripts/get_credential.sh https://keycloak-consumer.127.0.0.1.nip.io operator-credential operator); echo ${OPERATOR_CREDENTIAL}
 ```
 
 * Try to get an access-token from the Provider - nothing should be returned: