Skip to content

Commit fc585aa

Browse files
skdwritingskdwriting
authored
Merge pull request #25 from nguptaopensds/master
[User-Guide]Added user guide for object lifecycle management
2 parents 6f879a6 + 0b1ecfa commit fc585aa

File tree

10 files changed

+193
-0
lines changed

10 files changed

+193
-0
lines changed

‎content/guides/developer-guides/object-lifecycle-management/_index.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
title: Object Lifecycle Management
3+
description: "Developer guide for Object Lifecycle Management."
4+
weight: 40
5+
disableToc: false
6+
tags: ["developer guide", "object lifecycle"]
7+
---
8+
## Overview
9+
This guide is intended for Object Lifecycle feature developers. It provides detailed steps to write new API using OpenSDS. To understand the feature and requirements in detail, please refer to the Object Lifecycle Management user guide and Object Lifecycle design doc.
10+
11+
## APIs
12+
APIs are a set of functions and procedures that allow for the creation of applications that access data and features of other applications, services or operating system. In OpenSDS lifecycle management feature we are using APIs to communicate with different storage backend.
13+
14+
## Write new API
15+
Before starting to write the APIs, it is required to have basic understanding of XML and JSON syntax.
16+
17+
## How to write the APIs
18+
19+
### There are three major steps involved:
20+
a) All APIs goes into the 's3' module of the OpenSDS multi-cloud package, i,e. opensds/multi-cloud/api/pkg/s3 directory
21+
b) The corresponding router for the API are defined same directory.
22+
c) For every API there must be a function defined in proto file.
23+
Example of adding API router (router.go):
24+
```
25+
func RegisterRouter(ws *restful.WebService) {
26+
handler := NewAPIService(client.DefaultClient)
27+
ws.Route(ws.POST("/{tenantId}/backends/{id}").To(handler.postBackend)).
28+
Doc("post backend details")
29+
```
30+
31+
#### Points to remember:
32+
a) After adding a router to API just create the API file in the same directory.
33+
b) Try to call the API using curl or postman to see if the API is been called successfully.
34+
c) If your API has XML struct in Request body then unmarshal it into json struct
35+
36+
Example of xml (xmlstruct.go):
37+
```
38+
type Rule struct {
39+
ID string `xml:"ID"`
40+
param1 int32 `xml:"param1"`
41+
param2 string `xml:"param2"`
42+
}
43+
```
44+
Example of json struct (in protobuf):
45+
```
46+
type LifecycleRule struct {
47+
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
48+
param1 int32 `protobuf:"bytes,2,opt,name=param1,proto3" json:"param1,omitempty"`
49+
param2 string `protobuf:"bytes,3,opt,name=param2,proto3" json:"param2,omitempty"`
50+
}
51+
```
52+
### Points to remember:
53+
a) After unmarshalling into json struct and populating the fields, call the respective function from the s3 client and also in database.
54+
b) Validate all the functions using log messages.
55+
c) The API should successfully return and all the changes to be reflected in database should be seen in database client(for lifecycle feature the database is MongoDB)
56+
57+
## How to start multi-cloud into the Docker container
58+
To start testing multi-cloud lifecycle feature, you need to first start multi-cloud service using steps below in the docker container.
59+
60+
```
61+
docker-compose up -d
62+
docker ps
63+
```
64+
#### Check if multi-cloud api service and mongo is running:
65+
```
66+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
67+
c245b39b9a7d opensdsio/multi-cloud-dataflow "/dataflow" 6 hours ago Up 6 hours multi-cloud_dataflow_1_b4432dc97238
68+
6bff89fd48b6 opensdsio/multi-cloud-datamover "/datamover" 6 hours ago Up 6 hours multi-cloud_datamover_1_6aa7bb101f33
69+
561d334d2ca8 opensdsio/multi-cloud-s3 "/s3" 6 hours ago Up 6 hours multi-cloud_s3_1_84b4c3f135b2
70+
a7288d668842 opensdsio/multi-cloud-api "/api" 6 hours ago Up 6 hours 0.0.0.0:8089->8089/tcp multi-cloud_api_1_783889b02933
71+
247d70f62574 wurstmeister/kafka:2.11-2.0.1 "start-kafka.sh" 12 days ago Up 6 hours 0.0.0.0:9092->9092/tcp multi-cloud_kafka_1_e666cc1a0f3b
72+
a2c136ae7421 opensdsio/multi-cloud-backend "/backend" 12 days ago Up 6 hours multi-cloud_backend_1_1983cccd8ac9
73+
eea40a48de97 wurstmeister/zookeeper "/bin/sh -c '/usr/sb…" 12 days ago Up 6 hours 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp multi-cloud_zookeeper_1_6643a83ddcdd
74+
5dbac9cca10e mongo "docker-entrypoint.s…" 12 days ago Up 6 hours 0.0.0.0:27017->27017/tcp multi-cloud_datastore_1_a0281d684ef3
75+
```
76+
77+
To check if the API changes are reflected in database, install mongoDB compass and check the metadata of the buckets
78+
79+
![mongo connection image ](opensds_mongo.PNG?raw=true)
80+
81+
#### The postman request for API call:
82+
83+
Here is a sample API call using postman, it contains request body information as well as response.
84+
85+
![postman api call image ](postman.png?raw=true)
86+
87+
Check log files of multi-cloud api service for more details and debug information

‎content/guides/developer-guides/object-lifecycle-management/opensds_mongo.PNG‎

93.2 KB
Loading

‎content/guides/developer-guides/object-lifecycle-management/postman.png‎

72.8 KB
Loading

‎content/guides/user-guides/object-lifecycle-management/_index.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
---
2+
title: Object Lifecycle Management
3+
description: "User guide for Object Lifecycle Management."
4+
weight: 40
5+
disableToc: false
6+
tags: ["user guide", "object lifecycle"]
7+
---
8+
9+
## Introduction to Object lifecycle
10+
OpenSDS is aimed at addressing the storage integration challenges of both the Cloud Native environment and traditional IT environment. OpenSDS multi-cloud allows user to manage distributed cloud environment on a single platform. Object lifecycle management feature gives freedom to user to manage buckets and objects in the cloud using OpenSDS interface.
11+
12+
## Getting Started!
13+
This is a guide that shows how to install, configure, and use Object lifecycle management feature in a simple OpenSDS setup.
14+
By following this guide you will be able to install OpenSDS multi-cloud with creating backends and buckets, uploading object and setting up the lifecycle for those objects.
15+
16+
## Installing Opensds multi-cloud
17+
18+
### Download opensds-installer code
19+
20+
```cassandraql
21+
git clone https://github.com/opensds/opensds-installer.git
22+
cd opensds-installer
23+
# Checkout the latest release, give master by default or specify the branch
24+
git checkout master
25+
```
26+
27+
### How to enable multi-cloud installation
28+
29+
Install docker container to run multi-cloud in container environment.
30+
31+
Update the file : opensds-installer/ansible/group_vars/gelato.yml to install multi-cloud.
32+
```cassandraql
33+
# repository branch
34+
gelato_repo_branch: master
35+
```
36+
Detailed instruction is in this link
37+
38+
- [OpenSDS Local Cluster Installation through Ansible On Ubuntu](https://github.com/opensds/opensds/wiki/OpenSDS-Cluster-Installation-through-Ansible) (Recommended)
39+
40+
#### Export required OpenSDS environment variables
41+
```
42+
export HOST_IP={your_real_host_ip}
43+
export OPENSDS_ENDPOINT=http://{{ apiserver_cluster_ip }}:50040
44+
export OPENSDS_AUTH_STRATEGY=keystone
45+
export OS_AUTH_URL=http://{{ authchecker_cluster_ip }}/identity
46+
```
47+
#### Run OpenSDS multi-cloud in docker container
48+
```
49+
docker-compose up -d
50+
docker ps
51+
```
52+
Note: check if multi-cloud services are running in the docker container.
53+
```cassandraql
54+
# docker ps
55+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
56+
3222f0b4c25d opensdsio/multi-cloud-dataflow "/dataflow" 2 minutes ago Up 2 minutes multi-cloud_dataflow_1_4a975c39f8ff
57+
33de3d47b1ef opensdsio/multi-cloud-datamover "/datamover" 2 minutes ago Up 2 minutes multi-cloud_datamover_1_8ccb3d34f551
58+
3d3661466b3d wurstmeister/kafka:2.11-2.0.1 "start-kafka.sh" 2 minutes ago Up 2 minutes 0.0.0.0:9092->9092/tcp multi-cloud_kafka_1_e399b2c2167a
59+
e370acd6c922 mongo "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:27017->27017/tcp multi-cloud_datastore_1_a8c5edcc4e62
60+
6d5239b01503 wurstmeister/zookeeper "/bin/sh -c '/usr/sb…" 2 minutes ago Up 2 minutes 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp multi-cloud_zookeeper_1_82daeecd7252
61+
7d9f01911356 opensdsio/multi-cloud-backend "/backend" 2 minutes ago Up 2 minutes multi-cloud_backend_1_b9cbaa8bcf83
62+
f139d2e383ab opensdsio/multi-cloud-s3 "/s3" 2 minutes ago Up 2 minutes multi-cloud_s3_1_3c7878089fa4
63+
```
64+
65+
## Open OpenSDS Object lifecycle management UI
66+
67+
OpenSDS Object lifecycle management feature specific UI can be accessed via
68+
69+
`http://{your_host_ip}:8088/#/home`
70+
71+
login using the default admin credentials: `admin/opensds@123`.
72+
73+
![multi-cloud UI image ](opensds_home.PNG?raw=true)
74+
75+
### Create new backend using remote cloud bucket
76+
Click on (+) for registering a storage backend
77+
78+
![multi-cloud backend image ](opensds_backend.PNG?raw=true)
79+
80+
### Create bucket and upload the object
81+
After creating new backend follow the steps given below to create a bucket and upload an object in that bucket:
82+
1. Launch Resource from OpenSDS home page
83+
84+
2. Create a new bucket with appropriate backend
85+
86+
![multi-cloud bucket image ](opensds_bucket.PNG?raw=true)
87+
3. Click on upload button to upload an object in the selected bucket
88+
89+
![multi-cloud object image ](opensds_object.PNG?raw=true)
90+
4. Click on lifecycle tab and create a new lifecycle rule
91+
92+
![multi-cloud lifecycle image ](opensds_lifecycle.PNG?raw=true)
93+
94+
After creating the lifecycle rule user will be able to see the lifecycle configuration on bucket.
95+
96+
![multi-cloud lifecycle config image ](opensds_lifecycle_config.PNG?raw=true)
97+
98+
Save dashboard and return.
99+
### Check lifecycle in remote cloud service provider
100+
After configuring lifecycle in dashboard, a routine scheduler will run and it will wait till the date lifecycle rule comes in-effect. Once the action is performed user can log in to remote cloud ( backend cloud service provider) and check the object status.
101+
102+
### FAQ
103+
1. How to check if the lifecycle rule is applied on bucket/object ?
104+
105+
Ans:
106+
OpenSDS does not create the rule in the cloud backend (CSP). Once the bucket/object satisfies the rule, OpenSDS will automatically call cloud API and the action will be performed. If there are any errors or issues in rule, user will know at the time of lifecycle action.

‎content/guides/user-guides/object-lifecycle-management/opensds_backend.PNG‎

485 KB
Loading

‎content/guides/user-guides/object-lifecycle-management/opensds_bucket.PNG‎

180 KB
Loading

‎content/guides/user-guides/object-lifecycle-management/opensds_home.PNG‎

695 KB
Loading

‎content/guides/user-guides/object-lifecycle-management/opensds_lifecycle.PNG‎

161 KB
Loading

‎content/guides/user-guides/object-lifecycle-management/opensds_lifecycle_config.PNG‎

164 KB
Loading

‎content/guides/user-guides/object-lifecycle-management/opensds_object.PNG‎

157 KB
Loading

0 commit comments

Comments
 (0)