-
Notifications
You must be signed in to change notification settings - Fork 15
RTDEV-577 - Archive-old-artifacts-worker-sample-for-deprecated-plugin #27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
poojadonode28
wants to merge
1
commit into
jfrog:main
Choose a base branch
from
poojadonode28:RTDEV-57711-archive-old-artifacts-worker-sample-for-deprecated-plugin
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
44 changes: 44 additions & 0 deletions
44
samples/artifactory/GENERIC_EVENT/archive-old-artifact/README.md
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This Worker takes a lot of parameters. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| Artifactory Archive Old Artifacts User Worker | ||
| This worker is used to archive artifacts from a given source repository in Artifactory to a given destination repository. The artifacts are chosen based on a mixture of available parameters. | ||
|
|
||
| Note that this worker will delete your artifacts. The archive process is designed to preserve the name, path, and properties of an artifact, but save disk space by deleting the file contents. This worker is to be used for build artifacts that are no longer needed, when it's still useful to keep the build around for auditing or history purposes. | ||
|
|
||
|
|
||
| Features | ||
| Re-deploys artifacts that are to be archived, to save disk space. | ||
| Archived artifacts are moved to an archive repository, to be separate from non-archived artifacts. | ||
| Archived artifacts retain all properties that were set, and are also tagged with the archival timestamp. | ||
| Input Parameters | ||
| filePattern - the file pattern to match against in the source repository | ||
| srcRepo - the source repository to scan for artifacts to be archived | ||
| archiveRepo - the repository where matching artifacts are archived to | ||
| archiveProperty - the name of the property to use when tagging the archived artifact with the archive timestamp | ||
| Available 'time period' archive policies: | ||
| lastModified - the last time the artifact was modified | ||
| lastUpdated - the last time the artifact was updated | ||
| created - the creation date of the artifact | ||
| lastDownloaded - the last time the artifact was downloaded | ||
| age - the age of the artifact | ||
| NOTE: the time period archive policies are all specified in number of days | ||
|
|
||
| Available 'property' archive policies: | ||
| includePropertySet - the artifact will be archived if it possesses all of the passed in properties | ||
| excludePropertySet - the artifact will not be archived if it possesses all of the passed in properties | ||
| NOTE: property set format ⇒ prop[:value1[;prop2[:value2]......[;propN[:valueN]]]) | ||
|
|
||
| A property key must be provided, but a corresponding value is not necessary. If a property is set without a value, then a check is made for just the key. | ||
|
|
||
| Available artifact keep policy: | ||
| numKeepArtifacts - the number of artifacts to keep per directory | ||
| NOTE: This allows one to keep X number of artifacts (based on natural directory sort per directory). So, if your artifacts are laid out in a flat directory structure, you can keep the last X artifacts in each directory with this setting. | ||
|
|
||
| One can set any number of 'time period' archive policies as well as any number of include and exclude attribute sets. It is up to the caller to decide how best to archive artifacts. If no archive policy parameters are sent in, the plugin aborts in order to not allow default deleting of every artifact. | ||
|
|
||
| Archive Process | ||
| The 'archive' process performs the following: | ||
|
|
||
| Grabs all of the currently set properties on the artifact | ||
| Does a deploy over top of the artifact, to conserve space | ||
| Adds all of the previously held attributes to the newly deployed artifact | ||
| Moves the artifact from the source repository to the destination repository specified | ||
| Adds a property containing the archive timestamp to the artifact |
9 changes: 9 additions & 0 deletions
9
samples/artifactory/GENERIC_EVENT/archive-old-artifact/manifest.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "name": "archive-old-artifact", | ||
| "description": "This worker script is designed to archive old artifacts in Artifactory based on configurable criteria. It is useful for managing storage and maintaining repository hygiene.", | ||
| "secrets": {}, | ||
| "sourceCodePath": "./worker.ts", | ||
| "action": "GENERIC_EVENT", | ||
| "enabled": false, | ||
| "debug": true | ||
| } |
40 changes: 40 additions & 0 deletions
40
samples/artifactory/GENERIC_EVENT/archive-old-artifact/package.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| { | ||
| "name": "archive-old-artifact", | ||
| "description": "Run a script on GENERIC_EVENT", | ||
| "version": "1.0.1", | ||
| "scripts": { | ||
| "deploy": "jf worker deploy", | ||
| "undeploy": "jf worker rm \"archive-old-artifact\"", | ||
| "test": "jest" | ||
| }, | ||
| "license": "ISC", | ||
| "devDependencies": { | ||
| "jfrog-workers": "^0.4.0", | ||
| "@golevelup/ts-jest": "^0.4.0", | ||
| "@types/jest": "^29.5.12", | ||
| "jest": "^29.7.0", | ||
| "jest-jasmine2": "^29.7.0", | ||
| "ts-jest": "^29.1.2" | ||
| }, | ||
| "jest": { | ||
| "moduleFileExtensions": [ | ||
| "ts", | ||
| "js" | ||
| ], | ||
| "rootDir": ".", | ||
| "testEnvironment": "node", | ||
| "clearMocks": true, | ||
| "maxConcurrency": 1, | ||
| "testRegex": "\\.spec\\.ts$", | ||
| "moduleDirectories": ["node_modules"], | ||
| "collectCoverageFrom": [ | ||
| "**/*.ts" | ||
| ], | ||
| "coverageDirectory": "../coverage", | ||
| "transform": { | ||
| "^.+\\.(t|j)s$": "ts-jest" | ||
| }, | ||
| "testRunner": "jest-jasmine2", | ||
| "verbose": true | ||
| } | ||
| } |
14 changes: 14 additions & 0 deletions
14
samples/artifactory/GENERIC_EVENT/archive-old-artifact/payload-example.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "filePattern": "example.txt", | ||
| "srcRepo": "libs-release-local", | ||
| "archiveRepo": "archive-repo", | ||
| "lastModifiedDays": 30, | ||
| "lastUpdatedDays": 0, | ||
| "createdDays": 1, | ||
| "lastDownloadedDays": 60, | ||
| "ageDays": 30, | ||
| "excludePropertySet": "keeper:true", | ||
| "includePropertySet": "archive:true", | ||
| "archiveProperty": "archived.timestamp", | ||
| "numKeepArtifacts": 5 | ||
| } |
15 changes: 15 additions & 0 deletions
15
samples/artifactory/GENERIC_EVENT/archive-old-artifact/tsconfig.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| { | ||
| "compilerOptions": { | ||
| "module": "commonjs", | ||
| "declaration": true, | ||
| "target": "es2017", | ||
| "skipLibCheck": true, | ||
| "forceConsistentCasingInFileNames": false, | ||
| "noFallthroughCasesInSwitch": false, | ||
| "allowJs": true | ||
| }, | ||
| "include": [ | ||
| "**/*.ts", | ||
| "node_modules/@types/**/*.d.ts" | ||
| ] | ||
| } |
37 changes: 37 additions & 0 deletions
37
samples/artifactory/GENERIC_EVENT/archive-old-artifact/types.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| export interface PropertyMap { | ||
| [key: string]: string; | ||
| } | ||
|
|
||
| export interface Checksums { | ||
| sha1: string; | ||
| md5: string; | ||
| sha256: string; | ||
| } | ||
|
|
||
| export interface Artifact { | ||
| repo: string; | ||
| path: string; | ||
| created: string; | ||
| createdBy: string; | ||
| lastModified: string; | ||
| modifiedBy: string; | ||
| lastUpdated: string; | ||
| downloadUri: string; | ||
| mimeType: string; | ||
| size: string; | ||
| checksums: Checksums; | ||
| originalChecksums: Checksums; | ||
| uri: string; | ||
| } | ||
|
|
||
| export interface ArtifactResponse { | ||
| artifact: Artifact; | ||
| } | ||
|
|
||
| export interface ArtifactStatistics { | ||
| uri: string; | ||
| downloadCount: number; | ||
| lastDownloaded: number; | ||
| remoteDownloadCount: number; | ||
| remoteLastDownloaded: number; | ||
| } |
63 changes: 63 additions & 0 deletions
63
samples/artifactory/GENERIC_EVENT/archive-old-artifact/worker.spec.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| import { PlatformContext } from 'jfrog-workers'; | ||
| import archiveOldArtifacts from './worker'; | ||
|
|
||
| describe('archiveOldArtifacts', () => { | ||
| let context: PlatformContext; | ||
|
|
||
| beforeEach(() => { | ||
| context = { | ||
| clients: { | ||
| platformHttp: { | ||
| get: jest.fn(), | ||
| post: jest.fn(), | ||
| put: jest.fn(), | ||
| }, | ||
| }, | ||
| } as unknown as PlatformContext; | ||
| }); | ||
|
|
||
|
|
||
| it('should throw an error when archiveRepo is missing', async () => { | ||
| const params = { | ||
| filePattern:"example.txt", | ||
| srcRepo: "source-repo", | ||
| archiveRepo:"", | ||
| lastModifiedDays:30, | ||
| lastUpdatedDays:80, | ||
| createdDays:102, | ||
| lastDownloadedDays:20, | ||
| ageDays:102, | ||
| excludePropertySet: "keeper:true", | ||
| includePropertySet: "archive:true", | ||
| archiveProperty: "archived.timestamp", | ||
| numKeepArtifacts: 5 | ||
| }; | ||
|
|
||
| await expect(() => archiveOldArtifacts(context, params)).rejects.toThrow( | ||
| 'Both srcRepo and archiveRepo must be defined, srcRepo: source-repo, archiveRepo: ' | ||
| ); | ||
| }); | ||
|
|
||
| it('should throw an error when any of the day is missing', async () => { | ||
| const params = { | ||
| filePattern: "example.txt", | ||
| srcRepo: "source-repo", | ||
| archiveRepo: "archive-repo", | ||
| lastModifiedDays: 0, | ||
| lastUpdatedDays: 0, | ||
| createdDays: 0, | ||
| lastDownloadedDays: 0, | ||
| ageDays: 0, | ||
| excludePropertySet: "", | ||
| includePropertySet: "", | ||
| archiveProperty: "archived.timestamp", | ||
| numKeepArtifacts: 5 | ||
| }; | ||
|
|
||
| await expect(archiveOldArtifacts(context, params)).rejects.toThrow( | ||
| 'No selection criteria specified!' | ||
| ); | ||
| }); | ||
|
|
||
|
|
||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could be worth adding some Markdown formatting to make this README more readable.
Like for instance this README file.