Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
c7ad172
GPII-2966 In progress work for prefsSafe CRUD
sgithens Jul 13, 2018
1252d6b
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Jul 18, 2018
9cb199d
GPII-2966 Removing prototype safe password unlocking which will be ha…
sgithens Jul 18, 2018
897e41c
GPII-2966 Removing pouchdb prefs server config, Cindy has added an of…
sgithens Jul 19, 2018
61d906f
GPII-2966 Seperating Prefs Safes endpoints from Prefs Sets Endpoints
sgithens Jul 20, 2018
abae292
GPII-2966 Using dbstoreutils cleandoc function.
sgithens Jul 20, 2018
49adad7
GPII-2966 Removing changes from prefs post handler
sgithens Jul 20, 2018
56da240
GPII-2966 Code for creating keys and minor fixes for saving prefsSafes.
sgithens Aug 12, 2018
1ac7a4f
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Aug 16, 2018
dd187de
GPII-2966 Next sprint of accounts work
sgithens Aug 26, 2018
39160ce
GPII-3193: Fixed the issue with saving supporting tool dictionary whi…
cindyli Aug 2, 2018
6a8e564
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Aug 30, 2018
ecd08b8
GPII-2966 GPII-3193 Temporary removing the patch needed for QSS save …
sgithens Aug 31, 2018
d3c5caf
GPII-2966 Rename listKeysForPrefsSafe
sgithens Aug 31, 2018
b1c3693
GPII-2966 Removing extra white space
sgithens Aug 31, 2018
fbe91e4
GPII-2966 Cleanup based on PR Review
sgithens Sep 18, 2018
db02d36
GPII-2966 Clean up and safe creation work
sgithens Sep 19, 2018
0f07b8a
GPII-2966 Branch cleanup
sgithens Sep 19, 2018
a55a043
GPII-2966 Clean up of cloud safe promise sequences.
sgithens Sep 20, 2018
1808d3b
GPII-2966 Doco
sgithens Sep 21, 2018
e30021d
GPII-2966 Removing audit-demo patch
sgithens Sep 21, 2018
0cb4998
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Sep 21, 2018
34eccda
GPII-2966 Document types Doco
sgithens Sep 21, 2018
65e1732
GPII-2966 Minor doco formatting
sgithens Sep 21, 2018
4cf8eed
GPII-2966 Event chains for Cloudsafe operations
sgithens Oct 7, 2018
9dcab0c
GPII-2966 Adding user lookup views to testdata
sgithens Oct 9, 2018
af796ee
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Oct 9, 2018
5810582
GPII-2966 Adding prefsafe id to return payload.
sgithens Oct 24, 2018
176b3bc
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Oct 24, 2018
0fe1208
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Oct 29, 2018
ff232fb
GPII-2966 Unit tests updates
sgithens Nov 6, 2018
da79bce
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Nov 6, 2018
2f8b396
GPII-2966 Minor fix to use strict and upping gpii-pouchdb
sgithens Nov 6, 2018
203a62f
GPII-2966 Backing gpii-pouchdb back to 1.0.12 for now
sgithens Nov 7, 2018
c5846cd
Merge remote-tracking branch 'GPII' into GPII-2966
sgithens Nov 7, 2018
c21c2e1
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Nov 14, 2018
9fa93fd
GPII-2966 Minor changes from PR Review
sgithens Nov 15, 2018
ff1a8bd
GPII-2966 Updated version of gpii-express-user
sgithens Nov 15, 2018
33b28aa
GPII-2966 Adding preferencesServiceTests to all-tests
sgithens Nov 15, 2018
604b2a1
GPII-2966 Fixed termmap param name
sgithens Nov 17, 2018
74015c5
GPII-2966 Fixing error style for return payload
sgithens Nov 19, 2018
753b52f
GPII-2966 Minor doco typo
sgithens Nov 19, 2018
e0935e7
GPII-2966 More tests
sgithens Nov 21, 2018
fed8067
GPII-2966 Tests for already used login name
sgithens Nov 21, 2018
eac3195
GPII-2966 Filling in tests for invokers added to preferences service.
sgithens Nov 27, 2018
4a5aaae
GPII-2966 Changes from PR and more RESTful PrefsSafe Endpoints
sgithens Dec 14, 2018
ca60fb5
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Dec 14, 2018
a1167a5
Merge remote-tracking branch 'GPII/master' into GPII-2966
Jan 14, 2019
e6baad4
GPII-2966 Another round of PR updates. Doco, tests, minor idiom updates
sgithens Jan 21, 2019
b8c7645
GPII-2966 Removing unused findPrefsSafeByName view
sgithens Jan 21, 2019
7d3a1e9
GPII-2966 Adding tests for remaining dbstore methods.
sgithens Jan 21, 2019
ead61ff
GPII-2966 Adding tests for remaining dbstore methods.
sgithens Jan 21, 2019
e3073c0
GPII-2966 Changing findKeys to findRelatedDocs since we also return t…
sgithens Jan 21, 2019
e52e426
GPII-2966 Updates to doco from PR comments.
sgithens Jan 31, 2019
66ac6c2
GPII-2966 Added check for addGpiiKey to make sure prefsSafe exists.
sgithens Mar 12, 2019
8f3026d
GPII-2966 Removing old password prefsSafe field.
sgithens Mar 12, 2019
17d832e
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Mar 14, 2019
a1bf6aa
GPII-2966 Removing a few more occurances of `old` password field.
sgithens Mar 14, 2019
fcfcdb2
GPII-2966 Converts prefsSafes tests to couch testing.
sgithens Mar 14, 2019
82491ed
GPII-2966 Converts cloudSafeCred tests to couch testing.
sgithens Mar 14, 2019
f180848
GPII-2966 Convert leftover DbDataStore tests to couch testing.
sgithens Mar 14, 2019
34a1535
GPII-2966 Minor lint cleanup
sgithens Mar 14, 2019
00a2a26
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Mar 19, 2019
7d1becf
GPII-2966 Moving statusCode population to handler methods.
sgithens Mar 19, 2019
1dbdb28
GPII-2966 Moving error messages and codes to preferencesServerConst
sgithens Mar 20, 2019
82c1137
GPII-2966 Updating .nycrc with new source parts
sgithens Mar 20, 2019
52f15b4
GPII-2966 Adding error message and codes, linting, adding more tests …
sgithens Mar 26, 2019
0c56fe9
GPII-2966 Minimal validation for usernames and passwords.
sgithens Mar 26, 2019
c614c31
GPII-2966 Updated docs for adding credentials endpoint.
sgithens Mar 27, 2019
f2a2d15
GPII-2966 More cleanup for reject promise messages and errorCodes.
sgithens Mar 27, 2019
c98ea65
GPII-2966 Minor fix to tests to compensate for unpredictable array or…
sgithens Mar 27, 2019
2ff8f59
GPII-2966 Minor typo and comment updates based on PR.
sgithens Apr 10, 2019
d3f56f8
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Apr 10, 2019
bdac5f5
GPII-2966 Swapping out token terminology with key
sgithens Apr 10, 2019
2be8db1
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens May 21, 2019
db44149
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Jun 20, 2019
7db4a96
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Jun 27, 2019
8b2b97b
GPII-2966 Bumping dev release of gpii-express-user
sgithens Jun 27, 2019
f1c9207
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Oct 29, 2019
d0904d4
GPII-2966 Updating prefSafe and gpiiKey schema versions in test data
sgithens Dec 19, 2019
c9a9bbe
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Dec 19, 2019
b3eccba
GPII-2966 Minor updates to test data
sgithens Dec 19, 2019
e4956ff
GPII-2966 Moving existing work up to schemaVersion 0.3 before adding …
sgithens Dec 20, 2019
f4a1e68
GPII-2966 Migrations
sgithens Dec 27, 2019
1ee531f
GPII-2966 Bumping schemaVersion from 0.2 to 0.3
sgithens Dec 27, 2019
2f365c4
GPII-2966 Incrementing gpii.dbOperation.schemaVersion
sgithens Dec 27, 2019
ddd6cda
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Jan 23, 2020
2045391
GPII-2966 Eliminating separate userLookup.json views file and adding …
sgithens Jan 23, 2020
5400a69
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Mar 20, 2020
7e7fdf9
GPII-2966 Updating test payloads
sgithens Mar 20, 2020
bf6e5b3
GPII-2966 Breaking up promises into event firing transform.
sgithens Mar 30, 2020
331cfac
GPII-2966 minor typo
sgithens Mar 30, 2020
18ff970
GPII-2966 prefssafe -> prefsSafe
sgithens Mar 30, 2020
ad6d7a6
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Apr 6, 2020
2a6dbd5
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens May 26, 2020
9343166
GPII-2966 The lifecycle manager utils tests need to run in node.
sgithens May 26, 2020
04d6abe
Merge remote-tracking branch 'GPII/master' into GPII-2966
sgithens Jun 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .nycrc
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,14 @@
"!**/gpii/node_modules/ontologyHandler/src/ontologyHandler.js",
"!**/gpii/node_modules/ontologyHandler/src/ontologyHandlerUtilities.js",
"!**/gpii/node_modules/preferencesServer/index.js",
"!**/gpii/node_modules/preferencesServer/src/cloudSafeCredHandlers.js",
"!**/gpii/node_modules/preferencesServer/src/preferencesGetHandler.js",
"!**/gpii/node_modules/preferencesServer/src/preferencesPostHandler.js",
"!**/gpii/node_modules/preferencesServer/src/preferencesPutHandler.js",
"!**/gpii/node_modules/preferencesServer/src/preferencesServer.js",
"!**/gpii/node_modules/preferencesServer/src/preferencesServerConst.js",
"!**/gpii/node_modules/preferencesServer/src/preferencesService.js",
"!**/gpii/node_modules/preferencesServer/src/prefsSafesHandlers.js",
"!**/gpii/node_modules/preferencesServer/src/readyGetHandler.js",
"!**/gpii/node_modules/processReporter/index.js",
"!**/gpii/node_modules/processReporter/src/ProcessReporter.js",
Expand Down
4 changes: 2 additions & 2 deletions documentation/AuthorizationService.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Grant](https://wiki.gpii.net/w/GPII_OAuth_2_Guide#Resource_Owner_GPII_Key_Grant)
"accessToken": "gpii-app-installation-accessToken-1",
"clientCredential": {
"type": "clientCredential",
"schemaVersion": "0.2",
"schemaVersion": "0.3",
"clientId": "gpiiAppInstallationClient-1",
"oauth2ClientId": "net.gpii.ajc.bakersfield",
"oauth2ClientSecret": "client_secret_ajc_bakersfield",
Expand All @@ -37,7 +37,7 @@ Grant](https://wiki.gpii.net/w/GPII_OAuth_2_Guide#Resource_Owner_GPII_Key_Grant)
},
"authorization": {
"type": "gpiiAppInstallationAuthorization",
"schemaVersion": "0.2",
"schemaVersion": "0.3",
"clientId": "gpiiAppInstallationClient-1",
"gpiiKey": "chrome_high_contrast",
"accessToken": "gpii-app-installation-accessToken-1",
Expand Down
121 changes: 121 additions & 0 deletions documentation/DataModel.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,124 @@ GPII uses CouchDB to store data in JSON documents. The two GPII components that
storage are the Preferences Server and the Authorization Server.

The details of the GPII data model can be found [here](https://wiki.gpii.net/w/Keys,_KeyTokens,_and_Preferences).

## Preference Safes Overview

In this section we discuss the CouchDB documents for all the data associated with a users preferences safe. This
includes their preference sets, keys, full login credentials, and a minimal amount of metadata for the user,
such as name and email. Currently, this consists of three document types: `prefsSafe`, `gpiiKey`,
`gpiiCloudSafeCredential`.

### Preference Safes

Preference Safes consist of a single primary document of type `prefsSafe`. These contain some optional metadata such
as `name` and `email`, and the `preferences` section which contains the users preference sets. This is the central
document for a safe, any documents relating to a safe should have a property `prefsSafeId` which contains the id of
the preferences safe.

An example document:

```json
{
"_id": "prefsSafe-7",
"type": "prefsSafe",
"schemaVersion": "0.3",
"prefsSafeType": "user",
"name": null,
"email": null,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just came to me that, with the data model change (the removal of "password" field from "prefsSafe" document type and the addition of "timestampRevoked" into "gpiiKey" type), other relevant places in the repo should be changed accordingly. This includes the conversion script that converts preferences json files at testData/preferences to couchDB document structures as well as other data used for unit and integration tests. Doing a search using keywords like "password", "gpiiKey" will reveal more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I've removed all the old password fields from everywhere: code, scripts, and tests. I'm not familiar with the timestamp revoked portions of the keys as I don't believe this ticket/work has touched that, so I've left gpiiKey as is.

"preferences": {
"flat": {
"name": "bit of stuff",
"contexts": {
"gpii-default": {
"name": "Default preferences",
"preferences": {
"http://registry.gpii.net/common/onScreenKeyboard/enabled": true
},
"metadata": []
}
},
"metadata": []
}
},
"timestampCreated": "2017-12-14T19:55:11.641Z",
"timestampUpdated": null
}
```

### Key-in Documents

When users key-in to the GPII using a USB stick, NFC card, or other mechanism, the unique `gpiiKey` on the device will
be matched to the CouchDB `_id` on a document with type `gpiiKey`. This document contains the important fields `prefsSafeId`
and `prefsSetId` linking it to the safe, and to a specific preference set within that safe to key in with.

An example document:

```json
{
"_id": "np_tiny",
"type": "gpiiKey",
"schemaVersion": "0.3",
"prefsSafeId": "prefsSafe-7",
"prefsSetId": "gpii-default",
"revoked": false,
"revokedReason": null,
"timestampCreated": "2017-12-14T19:55:11.641Z",
"timestampUpdated": null,
"timestampRevoked": null
}
```

### Login Documents

In order to have full permissions to edit all aspects of their preferences safe, users must login to their safe using a
username and password. The current implementation of this is backed by the `gpii-express-user` library which creates
records in the same format as native CouchDB accounts and manages password hashing, unlocking, etc. In order to avoid
making changes to this external library, we introduce a document type 'gpiiCloudSafeCredential' which tracks the native
record that is created by `gpii-express-user`. Note that the `gpiiExpressuserId` entries are prefixed with `org.couch.db.user:`
which is the convention for both internal CouchDB users and users created with gpii-express-user.

An example document:

```json
{
"_id": "8f3085a7-b65b-4648-9a78-8ac7de766997",
"type": "gpiiCloudSafeCredential",
"schemaVersion": "0.3",
"prefsSafeId": "prefsSafe-7",
"gpiiExpressUserId": "org.couch.db.user:prefs7user"
}
```

For reference, the internal account record for the above looks as follows:

```json
{
"_id": "org.couch.db.user:prefs7user",
"name": "prefs7user",
"type": "user",
"email": null,
"roles": [],
"username": "prefs7user",
Copy link
Contributor

@cindyli cindyli Nov 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If names/passwords are saved in a document type from gpii-express-user, shall "name" and "password" fields defined in the "prefsSafe" document type (line 32, 33 above) be removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. The password is definately not needed, but we may want to keep Name as a piece of metadata that stays around in case username/password logins are added or removed. For now I've removed password from the docs here.

"verified": true,
"iterations": 10,
"password_scheme": "pbkdf2",
"salt": "7cf6961e6ded3bd25732e5466512d116bf9908ba9629d4ed060a03a965e5341d",
"derived_key": "e8bd265e7d82fd0f662e9ddaaf2e75acb294da1b",
"verification_code": "618fa72aa62af282704b556e34957a79"
}
```

`gpii-express-user` handles the lookup of unique names when attempting to unlock a user with a password.
This means that in the above example, the login username for preferences safe `prefsSafe-7` would be `prefs7user`.
This also means that it could be possible for a preferences safe to have more than one `gpiiCloudSafeCredential`
similar to how it can have more than one `gpiiKey`. This allows flexibility for the addition, management, and revokation
of key-in and log-in methods.

### Future Document Types

In the future we may add additional document types, or sub-types for other features such as CAS or single sign-on
authentication. For any documents that add information to a preferences safe, the most important thing is that they
have a `prefsSafeId` attribute. In general, we want to avoid having documents several linkages away from the primary
preferences safe document. In the case of `gpii-express-user` above we have 1 extra hop to avoid modifying the external
library, but in general, the document relations should be kept as simple as possible.
Loading