Skip to content

Commit 6356fe4

Browse files
RangerMauveikreymer
andcommitted
new ipfs infra + service worker refactor + misc fixes
IPFS Changes: * switch to using auto-js-ipfs, no longer attempt to run local node in browser / electron via js-ipfs * IPFS sharing only enabled for local node (Brave for Extension, IPFS Desktop / kubo in Electron app) * unify IPFS access via service worker api * remove extension and electron specific IPFS code * refactor service worker API, keystore + IPFS funtionality to @webrecorder/awp-sw package * UI: Show type of IPFS sharing (Brave vs Local Node vs web3.storage, etc...) * Don't enable IPFS without local node (eg. web3.storage) yet * awp-sw: use CAR serialization with sharding + parallel upload of CAR streams * awp-sw: Use custom WARC record-grouped chunking for IPFS w/o gzip to maximize content-addressed deduplication Electron App: * Update to latest Electron * Backend: intercept POST to local kubo to remove CORS restrictions * UI: ensure default collection is set when recording from top record button * UI: fix selected collection occasionally being reset when resorted * UI: ensure url updates are shown for in-page nav * UI: disable maximize on record window Downloads: - reproducible / consistent downloads, eg. downloading same collection w/o additional changes should result in exact WACZ / WARC (including same signature, which is cached) Dependencies Updates - new webrecorder/[email protected] for service worker - electron 21.3.1 - auto-js-ipfs 2.0.0 - wabac.js 2.13.10 - replaywebpage 1.7.5 bump to 0.9.0 Co-authored-by: Ilya Kreymer <[email protected]>
1 parent 8b1ff7e commit 6356fe4

File tree

22 files changed

+1834
-4871
lines changed

22 files changed

+1834
-4871
lines changed

.github/workflows/buildapp.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
- name: Install Node.js, NPM and Yarn
2222
uses: actions/setup-node@v1
2323
with:
24-
node-version: 14
24+
node-version: 16
2525

2626
- name: Cache Dirs
2727
uses: actions/cache@v2
@@ -63,7 +63,7 @@ jobs:
6363
-v ${PWD}:/project \
6464
-v ~/.cache/electron:/root/.cache/electron \
6565
-v ~/.cache/electron-builder:/root/.cache/electron-builder \
66-
electronuserland/builder:wine \
66+
electronuserland/builder:16-wine \
6767
/bin/bash -c "yarn --link-duplicates --pure-lockfile && yarn install && yarn run $YARN_BUILD_CMD --linux --x64"
6868
else
6969
#mkdir -p plugins-win;
@@ -80,7 +80,7 @@ jobs:
8080
-v ${PWD}:/project \
8181
-v ~/.cache/electron:/root/.cache/electron \
8282
-v ~/.cache/electron-builder:/root/.cache/electron-builder \
83-
electronuserland/builder:wine \
83+
electronuserland/builder:16-wine \
8484
/bin/bash -c "yarn --link-duplicates --pure-lockfile && yarn install && yarn run $YARN_BUILD_CMD --win --x64 --ia32"
8585
fi
8686
else

dist/embed/replay/sw.js

Lines changed: 95 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

dist/embed/ui.js

Lines changed: 617 additions & 350 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,48 @@
11
{
22
"name": "@webrecorder/archivewebpage",
33
"productName": "ArchiveWeb.page",
4-
"version": "0.8.3",
4+
"version": "0.9.0",
55
"main": "index.js",
66
"description": "Create Web Archives directly in your browser",
77
"repository": "https://github.com/webrecorder/archiveweb.page",
88
"author": "Webrecorder Software",
99
"license": "AGPL-3.0-or-later",
1010
"dependencies": {
1111
"@fortawesome/fontawesome-free": "^5.13.0",
12-
"@webrecorder/wabac": "^2.12.5",
12+
"@ipld/car": "^5.0.1",
13+
"@ipld/unixfs": "^2.0.0",
14+
"@webrecorder/awp-sw": "^0.1.5",
15+
"@webrecorder/wabac": "^2.13.10",
16+
"auto-js-ipfs": "^2.0.0",
1317
"browsertrix-behaviors": "^0.3.4",
1418
"btoa": "^1.2.1",
1519
"bulma": "^0.9.3",
1620
"client-zip": "^2.2.2",
1721
"hash-wasm": "^4.9.0",
1822
"http-status-codes": "^2.1.4",
19-
"ipfs-core": "^0.10.8",
20-
"ipfs-http-client": "^52.0.5",
2123
"keyword-mark-element": "^0.1.2",
24+
"multiformats": "^10.0.2",
2225
"node-fetch": "2.6.7",
26+
"p-queue": "^7.3.0",
2327
"pretty-bytes": "^5.6.0",
2428
"querystring-es3": "^0.2.1",
25-
"replaywebpage": "1.6.5",
29+
"replaywebpage": "^1.7.5",
2630
"stream-browserify": "^3.0.0",
2731
"unused-filename": "^4.0.1",
2832
"url": "^0.11.0",
2933
"uuid": "^8.3.2",
30-
"warcio": "^1.5.1"
34+
"warcio": "^1.6.1"
3135
},
3236
"devDependencies": {
3337
"copy-webpack-plugin": "^9.0.1",
3438
"css-loader": "^6.2.0",
35-
"electron": "^18.3.15",
39+
"electron": "^21.3.1",
3640
"electron-builder": "^23.0.3",
3741
"electron-notarize": "^1.0.0",
42+
"eslint": "^8.28.0",
3843
"generate-json-webpack-plugin": "^2.0.0",
3944
"mini-css-extract-plugin": "^2.3.0",
40-
"node-sass": "^7.0.1",
45+
"node-sass": "^8.0.0",
4146
"raw-loader": "^4.0.2",
4247
"sass-loader": "^12.1.0",
4348
"svg-inline-loader": "^0.8.2",
@@ -105,7 +110,7 @@
105110
"extraResources": [
106111
"plugins-mac"
107112
],
108-
"singleArchFiles": "dist/electron/prebuilds/**/*"
113+
"singleArchFiles": "{dist/electron/prebuilds,dist/electron/prebuilds/**}"
109114
},
110115
"linux": {
111116
"category": "Archiving;Utility;",

0 commit comments

Comments
 (0)