Skip to content

Commit 5631e21

Browse files
committed
fs: port SonicBoom module to fs module as FastUtf8Stream
As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as FastUtf8Stream. This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original.
1 parent 8b199ee commit 5631e21

File tree

6 files changed

+2104
-0
lines changed

6 files changed

+2104
-0
lines changed

doc/api/fs.md

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6909,6 +6909,145 @@ changes:
69096909
69106910
The path to the parent directory of the file this {fs.Dirent} object refers to.
69116911
6912+
### Class: `fs.FastUtf8Stream`
6913+
6914+
<!-- YAML
6915+
added: REPLACEME
6916+
-->
6917+
6918+
> Stability: 1 - Experimental
6919+
6920+
An optimized UTF-8 stream writer.
6921+
6922+
#### `new fs.FastUtf8Stream([options])`
6923+
6924+
* `options` {Object}
6925+
* `fd`: {number} A file descriptor, something that is returned by `fs.open()`
6926+
or `fs.openSync()`.
6927+
* `dest`: {string} A path to a file to be written to (mode controlled by the
6928+
append option).
6929+
* `minLength`: {number} The minimum length of the internal buffer that is
6930+
required to be full before flushing.
6931+
* `maxLength`: {number} The maximum length of the internal buffer. If a write
6932+
operation would cause the buffer to exceed `maxLength`, the data written is
6933+
dropped and a drop event is emitted with the dropped data
6934+
* `maxWrite`: {number} The maximum number of bytes that can be written;
6935+
**Default**: `16384`
6936+
* `periodicFlush`: {number} Calls flush every `periodicFlush` milliseconds.
6937+
* `sync`: {boolean} Perform writes synchronously.
6938+
* `fsync`: {boolean} Perform a `fs.fsyncSync()` every time a write is
6939+
completed.
6940+
* `append`: {boolean} Appends writes to dest file instead of truncating it.
6941+
**Default**: `true`.
6942+
* `mode`: {number|string} Specify the creating file mode (see `fs.open()`).
6943+
* `contentMode`: {string} Which type of data you can send to the write
6944+
function, supported values are `'utf8'` or `'buffer'`. **Default**:
6945+
`'utf8'`.
6946+
* `mkdir`: {boolean} Ensure directory for `dest` file exists when true.
6947+
**Default**: `false`.
6948+
* `retryEAGAIN` {Function} A function that will be called when `write()`,
6949+
`writeSync()`, or `flushSync()` encounters an `EAGAIN` or `EBUSY` error.
6950+
If the return value is `true` the operation will be retried, otherwise it
6951+
will bubble the error. The `err` is the error that caused this function to
6952+
be called, `writeBufferLen` is the length of the buffer that was written,
6953+
and `remainingBufferLen` is the length of the remaining buffer that the
6954+
stream did not try to write.
6955+
* `err` {any} An error or `null`.
6956+
* `writeBufferLen` {number}
6957+
* `remainingBufferLen`: {number}
6958+
6959+
#### `fastUtf8Stream.append`
6960+
6961+
* {boolean} Whether the stream is appending to the file or truncating it.
6962+
6963+
#### `fastUtf8Stream.contentMode`
6964+
6965+
* {string} The type of data that can be written to the stream. Supported
6966+
values are `'utf8'` or `'buffer'`. **Default**: `'utf8'`.
6967+
6968+
#### `fastUtf8Stream.destroy()`
6969+
6970+
Close the stream immediately, without flushing the internal buffer.
6971+
6972+
#### `fastUtf8Stream.end()`
6973+
6974+
Close the stream gracefully, flushing the internal buffer before closing.
6975+
6976+
#### `fastUtf8Stream.fd`
6977+
6978+
* {number} The file descriptor that is being written to.
6979+
6980+
#### `fastUtf8Stream.file`
6981+
6982+
* {string|Buffer|URL} The file that is being written to.
6983+
6984+
#### `fastUtf8Stream.flush(callback)`
6985+
6986+
* `callback` {Function}
6987+
* `err` {Error|null} An error if the flush failed, otherwise `null`.
6988+
6989+
Writes the current buffer to the file if a write was not in progress. Do
6990+
nothing if `minLength` is zero or if it is already writing.
6991+
6992+
#### `fastUtf8Stream.flushSync()`
6993+
6994+
Flushes the buffered data synchronously. This is a costly operation.
6995+
6996+
#### `fastUtf8Stream.fsync`
6997+
6998+
* {boolean} Whether the stream is performing a `fs.fsyncSync()` after every
6999+
write operation.
7000+
7001+
#### `fastUtf8Stream.maxLength`
7002+
7003+
* {number} The maximum length of the internal buffer. If a write
7004+
operation would cause the buffer to exceed `maxLength`, the data written is
7005+
dropped and a drop event is emitted with the dropped data.
7006+
7007+
#### `fastUtf8Stream.minLength`
7008+
7009+
* {number} The minimum length of the internal buffer that is required to be
7010+
full before flushing.
7011+
7012+
#### `fastUtf8Stream.mkdir`
7013+
7014+
* {boolean} Whether the stream should ensure that the directory for the
7015+
`dest` file exists. If `true`, it will create the directory if it does not
7016+
exist. **Default**: `false`.
7017+
7018+
#### `fastUtf8Stream.mode`
7019+
7020+
* {number|string} The mode of the file that is being written to.
7021+
7022+
#### `fastUtf8Stream.periodicFlush`
7023+
7024+
* {number} The number of milliseconds between flushes. If set to `0`, no
7025+
periodic flushes will be performed.
7026+
7027+
#### `fastUtf8Stream.reopen(file)`
7028+
7029+
* `file`: {string|Buffer|URL} A path to a file to be written to (mode
7030+
controlled by the append option).
7031+
7032+
Reopen the file in place, useful for log rotation.
7033+
7034+
#### `fastUtf8Stream.sync`
7035+
7036+
* {boolean} Whether the stream is writing synchronously or asynchronously.
7037+
7038+
#### `fastUtf8Stream.write(data)`
7039+
7040+
* `data` {string} The data to write.
7041+
* Returns {boolean}
7042+
7043+
#### `fastUtf8Stream.writing`
7044+
7045+
* {boolean} Whether the stream is currently writing data to the file.
7046+
7047+
#### `fastUtf8Stream[Symbol.dispose]()`
7048+
7049+
Calls `fastUtf8Stream.destroy()`.
7050+
69127051
### Class: `fs.FSWatcher`
69137052
69147053
<!-- YAML

lib/fs.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,11 @@ let ReadFileContext;
164164
// monkeypatching.
165165
let FileReadStream;
166166
let FileWriteStream;
167+
let FastUtf8Stream;
168+
169+
function lazyLoadFastUtf8Stream() {
170+
FastUtf8Stream ??= require('internal/streams/fast-utf8-stream');
171+
}
167172

168173
// Ensure that callbacks run in the global context. Only use this function
169174
// for callbacks that are passed to the binding layer, callbacks that are
@@ -3297,6 +3302,11 @@ module.exports = fs = {
32973302
FileWriteStream = val;
32983303
},
32993304

3305+
get FastUtf8Stream() {
3306+
lazyLoadFastUtf8Stream();
3307+
return FastUtf8Stream;
3308+
},
3309+
33003310
// For tests
33013311
_toUnixTimestamp: toUnixTimestamp,
33023312
};

0 commit comments

Comments
 (0)