Skip to content

Commit c6bd499

Browse files
committed
fs: port SonicBoom module to fs module as FastUtf8Stream
As a first step to porting portions of the pino structured logger into the runtime, this commit ports the SonicBoom module to the fs module as FastUtf8Stream. This is a faithful port of the SonicBoom module with some modern updates, such as converting to a Class and using Symbol.dispose. The bulk of the implementation is unchanged from the original.
1 parent 049664b commit c6bd499

18 files changed

+2775
-0
lines changed

doc/api/fs.md

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6970,6 +6970,149 @@ changes:
69706970
69716971
The path to the parent directory of the file this {fs.Dirent} object refers to.
69726972
6973+
### Class: `fs.FastUtf8Stream`
6974+
6975+
<!-- YAML
6976+
added: REPLACEME
6977+
-->
6978+
6979+
> Stability: 1 - Experimental
6980+
6981+
An optimized UTF-8 stream writer that allows for flushing all the internal
6982+
buffering on demand. It handles `EAGAIN` errors correctly, allowing for
6983+
customization, for example, by dropping content if the disk is busy.
6984+
6985+
#### `new fs.FastUtf8Stream([options])`
6986+
6987+
* `options` {Object}
6988+
* `append`: {boolean} Appends writes to dest file instead of truncating it.
6989+
**Default**: `true`.
6990+
* `contentMode`: {string} Which type of data you can send to the write
6991+
function, supported values are `'utf8'` or `'buffer'`. **Default**:
6992+
`'utf8'`.
6993+
* `dest`: {string} A path to a file to be written to (mode controlled by the
6994+
append option).
6995+
* `fd`: {number} A file descriptor, something that is returned by `fs.open()`
6996+
or `fs.openSync()`.
6997+
* `fs`: {Object} An object that has the same API as the `fs` module, useful
6998+
for mocking, testing, or customizing the behavior of the stream.
6999+
* `fsync`: {boolean} Perform a `fs.fsyncSync()` every time a write is
7000+
completed.
7001+
* `maxLength`: {number} The maximum length of the internal buffer. If a write
7002+
operation would cause the buffer to exceed `maxLength`, the data written is
7003+
dropped and a drop event is emitted with the dropped data
7004+
* `maxWrite`: {number} The maximum number of bytes that can be written;
7005+
**Default**: `16384`
7006+
* `minLength`: {number} The minimum length of the internal buffer that is
7007+
required to be full before flushing.
7008+
* `mkdir`: {boolean} Ensure directory for `dest` file exists when true.
7009+
**Default**: `false`.
7010+
* `mode`: {number|string} Specify the creating file mode (see `fs.open()`).
7011+
* `periodicFlush`: {number} Calls flush every `periodicFlush` milliseconds.
7012+
* `retryEAGAIN` {Function} A function that will be called when `write()`,
7013+
`writeSync()`, or `flushSync()` encounters an `EAGAIN` or `EBUSY` error.
7014+
If the return value is `true` the operation will be retried, otherwise it
7015+
will bubble the error. The `err` is the error that caused this function to
7016+
be called, `writeBufferLen` is the length of the buffer that was written,
7017+
and `remainingBufferLen` is the length of the remaining buffer that the
7018+
stream did not try to write.
7019+
* `err` {any} An error or `null`.
7020+
* `writeBufferLen` {number}
7021+
* `remainingBufferLen`: {number}
7022+
* `sync`: {boolean} Perform writes synchronously.
7023+
7024+
#### `fastUtf8Stream.append`
7025+
7026+
* {boolean} Whether the stream is appending to the file or truncating it.
7027+
7028+
#### `fastUtf8Stream.contentMode`
7029+
7030+
* {string} The type of data that can be written to the stream. Supported
7031+
values are `'utf8'` or `'buffer'`. **Default**: `'utf8'`.
7032+
7033+
#### `fastUtf8Stream.destroy()`
7034+
7035+
Close the stream immediately, without flushing the internal buffer.
7036+
7037+
#### `fastUtf8Stream.end()`
7038+
7039+
Close the stream gracefully, flushing the internal buffer before closing.
7040+
7041+
#### `fastUtf8Stream.fd`
7042+
7043+
* {number} The file descriptor that is being written to.
7044+
7045+
#### `fastUtf8Stream.file`
7046+
7047+
* {string|Buffer|URL} The file that is being written to.
7048+
7049+
#### `fastUtf8Stream.flush(callback)`
7050+
7051+
* `callback` {Function}
7052+
* `err` {Error|null} An error if the flush failed, otherwise `null`.
7053+
7054+
Writes the current buffer to the file if a write was not in progress. Do
7055+
nothing if `minLength` is zero or if it is already writing.
7056+
7057+
#### `fastUtf8Stream.flushSync()`
7058+
7059+
Flushes the buffered data synchronously. This is a costly operation.
7060+
7061+
#### `fastUtf8Stream.fsync`
7062+
7063+
* {boolean} Whether the stream is performing a `fs.fsyncSync()` after every
7064+
write operation.
7065+
7066+
#### `fastUtf8Stream.maxLength`
7067+
7068+
* {number} The maximum length of the internal buffer. If a write
7069+
operation would cause the buffer to exceed `maxLength`, the data written is
7070+
dropped and a drop event is emitted with the dropped data.
7071+
7072+
#### `fastUtf8Stream.minLength`
7073+
7074+
* {number} The minimum length of the internal buffer that is required to be
7075+
full before flushing.
7076+
7077+
#### `fastUtf8Stream.mkdir`
7078+
7079+
* {boolean} Whether the stream should ensure that the directory for the
7080+
`dest` file exists. If `true`, it will create the directory if it does not
7081+
exist. **Default**: `false`.
7082+
7083+
#### `fastUtf8Stream.mode`
7084+
7085+
* {number|string} The mode of the file that is being written to.
7086+
7087+
#### `fastUtf8Stream.periodicFlush`
7088+
7089+
* {number} The number of milliseconds between flushes. If set to `0`, no
7090+
periodic flushes will be performed.
7091+
7092+
#### `fastUtf8Stream.reopen(file)`
7093+
7094+
* `file`: {string|Buffer|URL} A path to a file to be written to (mode
7095+
controlled by the append option).
7096+
7097+
Reopen the file in place, useful for log rotation.
7098+
7099+
#### `fastUtf8Stream.sync`
7100+
7101+
* {boolean} Whether the stream is writing synchronously or asynchronously.
7102+
7103+
#### `fastUtf8Stream.write(data)`
7104+
7105+
* `data` {string|Buffer} The data to write.
7106+
* Returns {boolean}
7107+
7108+
#### `fastUtf8Stream.writing`
7109+
7110+
* {boolean} Whether the stream is currently writing data to the file.
7111+
7112+
#### `fastUtf8Stream[Symbol.dispose]()`
7113+
7114+
Calls `fastUtf8Stream.destroy()`.
7115+
69737116
### Class: `fs.FSWatcher`
69747117
69757118
<!-- YAML

lib/fs.js

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,11 @@ let ReadFileContext;
165165
// monkeypatching.
166166
let FileReadStream;
167167
let FileWriteStream;
168+
let FastUtf8Stream;
169+
170+
function lazyLoadFastUtf8Stream() {
171+
FastUtf8Stream ??= require('internal/streams/fast-utf8-stream');
172+
}
168173

169174
// Ensure that callbacks run in the global context. Only use this function
170175
// for callbacks that are passed to the binding layer, callbacks that are
@@ -3329,6 +3334,11 @@ module.exports = fs = {
33293334
FileWriteStream = val;
33303335
},
33313336

3337+
get FastUtf8Stream() {
3338+
lazyLoadFastUtf8Stream();
3339+
return FastUtf8Stream;
3340+
},
3341+
33323342
// For tests
33333343
_toUnixTimestamp: toUnixTimestamp,
33343344
};

0 commit comments

Comments
 (0)