|
1 | 1 | # elasticsearch-bulk-index-stream |
2 | 2 |
|
3 | | -A writable stream for bulk doing bulk actions, like indexing, in Elasticsearch. |
4 | | - |
5 | | -[](https://travis-ci.org/voldern/elasticsearch-bulk-index-stream) |
6 | | -[](https://david-dm.org/voldern/elasticsearch-bulk-index-stream) |
7 | | - |
8 | | -[](https://nodei.co/npm/elasticsearch-bulk-index-stream) |
9 | | - |
10 | | -# Usage |
11 | | - |
12 | | -## Format |
13 | | - |
14 | | -The records written to the stream has to have the following format: |
15 | | -```javascript |
16 | | -{ |
17 | | - index: 'name-of-index', |
18 | | - type: 'recordType', |
19 | | - id: 'recordId', |
20 | | - parent: 'parentRecordType', // optional |
21 | | - action: 'update', // optional (default: 'index') |
22 | | - body: { |
23 | | - name: 'Foo Bar' |
24 | | - } |
25 | | -} |
26 | | -``` |
27 | | - |
28 | | -## Buffering |
29 | | - |
30 | | -The `highWaterMark` option set on the stream defines how many items |
31 | | -will be buffered before doing a bulk operation. The stream will also |
32 | | -write all buffered items if its is closed, before emitting the |
33 | | -`finish` event. |
34 | | - |
35 | | -## Flushing |
36 | | - |
37 | | -Its also possible to send in the option `flushTimeout` to indicate |
38 | | -that the items currently in the buffer should be flushed after the |
39 | | -given amount of milliseconds if the `highWaterMark` haven't been |
40 | | -reached. |
41 | | - |
42 | | -## Logging |
43 | | - |
44 | | -A [bunyan](https://www.npmjs.com/package/bunyan), |
45 | | -[winston](https://www.npmjs.com/package/winston) or similar logger |
46 | | -instance that have methods like `debug`, `error` and `info` may be |
47 | | -sent in as `options.logger` to the constructor. |
48 | | - |
49 | | -# Example |
50 | | - |
51 | | -```javascript |
52 | | -var ElasticsearchBulkIndexStream = require('elasticsearch-bulk-index-stream'); |
53 | | - |
54 | | -var stream = new ElasticsearchBulkIndexStream(elasticsearchClient, { |
55 | | - highWaterMark: 256, |
56 | | - flushTimeout: 500 |
57 | | -}); |
58 | | - |
59 | | -someInputStream |
60 | | - .pipe(stream) |
61 | | - .on('error', function(error) { |
62 | | - // Handle error |
63 | | - }) |
64 | | - .on('finish', function() { |
65 | | - // Clean up Elasticsearch client? |
66 | | - }) |
67 | | -``` |
68 | | - |
69 | | -# API |
70 | | - |
71 | | -See [api.md](api.md). |
72 | | - |
73 | | -# See |
74 | | - |
75 | | -- [elasticsearch-streams](https://www.npmjs.com/package/elasticsearch-streams) |
76 | | - |
77 | | -Elasticsearch readable and writable streams. The main difference |
78 | | -between the bulk writer in `elasticsearch-streams` and this library is |
79 | | -that this library requires the `index` and `type` of the data being |
80 | | -written to exist in the record instead of being set in a callback when |
81 | | -the records written. |
82 | | - |
83 | | -`elasticsearch-streams` also implements its own event named `close` to |
84 | | -indicate that all the data has been written to Elasticsearch. This |
85 | | -will break modules like [pump](https://www.npmjs.com/package/pump) |
86 | | -that depend on the `finish` event. |
| 3 | +This package has been renamed to [elasticsearch-writable-stream](https://github.com/voldern/elasticsearch-writable-stream). |
87 | 4 |
|
88 | 5 | # License |
89 | 6 |
|
|
0 commit comments