-
-
Notifications
You must be signed in to change notification settings - Fork 42
[GoogleSheet] Add a new Google Sheet loader #1609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 1.x
Are you sure you want to change the base?
Conversation
Flow PHP - BenchmarksResults of the benchmarks from this PR are compared with the results from 1.x branch. Extractors+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark | subject | revs | its | mem_peak | mode | rstdev |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench | bench_extract_10k | 1 | 3 | 4.727mb +0.68% | 598.462ms -1.30% | ±0.18% -67.83% |
| JsonExtractorBench | bench_extract_10k | 1 | 3 | 4.971mb +0.64% | 1.262s -0.61% | ±0.48% +70.51% |
| ParquetExtractorBench | bench_extract_10k | 1 | 3 | 86.273mb +0.04% | 911.420ms +0.74% | ±0.97% -43.89% |
| TextExtractorBench | bench_extract_10k | 1 | 3 | 4.451mb +0.72% | 38.497ms +0.35% | ±0.99% +629.83% |
| XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.442mb +0.72% | 605.072ms -0.38% | ±0.45% -19.55% |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
Transformers+---------------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| benchmark | subject | revs | its | mem_peak | mode | rstdev |
+---------------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 123.187mb +0.03% | 65.568ms +0.48% | ±0.95% +267.62% |
| RenameEachEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 18.449mb +0.17% | 70.917ms -3.07% | ±0.32% -59.74% |
+---------------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
Loaders+--------------------+----------------+------+-----+------------------+-----------------+----------------+
| benchmark | subject | revs | its | mem_peak | mode | rstdev |
+--------------------+----------------+------+-----+------------------+-----------------+----------------+
| CSVLoaderBench | bench_load_10k | 1 | 3 | 62.386mb +0.05% | 88.141ms +3.21% | ±0.85% +50.02% |
| JsonLoaderBench | bench_load_10k | 1 | 3 | 79.658mb +0.04% | 94.094ms -3.00% | ±0.78% -11.06% |
| ParquetLoaderBench | bench_load_10k | 1 | 3 | 165.338mb +0.02% | 20.912s +2.24% | ±0.35% +43.92% |
| TextLoaderBench | bench_load_10k | 1 | 3 | 17.757mb +0.18% | 29.977ms -1.81% | ±0.44% -48.23% |
+--------------------+----------------+------+-----+------------------+-----------------+----------------+
Building Blocks+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark | subject | revs | its | mem_peak | mode | rstdev |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| EntryFactoryBench | bench_entry_factory | 1 | 3 | 101.736mb +0.03% | 650.989ms +0.75% | ±0.59% -47.17% |
| EntryFactoryBench | bench_entry_factory | 1 | 3 | 53.087mb +0.06% | 321.783ms -0.28% | ±0.39% -33.35% |
| EntryFactoryBench | bench_entry_factory | 1 | 3 | 14.337mb +0.22% | 68.674ms +0.08% | ±0.49% -47.73% |
| RowsBench | bench_chunk_10_on_10k | 2 | 3 | 93.341mb +0.03% | 3.248ms +3.21% | ±2.57% +606.53% |
| RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 110.710mb +0.03% | 234.885ms -2.54% | ±0.74% +78.95% |
| RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 93.430mb +0.03% | 23.303ms -1.03% | ±0.59% +35.44% |
| RowsBench | bench_drop_1k_on_10k | 2 | 3 | 94.216mb +0.03% | 1.267ms -3.49% | ±1.87% +91.04% |
| RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 94.216mb +0.03% | 1.328ms +5.10% | ±2.64% -3.40% |
| RowsBench | bench_entries_on_10k | 2 | 3 | 92.376mb +0.03% | 3.219ms -2.02% | ±1.22% -28.13% |
| RowsBench | bench_filter_on_10k | 2 | 3 | 92.905mb +0.03% | 15.026ms -3.17% | ±0.66% -80.15% |
| RowsBench | bench_find_on_10k | 2 | 3 | 92.905mb +0.03% | 15.303ms -0.87% | ±0.87% -41.80% |
| RowsBench | bench_find_one_on_10k | 10 | 3 | 91.594mb +0.03% | 1.906μs +5.54% | ±2.44% -5.17% |
| RowsBench | bench_first_on_10k | 10 | 3 | 91.594mb +0.03% | 0.400μs 0.00% | ±0.00% 0.00% |
| RowsBench | bench_flat_map_on_1k | 2 | 3 | 100.655mb +0.03% | 13.976ms -0.85% | ±0.19% -45.43% |
| RowsBench | bench_map_on_10k | 2 | 3 | 130.082mb +0.02% | 64.172ms -2.96% | ±0.71% -59.91% |
| RowsBench | bench_merge_1k_on_10k | 2 | 3 | 93.425mb +0.03% | 1.139ms -0.54% | ±2.54% +40.35% |
| RowsBench | bench_partition_by_on_10k | 2 | 3 | 96.793mb +0.03% | 61.028ms +0.19% | ±2.11% +969.42% |
| RowsBench | bench_remove_on_10k | 2 | 3 | 94.478mb +0.03% | 3.385ms +1.21% | ±2.19% +82.22% |
| RowsBench | bench_sort_asc_on_1k | 2 | 3 | 91.955mb +0.03% | 39.503ms +0.12% | ±3.34% +175.11% |
| RowsBench | bench_sort_by_on_1k | 2 | 3 | 91.956mb +0.03% | 39.710ms -0.24% | ±1.02% +77.73% |
| RowsBench | bench_sort_desc_on_1k | 2 | 3 | 91.955mb +0.03% | 39.414ms -2.38% | ±1.44% +86.99% |
| RowsBench | bench_sort_entries_on_1k | 2 | 3 | 94.037mb +0.03% | 7.927ms -1.72% | ±2.03% +296.83% |
| RowsBench | bench_sort_on_1k | 2 | 3 | 91.787mb +0.03% | 29.075ms -1.20% | ±0.79% -42.15% |
| RowsBench | bench_take_1k_on_10k | 10 | 3 | 91.594mb +0.03% | 13.600μs -6.62% | ±1.20% -58.83% |
| RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 91.594mb +0.03% | 15.400μs -2.86% | ±0.00% -100.00% |
| RowsBench | bench_unique_on_1k | 2 | 3 | 110.711mb +0.03% | 237.827ms -1.38% | ±0.48% +81.51% |
| TypeDetectorBench | bench_type_detector | 1 | 3 | 42.022mb +0.08% | 421.646ms +0.34% | ±0.35% -23.74% |
| TypeDetectorBench | bench_type_detector | 1 | 3 | 11.400mb +0.28% | 85.545ms +0.65% | ±0.19% -62.27% |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 1.x #1609 +/- ##
==========================================
- Coverage 82.26% 82.17% -0.09%
==========================================
Files 694 701 +7
Lines 18852 19056 +204
==========================================
+ Hits 15508 15659 +151
- Misses 3344 3397 +53
🚀 New features to boost your workflow:
|
...etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/RowsNormalizer/EntryNormalizer.php
Outdated
Show resolved
Hide resolved
...etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/RowsNormalizer/EntryNormalizer.php
Outdated
Show resolved
Hide resolved
631a98f
to
dd392bc
Compare
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
...dapter-google-sheet/tests/Flow/ETL/Adapter/GoogleSheet/Tests/Integration/GoogleSheetTest.php
Show resolved
Hide resolved
...dapter-google-sheet/tests/Flow/ETL/Adapter/GoogleSheet/Tests/Integration/GoogleSheetTest.php
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
One last thing we will need in this PR is an example in examples, to your point that we are missing a lot of docs/examples 😅 |
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Show resolved
Hide resolved
...etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/RowsNormalizer/EntryNormalizer.php
Outdated
Show resolved
Hide resolved
src/adapter/etl-adapter-google-sheet/src/Flow/ETL/Adapter/GoogleSheet/GoogleSheetLoader.php
Outdated
Show resolved
Hide resolved
45feb4b
to
ae5ad92
Compare
Change Log
Added
Fixed
Changed
Removed
Deprecated
Security
Description