Skip to content

Conversation

@rolnico
Copy link
Member

@rolnico rolnico commented Oct 23, 2025

Please check if the PR fulfills these requirements

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • A PR or issue has been opened in all impacted repositories (if any)

Does this PR already have an issue describing the problem?
No

What kind of change does this PR introduce?
Change of dependency

What is the current behavior?
PowSyBl uses univocity-parsers for handling CSV files (parsing and writing), but this lib is abandoned since 2021.

What is the new behavior (if this is a feature change)?
PowSyBl now uses FastCSV for handling CSV files.

Does this PR introduce a breaking change or deprecate an API?

  • Yes
  • No

If yes, please check if the following requirements are fulfilled

  • The Breaking Change or Deprecated label has been added
  • The migration steps are described in the following section

What changes might users need to make in their application due to this PR? (migration steps)

Other information:

PSS-E handling with be managed in a specific PR

@rolnico rolnico self-assigned this Oct 23, 2025
@rolnico rolnico added dependencies Pull requests that update a dependency file Release 12/2025 labels Oct 23, 2025
@sonarqubecloud
Copy link

@rolnico
Copy link
Member Author

rolnico commented Nov 3, 2025

I did some tests to compare the performances of this version vs main. Each tested operation was performed 50 times consecutively, and the results are based on the last 30 draws.

The operations tested where:

  • For the Anonymizer, I tested writeCsv and readCsv for 5000000 values anonymized.
  • For the Time Series, I tested the time series generation, TimeSeriesTable.writeCsv, and TimeSeries.parseCsv. The dataset consisted of 1000 double time series of 3000 time steps in 5 versions, which represents a CSV file of 290 Mo once exported.
  • For the IEEE CDF, I tested the IeeeCdfNetworkFactory.create33 and IeeeCdfNetworkFactory.create69 methods.

The results are as follows:

Operation main (ms) FastCSV (ms) Difference (ms) Difference (%)
Anonymizer - Write 311,393 181,985 -129,408 -41,56 %
Anonymizer - Read 2006,383 1971,176 -35,207 -1,75 %
Time Series - Generation 198,604 181,077 -17,527 -8,82 %
Time Series - Write 1818,053 1800,404 -17,649 -0,97 %
Time Series - Read 2614,167 2729,676 +115,509 +4,42 %
IEEE CDF - 33 7,210 0,849 -6,361 -88,22 %
IEEE CDF - 69 5,048 1,020 -4,028 -79,80 %

The results here are very encouraging: it seems that the change of dependency can optimize the performances in some cases.
However, since there is not much change in the Time series performances, it would be interesting so see if the time is spent in the library or in the code around it.

@rolnico rolnico marked this pull request as ready for review November 13, 2025 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file IEEE Release 12/2025 Time series

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants