Hilary dev #53

hilaryh · 2024-12-11T12:20:18Z

Description

Updated subtraction_plots.py to make plots easier to read
Edited run_herg_qc to allow for less specific export_config.py protocol dict

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Testing

Testing is done automatically and codecov shows test coverage
This cannot be tested automatically

Documentation checklist

I have updated all documentation in the code where necessary.
I have checked spelling in all (new) comments and documentation.
I have added a note to RELEASE.md if relevant (new feature, breaking change, or notable bug fix).

…tely to pcpostprocess

Allow for protocols in export_config.py that are not included in the dataset

…a protocol in the experiment (should now skip those protocols)

codecov · 2025-01-08T05:05:52Z

Codecov Report

Attention: Patch coverage is 34.32836% with 88 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@f5e5052). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
pcpostprocess/subtraction_plots.py	34.32%	88 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #53   +/-   ##
=======================================
  Coverage        ?   78.79%           
=======================================
  Files           ?        7           
  Lines           ?      646           
  Branches        ?        0           
=======================================
  Hits            ?      509           
  Misses          ?      137           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…tories

MichaelClerx · 2025-06-24T14:42:24Z

I don't remember what this was, but perhaps I can generate some before and after plots to see if we like it?

joeyshuttleworth · 2025-06-24T14:49:59Z

Seems like the changes are sensible. There a few commented outlines that should probably just be outright removed. Before and after plots would be nice, but maybe we only need to look at the "after" versions!

joeyshuttleworth · 2025-06-24T14:54:31Z

pcpostprocess/scripts/run_herg_qc.py


+    data_list = os.listdir(args.data_directory)
+    export_config.D2S_QC = {x: y for x, y in export_config.D2S_QC.items() if
+                            any([x == '_'.join(z.split('_')[:-1]) for z in data_list])}


I think this should work fine. But this could throw an error if the files aren't set out properly. Maybe we should just check that there is at least one "_" in all of these files. I think it's mostly safe to assume that the data folders will be formatted correctly, but there's nothing stopping someone from dragging any other sort of file in there!

joeyshuttleworth · 2025-06-24T14:59:55Z

pcpostprocess/subtraction_plots.py

+                label=r'$I_\mathrm{L}$.' f"g={b1:1E}, E={-b0/b1:.1e}")
+        # sortedy = sorted(before_currents[i, :])
+        # ax.set_ylim(sortedy[30]*1.1, sortedy[-30]*1.1)
+


Should probably just get rid of these commented-out lines

joeyshuttleworth · 2025-06-24T15:00:11Z

pcpostprocess/subtraction_plots.py

+                label=r"$I_\mathrm{L}$." f"g={b1:1E}, E={-b0/b1:.1e}")
+        # sortedy = sorted(after_currents[i, :])
+        # ax.set_ylim(sortedy[30]*1.1, sortedy[-30]*1.1)
+        # if ax.get_legend():


More comments to remove

joeyshuttleworth · 2025-06-24T15:00:25Z

pcpostprocess/subtraction_plots.py

+            ax.set_ylabel(r'leak-corrected traces')
+            first = False
+
+        # sortedy = sorted(corrected_after_currents+corrected_before_currents)


Comments to delete

joeyshuttleworth · 2025-06-24T15:00:36Z

pcpostprocess/subtraction_plots.py

+        # ax.set_ylim(sortedy[60]*1.1, sortedy[-60]*1.1)
+        ax.legend()
        # ax.tick_params(axis='y', rotation=90)
        # ax.yaxis.set_major_formatter(mtick.FormatStrFormatter('%.1e'))


Comments to delete

joeyshuttleworth · 2025-06-24T15:02:30Z

pcpostprocess/subtraction_plots.py


+    corr_dict = {'sweeps': sweeps, 'pcs': pcs}
+    return corr_dict
+


Duplicated code from leak_correct.py?

We could remove the other file, but I don't think it's been deleted by these commits

Yeah, I don't think we should be fitting the leak again with a new linear regression (just below here), we want to show the line we fitted elsewhere (in case we tweak the process there?)

joeyshuttleworth

Looks good. Should remove some commented-out lines and either delete or change leak_correct.py so we don't duplicate code

MichaelClerx · 2025-06-24T15:48:38Z

Before:

MichaelClerx · 2025-06-24T15:48:54Z

After:

MichaelClerx · 2025-06-24T15:49:27Z

Not sure this was finished? @hilaryh how's it going? Did you find a decent ☕ ?
Are these supposed to look like this?

mirams · 2025-06-24T16:30:14Z

Not sure this was finished? @hilaryh how's it going? Did you find a decent ☕ ? Are these supposed to look like this?

well they definitely don't look nice with the legend all over the top of the interesting bits and cut off text! But it is nice to use more plot area for plots than whitespace...

mirams · 2025-06-24T16:30:40Z

a bit more sharex and sharey and just showing tick marks instead of tick labels where they are shared would probably tidy it up a fair bit

MichaelClerx · 2025-06-24T18:36:46Z

I feel like that may have been the commented out bit, @joeyshuttleworth ?

hilaryh · 2025-06-25T05:10:06Z

Not sure this was finished? @hilaryh how's it going? Did you find a decent ☕ ? Are these supposed to look like this?

Hi everyone! Decent coffee here. None hold a candle to Effy yet but I have high hopes for next Tuesday's pick. The plots are definitely not supposed to look like that! I may have forgotten to add subtraction_plots.py to my last commit... Will do now!

…dded a tight_layout line to avoid overlap

hilaryh · 2025-06-25T06:28:57Z

I've fixed the layout problems (I hope!). The commented out code can all be deleted - the sortedy parts were from a brief attempt to cut out the capacitative spikes in some of the harder to see traces without going back to the remove_capacitative_spikes code but obv something a little more robust is necessary.

MichaelClerx · 2025-06-25T08:59:11Z

Thanks @hilaryh !

mirams

Could you paste up what the subtraction plot looks like now?

mirams · 2025-07-18T09:33:46Z

pcpostprocess/subtraction_plots.py


+    corr_dict = {'sweeps': sweeps, 'pcs': pcs}
+    return corr_dict
+


Yeah, I don't think we should be fitting the leak again with a new linear regression (just below here), we want to show the line we fitted elsewhere (in case we tweak the process there?)

MichaelClerx · 2025-07-18T10:14:21Z

OK will come back to this later

mirams · 2025-10-30T16:10:59Z

pcpostprocess/subtraction_plots.py

    all_leak_params_after = []
    for i in range(len(sweeps)):
-        before_params, _ = fit_linear_leak(before_currents, voltages, times,
+        before_params, _ = fit_linear_leak(before_currents[i, :], voltages, times,


@frankiepe - Hilary may already have done some of this, just in this branch!

mirams · 2025-10-30T18:46:09Z

I'll close this as a sort of duplicate / superceded by recent changes in #86, #87, #88, #89, #90, #91

hilaryh added 10 commits December 11, 2024 12:15

Update to subtraction_plots.py layout and function to generate separa…

cdbc4ed

…tely to pcpostprocess

Update run_herg_qc.py

7b86e2e

Allow for protocols in export_config.py that are not included in the dataset

Flake8 improvements

e2e4f28

Flake8 improvements

7acc743

Flake8 improvements

4ebde11

Flake8 improvements

47e50e5

Flake8 improvements

bc4c941

Flake8 improvements

8b0b1b4

Fixed bug that resulted in an error when there weren't two copies of …

a2f11a7

…a protocol in the experiment (should now skip those protocols)

Fixed bug that resulted in an error when there weren't two copies of …

aea86c4

…a protocol in the experiment (should now skip those protocols)

hilaryh and others added 3 commits January 14, 2025 14:33

Added bash script to copy export_config.py into experiment data direc…

312aefd

…tories

Merge branch 'main' into hilary_dev

4eac863

Style fixes

fea8f50

MichaelClerx requested review from joeyshuttleworth and mirams June 24, 2025 14:40

joeyshuttleworth reviewed Jun 24, 2025

View reviewed changes

joeyshuttleworth requested changes Jun 24, 2025

View reviewed changes

Updated formatting

d52f773

Removed superfluous xticks, moved all legends outside of plot area, a…

3b10476

…dded a tight_layout line to avoid overlap

Fixed style issue

5314cb1

mirams reviewed Jul 18, 2025

View reviewed changes

MichaelClerx mentioned this pull request Jul 18, 2025

Make sure plots don't redo any analysis #61

Open

mirams reviewed Oct 30, 2025

View reviewed changes

mirams closed this Oct 30, 2025

MichaelClerx deleted the hilary_dev branch November 5, 2025 15:40

Hilary dev #53

Hilary dev #53

Uh oh!

Conversation

hilaryh commented Dec 11, 2024 • edited by MichaelClerx Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Testing

Documentation checklist

Uh oh!

codecov bot commented Jan 8, 2025

Codecov Report

Uh oh!

MichaelClerx commented Jun 24, 2025

Uh oh!

joeyshuttleworth commented Jun 24, 2025

Uh oh!

joeyshuttleworth Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

joeyshuttleworth Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

joeyshuttleworth Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

joeyshuttleworth Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

joeyshuttleworth Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

joeyshuttleworth Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

mirams Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joeyshuttleworth left a comment

Choose a reason for hiding this comment

Uh oh!

MichaelClerx commented Jun 24, 2025

Uh oh!

MichaelClerx commented Jun 24, 2025

Uh oh!

MichaelClerx commented Jun 24, 2025

Uh oh!

mirams commented Jun 24, 2025

Uh oh!

mirams commented Jun 24, 2025

Uh oh!

MichaelClerx commented Jun 24, 2025

Uh oh!

hilaryh commented Jun 25, 2025

Uh oh!

hilaryh commented Jun 25, 2025

Uh oh!

MichaelClerx commented Jun 25, 2025

Uh oh!

mirams left a comment

Choose a reason for hiding this comment

Uh oh!

mirams Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaelClerx commented Jul 18, 2025

Uh oh!

mirams Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

mirams commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

hilaryh commented Dec 11, 2024 •

edited by MichaelClerx

Loading

mirams Jul 18, 2025 •

edited

Loading

mirams Jul 18, 2025 •

edited

Loading

mirams commented Oct 30, 2025 •

edited

Loading