Duplicated nights with ignorenonwear=TRUE

[Luca-Menghini]

Hello! Thank you so much for developing and maintaining this amazing package!

I'm writing as I'm observing an issue when I use the GGIR shell function with 4-night 3-axis acceleration data.
Briefly, my data were recorded with a wrist-worn device whose memory capacity is about 40 hours, so to collect 4 nights (Mon-Thu) I need to replace it with a new device on Wed afternoon (with a gap in the recording between Wed morning and Wed afternoon, but no gaps within nights) and I recollect the second device on Fry morning. So for each subject I get 2 acc data files (for the 2 devices) named "subjID-deviceID_acc.csv".

Here is the shell function with the settings that I'm using:
GGIR(
datadir="mydata_dir", outputdir="myoutput_dir",
rmc.firstrow.acc = 2, rmc.col.acc = 2:4, rmc.col.time = 1, rmc.sf = 64,
idloc=7, # ID location in the file name: 7 = string preceding "-" because my files are named "subjID-deviceID_acc.csv"
cosinor = TRUE,
ignorenonwear = FALSE,
includenightcrit = 10, # set to 10 rather than 16 because sometimes the device memory gets full in the early morning
loglocation = "mysleeplog_location",
save_ms5raw_format = "csv",
part5_agg2_60seconds = TRUE
)

Issue:
Since we recently implemented an algorithm that detects non-wear periods directly in the acc data and removes the associated acc dataset rows before using GGIR, we changed the ignorenonwear argument from TRUE (default) to FALSE.
Following that change, I realized that for most subjects (but not all of them!) nights 3 and 4 are duplicated in the GGIR output files as if the first device recorded 4 nights rather than 2.
Specifically, for these subjects I see:

• doubled nights 3 and 4 in the part4_nightsummary_sleep_full.csv file
• n_nights_acc = 2 for each device in the part4_summary_sleep_full.csv file (which is ok), n_nights_sleeplog = 2 for the second device (still ok), but n_nights_sleeplog = 4 for the first device (not ok)
• 4 rather than 2 nights associated with the first device in the meta/ms4.out folder

Importantly, if I remove the duplicates from part4_nightsummary_sleep_full.csv files (i.e., by only keeping the first encoded row for each subject-by-night pair), I noticed that the resulting sleep onset and wake-up times show a larger difference from sleep log times than what would have been shown if I kept the second duplicate rather than the first one (i.e., as if the second night showing the same subject-by-night value was "more accurate" than the second one). I generally don't trust subjects' recall of bedtimes, but most of my subjects use clock alarms, so the self-reported wake-up times should be accurate.

I'm attaching some screenshots with examples:
GGIR_issue_duplicated-nights_ignorenonwearTRUE.pdf

To solve this issue, I've tried changing several settings (e.g., recoding the sleeplog at the device level with idloc=1, joining the two acc.csv files in a single file, adding ".csv" after each ID in the sleeplog file, etc.) until I came back to ignorenonwear=TRUE. This completely solved the issue of duplicated nights (now I get zero duplicates), BUT I noticed that the retained nights 3 and 4 are the ones showing the largest differences from sleeplog-based times in those subjects that previously presented the issue of duplicated nights, whereas the "more accurate" duplicated nights are no longer there.

Any idea of what is happening here, i.e., why should the argument ignorenonwear=FALSE result in duplicated nights?
And particularly, what is making the retained nights under the ignorenonwear=TRUE less accurate than the excluded duplicates?

Thank you in advance!
Luca