Current data import is inconvenient - Rdata objects are needed

[cla023]

Currently almost each scripts is linked to an input folder where the raw/processed data must be copy pasted in before we can run the script. This leads to multiple copies of the raw/processed data, and possibly multiple versions which are hard to keep track of. It's also a very inconvenient and time consuming procedure.

A better solution is to create script(s) which reads the raw/processed data, does necessary reformatting and data cleanup, and finally stores .Rdata objects which all the other labelling scripts can import.

[cla023]

The folder and script structure needs to be cleaned up at the same time as the Rdata objects are introduced.

[cla023]

There is now a script that stores the processed data as rds objects. The plots and qc summary scripts imports data from the rds files.

What is missing:

• create rds objects from raw data
• read rds in merge script

Also add filter options (e.g. remove flush, remove first 3 months etc.). Each filter can be a function, and run the filter if its set to true in a config file.