Hosting Pipeline Test Data

[jscaber]

We used to host pipeline test data at: http://www.cgat.org/downloads/public/cgat/pipeline_ci_data (approximately 10GB).
If we wanted to make sure that these pipelines run safely across the various platforms and setups we are using there should be a way to have access to the same test data.
I believe @Acribbs you have this set up locally?
I found the test data on the old cluster and have copied it to the new one and will try to run it - any ideas on an easy way to host this so we are singing off the same hymn sheet, let me know.

[Acribbs]

Hi Jakub,

Im not sure this is really possible at the moment as it would require Jenkins running on CCB and I think it may be complicated (in terms of politics) to get it working. I did look at trying to automate this using GitHub actions but it was a while ago and I can't remember about memory/exectuion time limits, I can take a look again when I have time.

[jscaber]

Thanks,

I wasn't thinking of hosting the tests online, but still offline, but if I update the test files and checksums (they are all terribly out of date), we should have a way to host them to do the pipeline testing locally.