-
Notifications
You must be signed in to change notification settings - Fork 6
feat: Quick loading from Eigen binary files and Bump core version to 2.2.3_patches #95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
dbarrow257
wants to merge
9
commits into
develop
Choose a base branch
from
dbarrow257/feature/LoadFromEigen
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
7ab77f3
Working example of using Eigen to store and reload MC - gone from 10m…
dbarrow257 9d1cf49
Compare MD5 Checksum between saved config value and that calculated o…
dbarrow257 d5cc5a3
Cleanup MD5 code and move to Util header file
dbarrow257 7f37777
Add OpenSSL target to CMake
dbarrow257 be95270
By default, don't use Eigen binary files
dbarrow257 6f104fe
Add protection for ConvertToEigen in the case of attempting to do the…
dbarrow257 1db1767
Clean-up and add comments
dbarrow257 051f284
Bump to v2.2.3_patches core branch
dbarrow257 d43b702
Revert changes to Configs/EventRates_Beam.yaml
dbarrow257 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| #include "Samples/MaCh3DUNEFactory.h" | ||
| #include "Samples/SampleHandlerAtm.h" | ||
| #include "Fitters/MaCh3Factory.h" | ||
|
|
||
| int main(int argc, char * argv[]) { | ||
|
|
||
| MaCh3Utils::MaCh3Usage(argc, argv); | ||
| auto fitMan = MaCh3ManagerFactory(argc, argv); | ||
|
|
||
| //############################################################################################################################### | ||
| //Create SampleHandlerFD objects | ||
|
|
||
| ParameterHandlerGeneric* xsec = nullptr; | ||
| std::vector<SampleHandlerFD*> DUNEPdfs; | ||
| MakeMaCh3DuneInstance(fitMan, DUNEPdfs, xsec); | ||
|
|
||
| for (size_t iSample=0;iSample<DUNEPdfs.size();iSample++) { | ||
| SampleHandlerAtm* Sample = dynamic_cast<SampleHandlerAtm*>(DUNEPdfs[iSample]); | ||
| if (!Sample) { | ||
| MACH3LOG_ERROR("Can only convert SampleHandlerAtm CAFs to Eigen matrix binary input file"); | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
|
|
||
| std::string FileName = "AtmSample_"+Sample->GetTitle()+".eig"; | ||
| Sample->TransferToEigen(FileName); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| #pragma once | ||
|
|
||
| #include <iomanip> | ||
| #include <vector> | ||
| #include <iostream> | ||
|
|
||
| #include <Eigen/Dense> | ||
| #include <openssl/evp.h> | ||
|
|
||
| template<class Matrix> | ||
| inline void WriteEigenMatrixToFile(const std::string& filename, const Matrix& matrix){ | ||
| std::ofstream out(filename, std::ios::out | std::ios::binary | std::ios::trunc); | ||
| if(out.is_open()) { | ||
| typename Matrix::Index rows=matrix.rows(), cols=matrix.cols(); | ||
| out.write(reinterpret_cast<char*>(&rows), sizeof(typename Matrix::Index)); | ||
| out.write(reinterpret_cast<char*>(&cols), sizeof(typename Matrix::Index)); | ||
| out.write(reinterpret_cast<const char*>(matrix.data()), rows*cols*static_cast<typename Matrix::Index>(sizeof(typename Matrix::Scalar)) ); | ||
| out.close(); | ||
| } | ||
| else { | ||
| MACH3LOG_ERROR("Can not write to file:{}",filename); | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
| } | ||
|
|
||
| template<class Matrix> | ||
| inline void ReadEigenMatrixFromFile(const std::string& filename, Matrix& matrix){ | ||
| std::ifstream in(filename, std::ios::in | std::ios::binary); | ||
| if (in.is_open()) { | ||
| typename Matrix::Index rows=0, cols=0; | ||
| in.read(reinterpret_cast<char*>(&rows),sizeof(typename Matrix::Index)); | ||
| in.read(reinterpret_cast<char*>(&cols),sizeof(typename Matrix::Index)); | ||
| matrix.resize(rows, cols); | ||
| in.read(reinterpret_cast<char*>(matrix.data()), rows*cols*static_cast<typename Matrix::Index>(sizeof(typename Matrix::Scalar)) ); | ||
| in.close(); | ||
| } | ||
| else { | ||
| MACH3LOG_ERROR("Can not open binary matrix file:{}",filename); | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
| } | ||
|
|
||
| // Compute MD5 digest of buffer 'data'. Returns true on success and fills 'digest' (16 bytes). | ||
| inline void ComputeMD5(const unsigned char* data, size_t len, unsigned char digest[EVP_MAX_MD_SIZE], unsigned int &digest_len) { | ||
| EVP_MD_CTX *ctx = nullptr; | ||
|
|
||
| ctx = EVP_MD_CTX_new(); | ||
| if (!ctx) { | ||
| std::cerr << "EVP_MD_CTX_new failed\n"; | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
|
|
||
| // Use EVP_md5() — still available under OpenSSL 3.0's default provider. | ||
| if (1 != EVP_DigestInit_ex(ctx, EVP_md5(), nullptr)) { | ||
| std::cerr << "EVP_DigestInit_ex failed\n"; | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
|
|
||
| if (len > 0) { | ||
| if (1 != EVP_DigestUpdate(ctx, data, len)) { | ||
| std::cerr << "EVP_DigestUpdate failed\n"; | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
| } | ||
|
|
||
| if (1 != EVP_DigestFinal_ex(ctx, digest, &digest_len)) { | ||
| std::cerr << "EVP_DigestFinal_ex failed\n"; | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
|
|
||
| EVP_MD_CTX_free(ctx); | ||
| } | ||
|
|
||
| // Read entire file into vector<char>, returns false on error | ||
| inline bool ReadFileIntoVector(const std::string &path, std::vector<unsigned char> &out) { | ||
| std::ifstream ifs(path, std::ios::binary | std::ios::ate); | ||
| if (!ifs) return false; | ||
| std::streamsize size = ifs.tellg(); | ||
| ifs.seekg(0, std::ios::beg); | ||
| out.resize(static_cast<size_t>(size)); | ||
| if (!ifs.read(reinterpret_cast<char*>(out.data()), size)) return false; | ||
| return true; | ||
| } | ||
|
|
||
| // Convert binary digest to lowercase hex string | ||
| inline std::string ToHex(const unsigned char* data, size_t len) { | ||
| std::ostringstream oss; | ||
| oss << std::hex << std::setfill('0'); | ||
| for (size_t i = 0; i < len; ++i) { | ||
| oss << std::setw(2) << static_cast<int>(data[i]); | ||
| } | ||
| return oss.str(); | ||
| } | ||
|
|
||
| inline bool CompareMD5Sum(std::string KnownSum, std::string FilePathToCheck) { | ||
| MACH3LOG_INFO("Determining MD5 Sum for File:{}",FilePathToCheck); | ||
|
|
||
| std::vector<unsigned char> Data; | ||
| if (!ReadFileIntoVector(FilePathToCheck,Data)) { | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
|
|
||
| unsigned char Digest[EVP_MAX_MD_SIZE]; | ||
| unsigned int Digest_len = 0; | ||
| ComputeMD5(Data.data(), Data.size(), Digest, Digest_len); | ||
| std::string CalculatedMD5Sum = ToHex(Digest, Digest_len); | ||
|
|
||
| MACH3LOG_INFO("Comparing to known MD5 Sum: {}",KnownSum); | ||
| if (KnownSum != CalculatedMD5Sum) { | ||
| MACH3LOG_ERROR("MD5 Sums are different! Calculated MD5: {}",CalculatedMD5Sum); | ||
| throw MaCh3Exception(__FILE__, __LINE__); | ||
| } | ||
|
|
||
| MACH3LOG_INFO("MD5 Sums are identical!"); | ||
| return true; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ROOT has TMD5, might be easier than assuming every target has OpenSSL, though they probably do....