"AssertionError: Original and re-transformed parameters must be close to each other." During multi-round inference

[danielmk]

Hey everyone,

I got the following error trying to do multi-round inference. I assume it's a problem on my end (too many simulator parameters / not enough samples?), because the linear Gaussian example (https://sbi-dev.github.io/sbi/tutorial/03_multiround_inference/) works. However, I did not find documentation on this error so I'd appreciate any pointers as to what might be going wrong. Thank you for your support.

Best,
Daniel

Running 5000 simulations in 5000 batches.: 100%|██████████| 5000/5000 [2:31:14<00:00,  1.81s/it]  
WARNING:root:Found 2564 NaN simulations and 0 Inf simulations. They will be excluded from training.
 Neural network successfully converged after 344 epochs.Traceback (most recent call last):

  File ~\anaconda3\envs\sbi\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File c:\users\daniel\repos\sbiemd\sequential_npe.py:51
    posterior = inference.build_posterior(density_estimator)

  File ~\anaconda3\envs\sbi\lib\site-packages\sbi\inference\snpe\snpe_base.py:496 in build_posterior
    potential_fn, theta_transform = posterior_estimator_based_potential(

  File ~\anaconda3\envs\sbi\lib\site-packages\sbi\inference\potentials\posterior_based_potential.py:50 in posterior_estimator_based_potential
    theta_transform = mcmc_transform(

  File ~\anaconda3\envs\sbi\lib\site-packages\sbi\utils\sbiutils.py:702 in mcmc_transform
    check_transform(prior, transform)  # type: ignore

  File ~\anaconda3\envs\sbi\lib\site-packages\sbi\utils\sbiutils.py:720 in check_transform
    assert torch.allclose(

AssertionError: Original and re-transformed parameters must be close to each other.```

[janfb]

Hi Daniel, thanks for reporting the issue!

With the assertion we check whether the parameter transform is working correctly. In your example, applying the transform and then applying the inverse transform changes the parameter, which should not happen.

The problem lies in your simulations it seems. You are using 5000 simulations, but more than half of them contains NaNs. These are excluded from training, but are you sure the other simulations are fine?

[michaeldeistler]

If you have NaNs, multi-round SNPE-C can be tricky, see Appendix Sec 6.5 "Leakage because of invalid data" here.

You can use "Truncated proposals for scalable and hassle-free simulation-based inference" from here

[danielmk]

Thank you both for the lightning fast response. I will look more into both of these references. The large number of NaNs is expected because the spiking neuronal network model is very complex and covering the relevant parameter space will cause silent networks where some summary statistics become undefined. I am training a restriction estimator separately which seems to be going well for now. Might a good restriction estimator solve the problem? I thought I would not need it for a non-amortized inference.

[michaeldeistler]

Using a restriction estimator is also fine. I would not recommend to combine the restriction estimator with multi-round inference. I.e., I would not do multiple rounds of the restriction estimator and multiple rounds of SNPE.

[danielmk]

I resolved the issue and the most likely explanation is that I made a mistake in my prior. For some of the parameters I accidentally set low=high. With the prior range being a single point, I assume this caused leakage issues that even truncation could no fix. I now have the correct prior but I also changed some other things, so I cannot be certain this was the cause. Apologies for the avoidable error.