ValueError with group model visualization w/ segmentation maps

[kgandhi29]

I am using AstroPhot to obtain a cutout of an image and use it's segmentation map to fit the different objects in the image. However, as I am trying to do this and initialize the group model, I get the following error when I try to increase the cutout size past (600,600):

ValueError: zero-size array to reduction operation minimum which has no identity

Also, when it can handle that cutout size, this error pops up when I try to fit the model:

InvalidParameter: q has lower limit 0.0

Thank you in advance.

[ConnorStoneAstro]

Hi @kgandhi29 thanks for bringing this to my attention, this does sound like strange behaviour! Could you send me the files you are using that produce the error? Or if your data is proprietary, could you send a mock up fake image that also produces the same error? I can play around with the files and see exactly where the error is happening then to track it down!

[kgandhi29]

Hi Connor, Thank you for your response. I wasn't able to attach the fits file on GitHub, but I am able to do so here. CFIS.000.237.r.fits <https://drive.google.com/file/d/1Vw627c6EqjkYx9xeOvlZpoJfokW4-RF0/view?usp=drive_web> I am also using it's corresponding segmentation map to create the windows. I am trying to work through getting this code to work for a cutout, and then the entire image. Thank you in advance! Khushi Gandhi

…

On Tue, Feb 11, 2025 at 8:56 PM Connor Stone, PhD ***@***.***> wrote: Hi @kgandhi29 <https://github.com/kgandhi29> thanks for bringing this to my attention, this does sound like strange behaviour! Could you send me the files you are using that produce the error? Or if your data is proprietary, could you send a mock up fake image that also produces the same error? I can play around with the files and see exactly where the error is happening then to track it down! — Reply to this email directly, view it on GitHub <#243 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFJUPNM5ETHS34XJNBHXICL2PKS3PAVCNFSM6AAAAABW57QNGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJSGQ3DOMZRGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Regards, Khushi Gandhi

[ConnorStoneAstro]

Apologies @kgandhi29 , this looks like a quite standard (though large) FITS image; could you also post the python file you were using to load in the data? That way I can reproduce your exact error. Any other files that would be needed to run the script would be useful too (like the segmentation map).

[kgandhi29]

Hi, I just tried to send you an email with the files but the email failed as the file sizes were too large. I'm not sure of another way to send you the files. When I try to attach the files here, it says that the .ipynb file type is not allowed.

[ConnorStoneAstro]

Hmm, I think you could make a Google drive folder and send me the share link with read permission

[kgandhi29]

Here is the google drive link! https://drive.google.com/drive/folders/1DV6QaHQA3XHlC_HabzxS-ucsI8DzbJZP?usp=sharing

[ConnorStoneAstro]

Hi @kgandhi29 I think I have identified the problem. It seems that the automatic initialization of AstroPhot is running into an issue where a very small object has no pixels that are sufficiently bright for it to fit an initial model. There are a few options to deal with this, one would be to identify the offending model and simply set the initial parameters yourself (the first one to have an issue is object 11059), you can pick pretty basic values and AstroPhot will handle the rest during fitting. Another option would be to simply not fit any models that are too small, you can do this either by changing the sourceextractor parameters, or by using the AstroPhot util filter_windows which works similarly to the ap.utils.initialize.scale_windows function except it lets you set min/max sizes for the windows.

For the other error you were getting (InvalidParameter: q has lower limit 0.0) that is very strange as it should be impossible for the parameters to wander out of their intended ranges. To start, I suggest you order the windows from biggest to smallest so the bigger models get fit first (this tends to be more stable). Let me know if that doesn't solve the issue, I'll also continue looking into how a parameter could get outside its bounds.

[kgandhi29]

Thank you @ConnorStoneAstro, I will try this out and let you know how it goes. Thank you again for your help!

[kgandhi29]

Hi @ConnorStoneAstro, I just wanted to follow up on my issue. I was able to implement a minimum size for the windows. I was also able to correct the q limit problem.

However, for the cutout sizes, I still run into the same error as earlier when setting a specific minimum size and trying to increase the cutout size. The error persists when I try to run the code on the entire tile as well. I tried to run my code on a notebook in the CANFAR Science Portal, hoping that it would work there, but it returns the same error message.

Thank you for your help so far.

[ConnorStoneAstro]

Hmmm, ok I guess there's just so many models that the auto-initialization is bound to run into a challenging cutout where it can't get the data it needs to do its calculations. I will tinker around with the initializer code to hopefully make it avoid crashing in these scenarios, it should at least give some basic parameters. I think I know a solution that will work in your case for now though, you can just initialize each model individually rather than as a group. So the part of your code where you create a model for each window would be updated like this::

for win in windows:
    seg_models.append(
        ap.models.AstroPhot_Model(
            name=f"object {win:02d}",
            window=windows[win],
            model_type="sersic galaxy model",
            target=target,
            parameters={
                "center": np.array(centers[win]) * pixelscale,
                "PA": PAs[win],
                "q": qs[win],
            },
        )
    )
    try:
        seg_models[-1].initialize()
    except:
        print("error initializing", seg_models[-1].name, "setting basic parameter values")
        seg_models[-1]["n"].value = 1.0 # exponential
        seg_models[-1]["Ie"].value = 0.0 # relatively faint
        seg_models[-1]["Re"].value = 1.0 # 1 arcsec radius

This should initialize each model individually, while the group model tries to account for interactions between models (which clearly is running into an error). This should be more stable, and I've set it up so that if it runs into a problem then it will just set some basic default values for the parameters. You can tweak those values to whatever makes more sense for your data.

I hope that helps, let me know if you are still running into initialization problems!

[kgandhi29]

Hi Connor, I implemented this code piece that you provided, and although no errors were raised, the cell just kept running. I let it run for a couple of hours before I stopped it. Not sure of any way to get around this.

[ConnorStoneAstro]

Hmmm, the automated model initialization is not super fast, if you are initializing a lot of models then this will be very slow. You could add:

print(f"Initialized: {seg_models[-1].name}")

to the end of the for loop from before to see the progress for yourself. It takes a few seconds for each model so if you have thousands of models then it can take a very long time. Perhaps you should focus you modelling effort around a particular object of interest? Another option is to use a tool like SourceExtractor which can fairly quickly get estimates of many parameters, including the ones needed for a sersic model. Then you can skip the initialization step and just have AstroPhot fine tune the parameters.