TypeError: argument 'queries_ids': 'ndarray' object cannot be converted to 'PyArray<T, D>'

[samarthbhargav]

Hi Cosimo,

I'm still running into the error described in #5 . I've confirmed that I'm using pyseismic-lsr==0.1.3. I'm using numpy version 1.24.3 (I cannot change this due to other dependencies).

Can you let me know what I can do to fix this error?

In addition, are there any suggestions on how to pick good (default/optimized) values for query_cut and heap_factor?

[CosimoRulli]

Hello! The latest version of seismic (pyseismic-lsr==0.1.3) is compatible with the new NumPy C-API (numpy>=2.0.0),which, unfortunately, does not support backwards compatibility. If you cannot change your installed numpy version, you can try to:

1. Use the version 0.1.2 of pyseismic-lsr, which should be compatible with older versions of numpy. This may help you quickly fix your problem, but of course, it may cause problems in the futur, as you may not be able to access the features of seismic that we will introduce in the future.
2. Perform a manual installation of pyseismic-lsr. You can clone the repo, change the required versions for numpy and pyo3 in Cargo.toml to 0.19.0 and run

pip install maturin
RUSTFLAGS="-C target-cpu=native" maturin build -r
for whl in target/wheels/*.whl; do pip3 install $whl; done

from within the repo directory. Make sure to uninstall your previously installed version of pyseismic-lsr to avoid conflicts. Then you should be able to use pyseismic-lsr from python.I have not tested this solution, so the instructions may need some small tweaks.

3. Use SeismicIndexRaw. The problem you are facing is caused by string conversion in queries_id. SeismicIndexRaw does not support document and queries ids - its interface is similar to a FAISS index. You can find an example of how to use it here, in Section 5.
   First, you need to save your datasets to a jsonl file, then convert the data (both queries and documents) into seismic inner format (See Data Format here for both steps). The SeismicIndexRaw takes the path to the documents and the queries without the need to load them first. The conversion script also generates doc_ids.npy and queries_ids.npy if you need them.

[thibault-formal]

hi,
I am also facing the same issue, which happens when I try to increase MAX_TOKEN_LEN (bc for some datasets, original query ids can be quite long). It works with MAX_TOKEN_LEN=30, but breaks with MAX_TOKEN_LEN=50. Is this expected and caused by the same thing (I am using numpy==1.26.4)?

MAX_TOKEN_LEN = 30
string_type = f"U{MAX_TOKEN_LEN}"
queries_ids = np.array([q["id"] for q in queries], dtype=string_type)