Save Semi-Dense point observations

[DominikVincent]

Hello,

I would like to know if it is possible to save a filtered version of the semidense-point observations or if it is possible to read them in a streaming fashion.

Currently, I have to load them in, and it takes lots of memory >128GB, which is infeasible for me. However, I only need a subset of them.

Is it possible to filter them and then save the subset as the compressed csv? Or can you load them in a streaming fashion such that the memory constraint isn't so big?

Thanks

[chpeng-fb]

Hi,

It is currently not supported with this tooling. Could you try reading this file line by line and try to filter out sections that you dont need?
something like this could work

import gzip
import csv
with gzip.open('file.csv.gz', 'rt') as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

[SeaOtocinclus]

As the file is a csv.gz you can read it with Pandas, filter it, then save it again as csv.gz and load this one with our MPS interface. It should load much faster once you have filtered it.