poppunk_assign ERROR samples are missing from the final network

[drhchen]

poppunk v2.4.0
conda list

packages in environment at /mnt/sdb/hc/soft/microGWAS/.snakemake/conda/ad4c981ca2c33d1b316d6ed40a3521b4_:

Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
_x86_64-microarch-level 3 2_x86_64_v3 conda-forge
apscheduler 3.11.0 pyh29332c3_0 conda-forge
at-spi2-atk 2.38.0 h0630a04_3 conda-forge
at-spi2-core 2.40.3 h0630a04_0 conda-forge
atk-1.0 2.38.0 h04ea711_2 conda-forge
blinker 1.9.0 pyhff2d567_0 conda-forge
brotli 1.1.0 hb03c661_4 conda-forge
brotli-bin 1.1.0 hb03c661_4 conda-forge
brotli-python 1.1.0 py310hea6c23e_4 conda-forge
bzip2 1.0.8 hda65f42_8 conda-forge
c-ares 1.34.5 hb9d3cd8_0 conda-forge
ca-certificates 2025.10.5 hbd8a1cb_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cairo 1.16.0 h44eff21_7 defaults
cairomm-1.16 1.16.2 h7e731d7_1 conda-forge
certifi 2025.10.5 pyhd8ed1ab_0 conda-forge
cffi 2.0.0 py310h34a4b09_0 conda-forge
charset-normalizer 3.4.3 pyhd8ed1ab_0 conda-forge
click 8.3.0 pyh707e725_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_1 conda-forge
contourpy 1.3.2 py310h3788b33_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_1 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
dendropy 5.0.8 pyhdfd78af_1 bioconda
docopt 0.6.2 pyhd8ed1ab_2 conda-forge
epoxy 1.5.10 hb03c661_2 conda-forge
expat 2.7.1 hecca717_0 conda-forge
flask 3.1.2 pyhd8ed1ab_0 conda-forge
flask-apscheduler 1.13.1 pyhd8ed1ab_1 conda-forge
flask-cors 6.0.1 pyhe01879c_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_3 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.60.1 py310h3406613_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
fribidi 1.0.16 hb03c661_0 conda-forge
gdk-pixbuf 2.42.10 h5eee18b_1 defaults
gettext 0.25.1 h3f43e3d_1 conda-forge
gettext-tools 0.25.1 h3f43e3d_1 conda-forge
glib 2.84.2 h6a678d5_0 defaults
glib-tools 2.84.2 h6a678d5_0 defaults
gmp 6.3.0 hac33072_2 conda-forge
graph-tool 2.77 py310h0901348_300 conda-forge
graph-tool-base 2.77 py310h9e900f9_300 conda-forge
graphite2 1.3.14 hecca717_2 conda-forge
gtk3 3.24.38 h280cfa0_1 conda-forge
gunicorn 23.0.0 py310hff52083_2 conda-forge
h2 4.3.0 pyhcf101f3_0 conda-forge
h5py 3.13.0 nompi_py310h60e0fe6_100 conda-forge
harfbuzz 8.2.1 h3d44ed6_0 conda-forge
hdbscan 0.8.39 py310hf779ad0_1 conda-forge
hdf5 1.14.3 nompi_hdf9ad27_105 conda-forge
hicolor-icon-theme 0.17 ha770c72_2 conda-forge
hpack 4.1.0 pyhd8ed1ab_0 conda-forge
hyperframe 6.1.0 pyhd8ed1ab_0 conda-forge
icu 73.2 h59595ed_0 conda-forge
idna 3.10 pyhd8ed1ab_1 conda-forge
importlib-metadata 8.7.0 pyhe01879c_1 conda-forge
itsdangerous 2.2.0 pyhd8ed1ab_1 conda-forge
jinja2 3.1.6 pyhd8ed1ab_0 conda-forge
joblib 1.5.2 pyhd8ed1ab_0 conda-forge
jpeg 9e h0b41bf4_3 conda-forge
keyutils 1.6.3 hb9d3cd8_0 conda-forge
kiwisolver 1.4.9 py310haaf941d_1 conda-forge
krb5 1.21.3 h659f571_0 conda-forge
lcms2 2.12 hddcbb42_0 conda-forge
ld_impl_linux-64 2.44 ha97dd6f_2 conda-forge
lerc 3.0 h9c3ff4c_0 conda-forge
libaec 1.1.4 h3f801dc_0 conda-forge
libasprintf 0.25.1 h3f43e3d_1 conda-forge
libasprintf-devel 0.25.1 h3f43e3d_1 conda-forge
libblas 3.9.0 36_h4a7cf45_openblas conda-forge
libboost 1.84.0 hba137d9_3 conda-forge
libboost-python 1.84.0 py310ha2bacc8_7 conda-forge
libbrotlicommon 1.1.0 hb03c661_4 conda-forge
libbrotlidec 1.1.0 hb03c661_4 conda-forge
libbrotlienc 1.1.0 hb03c661_4 conda-forge
libcblas 3.9.0 36_h0358290_openblas conda-forge
libcups 2.3.3 h4637d8d_4 conda-forge
libcurl 8.8.0 hca28451_1 conda-forge
libdeflate 1.10 h7f98852_0 conda-forge
libdrm 2.4.125 hb03c661_1 conda-forge
libedit 3.1.20250104 pl5321h7949ede_0 conda-forge
libegl 1.7.0 ha4b6fd6_2 conda-forge
libegl-devel 1.7.0 ha4b6fd6_2 conda-forge
libev 4.33 hd590300_2 conda-forge
libexpat 2.7.1 hecca717_0 conda-forge
libffi 3.4.6 h2dba641_1 conda-forge
libgcc 15.2.0 h767d61c_7 conda-forge
libgcc-ng 15.2.0 h69a702a_7 conda-forge
libgettextpo 0.25.1 h3f43e3d_1 conda-forge
libgettextpo-devel 0.25.1 h3f43e3d_1 conda-forge
libgfortran 15.2.0 h69a702a_7 conda-forge
libgfortran-ng 15.2.0 h69a702a_7 conda-forge
libgfortran5 15.2.0 hcd61629_7 conda-forge
libgirepository 1.78.1 h003a4f0_1 conda-forge
libgl 1.7.0 ha4b6fd6_2 conda-forge
libgl-devel 1.7.0 ha4b6fd6_2 conda-forge
libglib 2.84.2 h37c7471_0 defaults
libglvnd 1.7.0 ha4b6fd6_2 conda-forge
libglx 1.7.0 ha4b6fd6_2 conda-forge
libglx-devel 1.7.0 ha4b6fd6_2 conda-forge
libgomp 15.2.0 h767d61c_7 conda-forge
libiconv 1.18 h3b78370_2 conda-forge
liblapack 3.9.0 36_h47877c9_openblas conda-forge
liblzma 5.8.1 hb9d3cd8_2 conda-forge
liblzma-devel 5.8.1 hb9d3cd8_2 conda-forge
libnghttp2 1.58.0 h47da74e_1 conda-forge
libnsl 2.0.1 hb9d3cd8_1 conda-forge
libopenblas 0.3.30 pthreads_h94d23a6_2 conda-forge
libpciaccess 0.18 hb9d3cd8_0 conda-forge
libpng 1.6.43 h2797004_0 conda-forge
librsvg 2.56.3 h98fae49_0 conda-forge
libsqlite 3.46.0 hde9e2c9_0 conda-forge
libssh2 1.11.0 h0841786_0 conda-forge
libstdcxx 15.2.0 h8f9b012_7 conda-forge
libstdcxx-ng 15.2.0 h4852527_7 conda-forge
libtiff 4.3.0 h0fcbabc_4 conda-forge
libuuid 2.41.2 he9a06e4_0 conda-forge
libwebp-base 1.6.0 hd42ef1d_0 conda-forge
libxcb 1.17.0 h8a09558_0 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxkbcommon 1.7.0 h2c5496b_1 conda-forge
libxml2 2.12.7 hc051c1a_1 conda-forge
libzlib 1.2.13 h4ab18f5_6 conda-forge
markupsafe 3.0.3 py310h3406613_0 conda-forge
matplotlib-base 3.10.1 py310h68603db_0 conda-forge
munkres 1.0.7 py_1 bioconda
ncurses 6.5 h2d0b736_3 conda-forge
networkx 3.4.2 pyh267e887_2 conda-forge
numpy 2.2.6 py310hefbff90_0 conda-forge
olefile 0.47 pyhd8ed1ab_1 conda-forge
openblas 0.3.30 pthreads_h6ec200e_2 conda-forge
openjpeg 2.5.0 h7d73246_0 conda-forge
openssl 3.5.4 h26f9b46_0 conda-forge
packaging 25.0 pyh29332c3_1 conda-forge
pandas 2.3.3 py310h0158d43_1 conda-forge
pango 1.50.14 ha41ecd1_2 conda-forge
pcre2 10.42 hcad00b1_0 conda-forge
pillow 8.4.0 py310h07f4688_0 conda-forge
pip 25.2 pyh8b19718_0 conda-forge
pixman 0.46.4 h54a6638_1 conda-forge
poppunk 2.4.0 py310h2579afa_2 bioconda
pp-sketchlib 2.1.3 py310h37665e0_0 conda-forge
pthread-stubs 0.4 hb9d3cd8_1002 conda-forge
pycairo 1.25.0 py310hda9f760_1 conda-forge
pycparser 2.22 pyh29332c3_1 conda-forge
pygobject 3.46.0 py310h30b043a_1 conda-forge
pyparsing 3.2.5 pyhcf101f3_0 conda-forge
pysocks 1.7.1 pyha55dd90_7 conda-forge
python 3.10.14 hd12c33a_0_cpython conda-forge
python-dateutil 2.9.0.post0 pyhe01879c_2 conda-forge
python-tzdata 2025.2 pyhd8ed1ab_0 conda-forge
python_abi 3.10 8_cp310 conda-forge
pytz 2025.2 pyhd8ed1ab_0 conda-forge
qhull 2020.2 h434a139_5 conda-forge
rapidnj 2.3.2 h9948957_6 bioconda
readline 8.2 h8c095d6_2 conda-forge
requests 2.32.5 pyhd8ed1ab_0 conda-forge
scikit-learn 1.7.2 py310h228f341_0 conda-forge
scipy 1.15.2 py310h1d65ade_0 conda-forge
setuptools 80.9.0 pyhff2d567_0 conda-forge
sigcpp-3.0 3.6.0 h59595ed_0 conda-forge
six 1.17.0 pyhe01879c_1 conda-forge
sparsehash 2.0.2 0 bioconda
threadpoolctl 3.6.0 pyhecae5ae_0 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
tqdm 4.67.1 pyhd8ed1ab_1 conda-forge
tzdata 2025b h78e105d_0 conda-forge
tzlocal 5.3.1 pyh8f84b5b_0 conda-forge
unicodedata2 16.0.0 py310h7c4b9e2_1 conda-forge
urllib3 2.5.0 pyhd8ed1ab_0 conda-forge
wayland 1.24.0 h3e06ad9_0 conda-forge
werkzeug 3.1.3 pyhd8ed1ab_1 conda-forge
wheel 0.45.1 pyhd8ed1ab_1 conda-forge
xkeyboard-config 2.46 hb03c661_0 conda-forge
xorg-libice 1.1.2 hb9d3cd8_0 conda-forge
xorg-libsm 1.2.6 he73a12e_0 conda-forge
xorg-libx11 1.8.12 h4f16b4b_0 conda-forge
xorg-libxau 1.0.12 hb9d3cd8_0 conda-forge
xorg-libxaw 1.0.16 hb9d3cd8_0 conda-forge
xorg-libxcomposite 0.4.6 hb9d3cd8_2 conda-forge
xorg-libxcursor 1.2.3 hb9d3cd8_0 conda-forge
xorg-libxdamage 1.1.6 hb9d3cd8_0 conda-forge
xorg-libxdmcp 1.1.5 hb9d3cd8_0 conda-forge
xorg-libxext 1.3.6 hb9d3cd8_0 conda-forge
xorg-libxfixes 6.0.2 hb03c661_0 conda-forge
xorg-libxi 1.8.2 hb9d3cd8_0 conda-forge
xorg-libxinerama 1.1.5 h5888daf_1 conda-forge
xorg-libxmu 1.2.1 hb9d3cd8_1 conda-forge
xorg-libxpm 3.5.17 hb9d3cd8_1 conda-forge
xorg-libxrandr 1.5.4 hb9d3cd8_0 conda-forge
xorg-libxrender 0.9.12 hb9d3cd8_0 conda-forge
xorg-libxt 1.3.1 hb9d3cd8_0 conda-forge
xorg-libxtst 1.2.5 hb9d3cd8_3 conda-forge
xorg-libxxf86vm 1.1.6 hb9d3cd8_0 conda-forge
xorg-xorgproto 2024.1 hb9d3cd8_1 conda-forge
xz 5.8.1 hbcc6ac9_2 conda-forge
xz-gpl-tools 5.8.1 hbcc6ac9_2 conda-forge
xz-tools 5.8.1 hb9d3cd8_2 conda-forge
zipp 3.23.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 h4ab18f5_6 conda-forge
zstandard 0.23.0 py310h7c4b9e2_3 conda-forge
zstd 1.5.6 ha6fb4c9_0 conda-forge

poppunk_assign --db data/poppunk/pa/pa_pruned_threshold_005 --query out/poppunk_input.txt --output out/poppunk --threads 80

`/mnt/sdb/hc/soft/microGWAS/.snakemake/conda/ad4c981ca2c33d1b316d6ed40a3521b4_/lib/python3.10/site-packages/PopPUNK/unwords.py:5: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
PopPUNK: assign
(with backend: sketchlib v2.1.4
sketchlib: /mnt/sdb/hc/soft/microGWAS/.snakemake/conda/ad4c981ca2c33d1b316d6ed40a3521b4_/lib/python3.10/site-packages/pp_sketchlib.cpython-310-x86_64-linux-gnu.so)

Graph-tools OpenMP parallelisation enabled: with 80 threads
Mode: Assigning clusters of query sequences

Loading previously refined model
Completed model loading
Sketching 453 genomes using 80 thread(s)
Progress (CPU): 453 / 453
Writing sketches to file
WARNING: versions of input databases sketches are different, results may not be compatible
Calculating distances using 80 thread(s)
Progress (CPU): 100.0%
Selected type isolate for distance QC is SAMD00019033
Network loaded: 6082 samples
ERROR: 10 samples are missing from the final network
`

[johnlees]

See https://poppunk-docs.bacpop.org/troubleshooting.html#samples-are-missing-from-the-final-network

[drhchen]

I used the complete Pseudomonas aeruginosa reference database from the site (https://www.bacpop.org/poppunk-databases/), but I am confused about the following statement:
“Make sure the provided network is the one created by applying the --model-dir to --distances, and that the same output directory has not been used and overwritten by different steps or inputs.”
I still don’t fully understand the specific steps. Could you please provide a more detailed explanation or example code? Thank you very much!

[drhchen]

@johnlees Could you please provide me with a more detailed tutorial？

[johnlees]

Sorry for the slow reply. Can you first please try upgrading poppunk to the most recent version (v2.7.6), v2.4.0 is over four years old

[drhchen]

@johnlees I was still getting the same error, even after the software upgraded to v2.7.6.

[johnlees]

Thanks for confirming. I can replicate this -- it's because our pseudomonas database is misformatted, sorry about that!

We will fix this on the website, but in the mean time you can get it to work if you rename the contents of the database directory pa_v1_full as follows:

pa_v1_full.dists.npy           pa_v1_full_clusters.csv        pa_v1_full_graph.gt
pa_v1_full.dists.pkl           pa_v1_full_fit.npz             pa_v1_full_refined_fit.png
pa_v1_full.h5                  pa_v1_full_fit.pkl             pa_v1_full_unword_clusters.csv

I did this by running:

rename 's/pa_pruned_threshold_005/pa_v1_full/' pa_v1_full/*
rename 's/pa_pruned_1sd/pa_v1_full/' pa_v1_full/*

[johnlees]

We've (hopefully) now fixed the databases in the downloads section