pnnx input npy files (libnpy-based)

[lancerstadium]

✨ pnnx Support for .npy Input Files (libnpy-based)

Related Issue: pnnx input npy files (libnpy-based) #6239
Context: Tencent Open Source Practical Project – ncnn task1
Date: 2025-09-02

---

🎯 Motivation

During graph export and verification, it is often useful to inject real input tensors into the pnnx graph:

• To enable data-driven passes such as constant folding, shape inference, or branch simplification.
• To streamline end-to-end validation without writing extra input scaffolding.
• To leverage .npy as a standard and portable format widely supported in Python/Numpy/PyTorch.

The chosen solution was to integrate npy.hpp (header-only, based on libnpy), ensuring zero link-time dependency and minimal intrusion.

---

🛠️ Key Changes

• New dependency: Added npy.hpp directly into pnnx/src.

• Graph binding API:

  int Graph::bind_operand(const std::string& path, int index);

  • Reads .npy files and attaches the tensor to a specific or auto-matched operand.
  • Data is stored in operand.params["__data__"] (keeping compatibility with existing passes).

• Command-line parsing:
  Added input=a.npy[,b.npy,...] option, reusing parse_string_list.

• Execution order:
  Binding occurs before pass_level2 so later passes can consume actual tensor data.

---

🔑 Binding Semantics

• Index ≥ 0 → Force binding to graph.operands[index].

• Index < 0 → Non-forced mode:

  1. Try to match by operand name (basename(path) vs. operand->name).
  2. If not found, bind sequentially to the next unbound input operand.

• Return codes: 0 success, -1 failure (file error, shape mismatch, etc.).

• Payload: Includes raw buffer, shape, dtype, and Fortran-order flag—kept intact for later processing.

---

⚡ Design Considerations & Challenges

1. Minimal Intrusion

   • Instead of altering core fields (dims, type), input data is side-attached as a parameter (__data__).
   • This ensures backward compatibility: passes unaware of it simply ignore, while future passes can exploit it.

2. Name vs. Index Matching

   • Models often have generated operand names (e.g., x.1) that don’t align with user file names (a.npy).
   • Solution: dual strategy → allow both strict index binding and flexible name/sequential matching.

3. Type & Layout Handling

   • .npy supports multiple dtypes (float32, int64, etc.) and memory layouts (C/Fortran).
   • Current design preserves this metadata without silent conversions, leaving the choice to later passes.

4. Error Handling

   • Multi-input scenarios may encounter missing or invalid files.
   • Errors are printed clearly but do not stop the whole process, ensuring robust workflows.

---

📌 Current Limitations & Future Directions

• Type normalization: planned support for auto-converting unsupported dtypes (e.g., float64 → float32).
• Layout conversion: optional flag to transpose Fortran-order arrays to C-order.
• Richer input syntax: e.g., input[0]=a.npy,input[name]=b.npy.
• Beyond .npy: extend parsing to .bin or .npz containers.
• Constant propagation: integrate bound tensors more tightly into constant folding and optimization passes.

---

💡 Open Questions for the Community

1. Should __data__ be standardized into a dedicated Attribute field in Operand?
2. How should Fortran-order arrays be handled by default—eagerly transposed or left for downstream?
3. Is the community open to extending the CLI syntax for explicit binding by index/name?

---

🙏 Acknowledgments

Thanks to the maintainers and community for the clear structure of pnnx, which made it possible to implement this feature with minimal and modular changes. I hope this contribution helps pave the way for more data-aware graph passes in the future. Feedback and suggestions are warmly welcome!