Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 28, 2025

Provides familiar NumPy interface to enable user migration while leveraging Paper's out-of-core capabilities for datasets larger than memory.

Implementation

Core API (paper/numpy_api.py)

  • ndarray class: Lazy evaluation with NumPy-like operations (+, *, @), properties (shape, dtype, T), and public to_numpy() method
  • Creation functions: array(), zeros(), ones(), eye(), random_rand()
  • I/O: load(), save()

Bug Fix (paper/plan.py)

Fixed pre-existing scalar multiplication bug where EagerNode matrices were closed prematurely, causing segfaults on reuse:

# Only close intermediate computed results, not input matrices
if not isinstance(self.left, EagerNode):
    TMP.close()

Testing

26 new tests covering array creation, operations, error handling, I/O, and large array support. All 62 tests passing.

Usage

from paper import numpy_api as pnp

# Create arrays with familiar syntax
a = pnp.array([[1, 2], [3, 4]])
b = pnp.array([[5, 6], [7, 8]])

# Build lazy computation plan
c = (a + b) * 2  # Automatic operator fusion

# Execute and convert to NumPy
result = c.compute().to_numpy()

Limitations

  • Currently supports 2D matrices only (foundational version)
  • Scalar addition not yet implemented

Documentation

  • Quick start in README
  • examples/numpy_api_example.py: 9 comprehensive examples
  • examples/simple_demo.py: minimal demo
  • NUMPY_API_SUMMARY.md: complete implementation details
Original prompt

This section details on the original issue you should resolve

<issue_title>NumPy-compatible API Layer</issue_title>
<issue_description>Rationale: This is the foundation for user adoption. Without NumPy compatibility, users won't migrate from existing solutions. It leverages your deep understanding of matrix operations from SystemDS while providing the familiar interface developers expect.​
</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Add NumPy-compatible API layer for user adoption Add NumPy-compatible API layer for out-of-core matrix operations Oct 28, 2025
Copilot AI requested a review from j143 October 28, 2025 02:40
Copy link
Owner

@j143 j143 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a first draft this is good.

@j143 j143 marked this pull request as ready for review October 28, 2025 14:48
@j143 j143 merged commit c994651 into main Oct 28, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NumPy-compatible API Layer

2 participants