Skip to content

Commit 40466dc

Browse files
seetadevseetadev
authored
Merge pull request libp2p#980 from sumanjeet0012/feature/file_sharing
Implement Bitswap in py-libp2p
2 parents e77f786 + 50b1fe4 commit 40466dc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+7177
-34
lines changed

‎Makefile‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,10 @@ PB = libp2p/crypto/pb/crypto.proto \
6262
libp2p/relay/circuit_v2/pb/circuit.proto \
6363
libp2p/relay/circuit_v2/pb/dcutr.proto \
6464
libp2p/kad_dht/pb/kademlia.proto \
65-
libp2p/discovery/rendezvous/pb/rendezvous.proto
65+
libp2p/discovery/rendezvous/pb/rendezvous.proto \
66+
libp2p/bitswap/pb/bitswap.proto \
67+
libp2p/bitswap/pb/dag_pb.proto \
68+
libp2p/bitswap/pb/unixfs.proto
6669

6770
PY = $(PB:.proto=_pb2.py)
6871
PYI = $(PB:.proto=_pb2.pyi)

‎docs/examples.bitswap.rst‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
Bitswap File Sharing
2+
====================
3+
4+
This example demonstrates peer-to-peer file sharing using the Bitswap protocol with Merkle DAG structure.
5+
6+
What is Bitswap?
7+
----------------
8+
9+
Bitswap is a data exchange protocol that enables peers to request and share content-addressed blocks.
10+
Files are split into blocks, organized in a Merkle DAG (Directed Acyclic Graph), and transferred efficiently between peers.
11+
12+
Quick Start
13+
-----------
14+
15+
**Start a provider to share a file:**
16+
17+
.. code-block:: bash
18+
19+
python examples/bitswap/bitswap.py --mode provider --file myfile.pdf
20+
21+
**Download the file from another machine:**
22+
23+
.. code-block:: bash
24+
25+
python examples/bitswap/bitswap.py --mode client \
26+
--provider "/ip4/192.168.1.10/tcp/8000/p2p/QmXXX..." \
27+
--cid "017012207b..."
28+
29+
Usage
30+
-----
31+
32+
Provider Mode
33+
~~~~~~~~~~~~~
34+
35+
Share a file with other peers:
36+
37+
.. code-block:: bash
38+
39+
# Share a file (auto port)
40+
python bitswap.py --mode provider --file document.pdf
41+
42+
# Share on specific port
43+
python bitswap.py --mode provider --file photo.jpg --port 8000
44+
45+
The provider will:
46+
47+
1. Create a Merkle DAG from the file
48+
2. Store all blocks in memory
49+
3. Display the root CID and connection command
50+
4. Wait for client connections
51+
52+
Client Mode
53+
~~~~~~~~~~~
54+
55+
Download a file from a provider:
56+
57+
.. code-block:: bash
58+
59+
python bitswap.py --mode client \
60+
--provider "/ip4/192.168.1.10/tcp/8000/p2p/QmProviderID..." \
61+
--cid "01701220abc..."
62+
63+
# Specify output directory
64+
python bitswap.py --mode client \
65+
--provider "<multiaddr>" \
66+
--cid "<root_cid>" \
67+
--output ~/Downloads
68+
69+
The client will:
70+
71+
1. Connect to the provider
72+
2. Request blocks by CID
73+
3. Reconstruct the file from blocks
74+
4. Save to disk with original filename
75+
76+
How It Works
77+
------------
78+
79+
The implementation uses a Merkle DAG structure for all files:
80+
81+
1. **File Chunking**: Files are split into 256KB blocks
82+
2. **DAG Creation**: Blocks are organized in a tree structure
83+
3. **Root CID**: A single identifier for the entire file
84+
4. **Block Exchange**: Client requests blocks, provider sends them
85+
5. **Reconstruction**: Client rebuilds file from blocks
86+
87+
Example Session
88+
---------------
89+
90+
Provider Output
91+
~~~~~~~~~~~~~~~
92+
93+
.. code-block:: text
94+
95+
======================================================================
96+
PROVIDER NODE STARTING
97+
======================================================================
98+
File: document.pdf
99+
Size: 2.5 MB
100+
Port: auto
101+
======================================================================
102+
Peer ID: QmSK4bN4fDCxwvSVYvxxgHex2wob6VwzpEfpw8hc2Xxbow
103+
Listening on 2 address(es):
104+
/ip4/192.168.1.101/tcp/50182/p2p/QmSK4bN4fDCxwvSVYvxxgHex2wob6VwzpEfpw8hc2Xxbow
105+
/ip4/127.0.0.1/tcp/50182/p2p/QmSK4bN4fDCxwvSVYvxxgHex2wob6VwzpEfpw8hc2Xxbow
106+
✓ Bitswap started
107+
108+
Adding file to DAG...
109+
📤 completed: 100.0% (2.5 MB/2.5 MB)
110+
111+
======================================================================
112+
FILE READY TO SHARE!
113+
======================================================================
114+
Root CID: 01701220336d0f55eac9b5536e1d5f4a5429bbc9a7343f1e1d19b7757baf76b61f4f4731
115+
116+
📋 COPY THIS COMMAND TO RUN CLIENT:
117+
======================================================================
118+
python bitswap.py --mode client --provider "..." --cid "..."
119+
======================================================================
120+
121+
Provider is running. Press Ctrl+C to stop...
122+
123+
Client Output
124+
~~~~~~~~~~~~~
125+
126+
.. code-block:: text
127+
128+
======================================================================
129+
CLIENT NODE STARTING
130+
======================================================================
131+
Provider: /ip4/192.168.1.101/tcp/50182/p2p/QmSK4b...
132+
Root CID: 01701220336d0f55eac9b5536e1d5f4a5429bbc9a7343f1e...
133+
Output dir: /tmp
134+
======================================================================
135+
Client Peer ID: QmTaLxNyPszMamvE7X8oYaso1eFceB8Dqjqo3v157kfioY
136+
✓ Bitswap started
137+
138+
Connecting to provider...
139+
✓ Connected
140+
141+
Fetching file...
142+
143+
======================================================================
144+
FETCH STATISTICS:
145+
======================================================================
146+
Total blocks fetched: 12
147+
✓ 1. 01701220... (256.0 KB)
148+
✓ 2. 01551220... (256.0 KB)
149+
...
150+
151+
======================================================================
152+
FILE DOWNLOADED!
153+
======================================================================
154+
Size: 2.5 MB
155+
Filename: document.pdf (from metadata)
156+
✓ Saved to: /tmp/document.pdf
157+
======================================================================
158+
159+
Features
160+
--------
161+
162+
* **Content Addressing**: Files identified by cryptographic hash (CID)
163+
* **Merkle DAG Structure**: Efficient handling of files of any size
164+
* **Block-level Transfer**: Parallel block fetching for speed
165+
* **File Metadata**: Original filename preserved in DAG
166+
* **Resume Support**: Can request specific missing blocks
167+
* **Integrity Verification**: All blocks verified by CID
168+
169+
See Also
170+
--------
171+
172+
* :doc:`libp2p.bitswap` - Bitswap API documentation
173+
* `Bitswap Protocol Specification <https://specs.ipfs.tech/bitswap-protocol/>`_
174+
* :doc:`examples` - Other py-libp2p examples

‎docs/examples.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ Examples
1212
examples.echo_quic
1313
examples.ping
1414
examples.pubsub
15+
examples.bitswap
1516
examples.circuit_relay
1617
examples.kademlia
1718
examples.mDNS

‎docs/libp2p.bitswap.pb.rst‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
libp2p.bitswap.pb package
2+
=========================
3+
4+
Submodules
5+
----------
6+
7+
libp2p.bitswap.pb.bitswap\_pb2 module
8+
-------------------------------------
9+
10+
.. automodule:: libp2p.bitswap.pb.bitswap_pb2
11+
:members:
12+
:show-inheritance:
13+
:undoc-members:
14+
15+
libp2p.bitswap.pb.dag\_pb\_pb2 module
16+
-------------------------------------
17+
18+
.. automodule:: libp2p.bitswap.pb.dag_pb_pb2
19+
:members:
20+
:show-inheritance:
21+
:undoc-members:
22+
23+
libp2p.bitswap.pb.unixfs\_pb2 module
24+
------------------------------------
25+
26+
.. automodule:: libp2p.bitswap.pb.unixfs_pb2
27+
:members:
28+
:show-inheritance:
29+
:undoc-members:
30+
31+
Module contents
32+
---------------
33+
34+
.. automodule:: libp2p.bitswap.pb
35+
:members:
36+
:show-inheritance:
37+
:undoc-members:

0 commit comments

Comments
 (0)