kubectl-syncpod

High-Speed File Transfer to and from Kubernetes PVCs

Table of Contents

• About
  • Features
  • Typical Use Cases
• Example CLI Usage
  • Upload local directory to PVC
  • Download directory from PVC to local machine
• Installation
  • Using krew
  • Homebrew installation
  • Manual Installation
    • Example installation script
  • Package-Based installation
    • Debian
    • Alpine Linux
• Execution Flow
• Comparison Table
• License

---

About

• While kubectl cp and kubectl exec can both be used to copy files, performance degrades significantly when the target size is large (e.g., ~100Gi). In such cases, execution becomes drastically slow.

• Additionally, both approaches have limitations: they either require extra tools (such as tar) to be installed in the container, or they cannot be used at all with minimal base images like distroless or scratch.

• Most importantly, these methods do not support concurrent read/write operations - a critical limitation when performance and throughput matter.

Features

• Upload local files or directories to a pod's mounted volume
• Download pod volume files back to local machine
• Safe overwrite protection
• Auto-rename existing remote directories (pgdata-new-original-YYYY-MM-DD-HHMMSS)
• Concurrent file transfer with worker pool
• Preserves directory structure
• Optional automatic chown of uploaded files inside the pod (via Kubernetes exec API)
• Fully based on SFTP + Kubernetes Exec API — no side effects on other pod processes

Typical Use Cases

You’re running PostgreSQL as a StatefulSet, and you need to restore a database from a basebackup and a WAL archive. If the volume is hostPath-based, this is relatively straightforward - you simply copy the required files onto the target node. But when using CSI-backed volumes (e.g., via a cloud provider), where the PVC is mounted as a block device, the situation becomes more complex. In such cases, conventional tools fall short.

Also - you may want to scale your StatefulSet to zero and back up the PVC contents safely and efficiently - for local testing, migration, or recovery.

Basic Scenarios:

• Download backup from PVC for verification / restore
• Sync files between PVCs and local environment
• Testing PVC mount behavior
• CI/CD pipelines to prepare volume data

---

Example CLI Usage

^ back to top ^

Upload local directory to PVC:

kubectl-syncpod upload \
  --namespace pgrwl-test \
  --pvc postgres-data \
  --mount-path=/var/lib/postgresql/data \
  --src=backups \
  --dst=pgdata-new \
  --allow-overwrite \
  --owner="999:999"

Behavior:

• If /var/lib/postgresql/data/pgdata-new exists -> it is renamed (safe overwrite)
• Contents of backups/ are uploaded into /var/lib/postgresql/data/pgdata-new/
• File ownership is set to 999:999 inside the helper pod

Download directory from PVC to local machine:

kubectl-syncpod download \
  --namespace pgrwl-test \
  --pvc postgres-data \
  --mount-path=/var/lib/postgresql/data \
  --src=pgdata-new \
  --dst=backups-copy

Behavior:

• Contents of /var/lib/postgresql/data/pgdata-new/ are downloaded
• Files are written to ./backups-copy/
• Directory structure is preserved

Installation

^ back to top ^

Using krew (coming soon)

Coming soon, PR is on review

1. Install the Krew plugin manager if you haven’t already.
2. Run the following command:

kubectl krew install syncpod

Homebrew installation

brew tap hashmap-kz/homebrew-tap
brew install kubectl-syncpod

Manual Installation

1. Download the latest binary for your platform from the Releases page.
2. Place the binary in your system's PATH (e.g., /usr/local/bin).

Example installation script for Unix-Based OS (requirements: tar, curl, jq)

(
set -euo pipefail

OS="$(uname | tr '[:upper:]' '[:lower:]')"
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"
TAG="$(curl -s https://api.github.com/repos/hashmap-kz/kubectl-syncpod/releases/latest | jq -r .tag_name)"

curl -L "https://github.com/hashmap-kz/kubectl-syncpod/releases/download/${TAG}/kubectl-syncpod_${TAG}_${OS}_${ARCH}.tar.gz" |
tar -xzf - -C /usr/local/bin && \
chmod +x /usr/local/bin/kubectl-syncpod
)

Package-Based installation (suitable in CI/CD)

Debian

sudo apt update -y && sudo apt install -y curl
curl -LO https://github.com/hashmap-kz/kubectl-syncpod/releases/latest/download/kubectl-syncpod_linux_amd64.deb
sudo dpkg -i kubectl-syncpod_linux_amd64.deb

Alpine Linux

apk update && apk add --no-cache bash curl
curl -LO https://github.com/hashmap-kz/kubectl-syncpod/releases/latest/download/kubectl-syncpod_linux_amd64.apk
apk add kubectl-syncpod_linux_amd64.apk --allow-untrusted

---

Execution Flow

^ back to top ^

kubectl-syncpod spins up a temporary helper pod that:

• Mounts your target PVC
• Runs an sshd server with an in-memory public key
• Listens on a randomized NodePort
• Accepts connections only via a secure, ephemeral SSH private key (never written to disk)

The CLI then:

• Uses an in-memory SFTP client to recursively transfer files
• Skips files that are already present and match by SHA-256
• Cleans up the helper pod and service automatically

---

Comparison Table

^ back to top ^

Feature	kubectl cp	kubectl exec	kubectl-syncpod (SFTP mode)
Uses sidecar or helper pod	❌	❌	✅
Works with PVCs	⚠️ Only if mounted in container	⚠️ Manual path required	✅ Helper pod mounts PVC
Requires tools in container (tar, sh)	✅	✅	❌ (uses sshd in helper pod)
Supports readOnlyRootFilesystem pods	❌	❌	✅
Works on distroless/scratch images	❌	❌	✅
Affects main application container	✅	✅	❌
Requires container to run as root	Often yes	Often yes	❌ or configurable via helper pod spec
Safe for production workloads	⚠️ Risky	⚠️ Risky	✅ (safe for read)
Auto-cleans after sync	❌	❌	✅
Supports concurrent transfers	❌	❌	✅ (parallel SFTP workers)
Performance on large file trees	🐢 Slow	🐢 Slow	🚀 Fast (streaming + concurrency)

---

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.