Phase 1: Foundation - Implementation Tasks

Goal: Establish core CLI structure, configuration system, and SSH management

Milestone: User can manage configuration files, connect to hosts via SSH, and resolve secrets from multiple sources with proper instance scoping

Task Breakdown

Each task is designed to result in a working, testable state. Tests should be written alongside implementation.

1. Project Initialization ✅

Working State: Go project compiles and runs, outputs version information

Tasks:

Initialize Go module: go mod init github.com/catalystcommunity/foundry/v1
Create directory structure (v1/ version for future-proofing)
Create cmd/foundry/main.go with basic CLI scaffold using urfave/cli v3
Implement --version flag that outputs version info
Set up tools script with targets: build, test, clean, install (bash instead of Make)
.gitignore already comprehensive
LICENSE already exists
Initialize go.mod with core dependencies:
- github.com/urfave/cli/v3
- gopkg.in/yaml.v3
- github.com/stretchr/testify
- golang.org/x/crypto/ssh

Test Criteria:

go build ./cmd/foundry compiles successfully
./foundry --version outputs version
./foundry --help shows command structure
./tools test runs successfully

Files Created:

v1/cmd/foundry/main.go
v1/tools (bash script)
v1/go.mod
README.md with tools documentation

2. Configuration Data Structures ✅

Working State: Config structs defined, can parse valid YAML into structs

Tasks:

Define internal/config/types.go with core structs:

type Config struct {
    Version string         `yaml:"version"`
    Cluster ClusterConfig  `yaml:"cluster"`
    Components ComponentMap `yaml:"components"`
    Observability ObsConfig `yaml:"observability,omitempty"`
    Storage StorageConfig   `yaml:"storage,omitempty"`
}

type ClusterConfig struct {
    Name   string      `yaml:"name"`
    Domain string      `yaml:"domain"`
    Nodes  []NodeConfig `yaml:"nodes"`
}

type NodeConfig struct {
    Hostname string `yaml:"hostname"`
    Role     string `yaml:"role"` // control-plane, worker
}

type ComponentMap map[string]ComponentConfig

type ComponentConfig struct {
    Version string                 `yaml:"version,omitempty"`
    Hosts   []string               `yaml:"hosts,omitempty"`
    Config  map[string]interface{} `yaml:",inline"`
}

Add validation methods for each struct (e.g., Validate() error)
Write unit tests for validation logic
- Valid configs pass
- Missing required fields fail
- Invalid roles fail
- Invalid versions fail (if format-specific)

Test Criteria:

Can unmarshal valid YAML into Config struct
Validation catches missing required fields
Validation catches invalid enum values
Tests pass with 83.3% coverage

Files Created:

v1/internal/config/types.go
v1/internal/config/types_test.go
v1/test/fixtures/valid-config.yaml
v1/test/fixtures/invalid-config-*.yaml

3. Configuration File Loading ✅

Working State: Can load config from file path, parse YAML, validate structure

Tasks:

Implement internal/config/loader.go:

func Load(path string) (*Config, error)
func LoadFromReader(r io.Reader) (*Config, error)

Handle file reading errors gracefully
Parse YAML and unmarshal into Config
Run validation after parsing
Return detailed errors (file not found, parse error, validation error)
Write unit tests:
- Load valid config succeeds
- Load missing file returns error
- Load invalid YAML returns parse error
- Load invalid config returns validation error

Test Criteria:

Can load test/fixtures/valid-config.yaml
Loading non-existent file returns clear error
Loading invalid YAML returns parse error
Loading invalid config returns validation error
All tests pass

Files Created:

v1/internal/config/loader.go
v1/internal/config/loader_test.go

4. Configuration Path Resolution ✅

Working State: CLI can find and load config from multiple possible locations

Tasks:

Implement internal/config/paths.go:

func GetConfigDir() (string, error)  // Returns ~/.foundry/
func FindConfig(name string) (string, error)  // Finds config file
func ListConfigs() ([]string, error)  // Lists available configs

Use os.UserHomeDir() for cross-platform home directory
Check for FOUNDRY_CONFIG env var first
Then check --config flag value
Then look for default ~/.foundry/stack.yaml
If multiple configs and no explicit selection, return error
Write unit tests with temporary directories

Test Criteria:

GetConfigDir() returns correct path
FindConfig() finds existing config
FindConfig() returns error for non-existent config
ListConfigs() returns all *.yaml files in config dir
Tests use temp dirs, don't touch user's actual config

Files Created:

v1/internal/config/paths.go
v1/internal/config/paths_test.go

5. Secret Reference Parsing ✅

Working State: Can identify and parse secret references in config, with optional instance context

Tasks:

Implement internal/secrets/parser.go:

type SecretRef struct {
    Path     string  // path/to/secret
    Key      string  // key_name
    Instance string  // Optional: service instance (e.g., "myapp-prod", "myapp-stable")
    Raw      string  // original ${secret:path:key}
}

func ParseSecretRef(s string) (*SecretRef, error)
func IsSecretRef(s string) bool

Use regex to match ${secret:path/to/secret:key} pattern
Handle malformed references (missing parts, invalid syntax)
Instance is NOT part of the secret reference syntax in config
- Config contains: ${secret:database/prod:password}
- Instance context is provided at resolution time
Write unit tests:
- Valid reference parses correctly
- Invalid references return errors
- Non-secret strings return nil/false
- Edge cases (empty path, empty key, special chars)

Test Criteria:

${secret:database/prod:password} parses correctly
${secret:} returns error
plaintext returns false for IsSecretRef
All tests pass

Files Created:

v1/internal/secrets/parser.go
v1/internal/secrets/parser_test.go

Design Note: Secret references in config files don't include instance context. Instance is provided at resolution time based on what service/namespace is being deployed. This allows the same config to be used for multiple instances (e.g., myapp-prod, myapp-stable).

6. Secret Resolution Context ✅

Working State: Context type for secret resolution with instance scoping

Tasks:

Implement internal/secrets/context.go:

type ResolutionContext struct {
    Instance  string  // e.g., "myapp-prod", "foundry-core"
    Namespace string  // Optional: for namespace-scoped secrets
}

func NewResolutionContext(instance string) *ResolutionContext
func (rc *ResolutionContext) NamespacedPath(ref SecretRef) string

NamespacedPath() combines instance with secret path
- Example: instance="myapp-prod", ref.Path="database/main"
- Returns: "myapp-prod/database/main"
For foundry core components, use instance="foundry-core" or similar
Write unit tests for path construction

Test Criteria:

Context correctly combines instance + path
Handles edge cases (empty instance, etc.)
Tests pass

Files Created:

v1/internal/secrets/context.go
v1/internal/secrets/context_test.go

7. Secret Resolution - Environment Variables ✅

Working State: Can resolve secrets from environment variables with instance context

Tasks:

Implement internal/secrets/resolver.go:

type Resolver interface {
    Resolve(ctx *ResolutionContext, ref SecretRef) (string, error)
}

type EnvResolver struct{}

func (e *EnvResolver) Resolve(ctx *ResolutionContext, ref SecretRef) (string, error)

Convert secret ref to env var name using instance context:
- Pattern: FOUNDRY_SECRET_<instance>_<path>_<key>
- Example: FOUNDRY_SECRET_myapp_prod_database_main_password
Replace slashes, hyphens, and colons with underscores
Check if env var exists
Return value or error if not found
Write unit tests with os.Setenv

Test Criteria:

Can resolve secret from env var with instance context
Returns error if env var not set
Name conversion works correctly (including instance)
Tests clean up env vars after running

Files Created:

v1/internal/secrets/resolver.go
v1/internal/secrets/resolver_test.go

8. Secret Resolution - ~/.foundryvars ✅

Working State: Can resolve secrets from ~/.foundryvars file with instance scoping

Tasks:

Implement internal/secrets/foundryvars.go:

type FoundryVarsResolver struct {
    vars map[string]string  // instance/path:key -> value
}

func NewFoundryVarsResolver(path string) (*FoundryVarsResolver, error)
func (f *FoundryVarsResolver) Resolve(ctx *ResolutionContext, ref SecretRef) (string, error)

Parse file format: instance/path/to/secret:key=value
- Example: myapp-prod/database/main:password=secretpass123
- Example: foundry-core/openbao:token=root-token
Handle comments (lines starting with #)
Handle empty lines
Trim whitespace
Store in map for lookups
Use instance from ResolutionContext to construct lookup key
Write unit tests with temp files

Test Criteria:

Can parse valid .foundryvars file
Ignores comments and empty lines
Resolves secrets correctly with instance context
Returns error for missing secrets
Tests use temp files

Files Created:

v1/internal/secrets/foundryvars.go
v1/internal/secrets/foundryvars_test.go
v1/test/fixtures/test-foundryvars

9. Secret Resolution - OpenBAO (Stub) ✅

Working State: OpenBAO resolver interface defined, returns placeholder

Tasks:

Implement internal/secrets/openbao.go:

type OpenBAOResolver struct {
    client *openbaoClient  // Placeholder for now
}

func NewOpenBAOResolver(addr, token string) (*OpenBAOResolver, error)
func (o *OpenBAOResolver) Resolve(ctx *ResolutionContext, ref SecretRef) (string, error)

For now, return error: "OpenBAO integration not yet implemented"
Define interface that we'll implement in Phase 2
Document expected path structure: <instance>/<path>:<key>
Write stub tests that expect the error

Test Criteria:

Resolver can be instantiated
Resolve() returns not-implemented error
Tests pass

Files Created:

v1/internal/secrets/openbao.go
v1/internal/secrets/openbao_test.go

10. Secret Resolution - Chain Resolver ✅

Working State: Can resolve secrets by trying multiple sources in order

Tasks:

Implement internal/secrets/chain.go:

type ChainResolver struct {
    resolvers []Resolver
}

func NewChainResolver(resolvers ...Resolver) *ChainResolver
func (c *ChainResolver) Resolve(ctx *ResolutionContext, ref SecretRef) (string, error)

Try each resolver in order (env, foundryvars, openbao)
Return first successful resolution
If all fail, return aggregate error with details
Write unit tests with mock resolvers

Test Criteria:

Tries resolvers in order (env, foundryvars, openbao)
Returns first successful result
Returns error if all fail
Error message lists all failures
Tests use mock/stub resolvers

Files Created:

v1/internal/secrets/chain.go
v1/internal/secrets/chain_test.go

11. Configuration Secret Resolution ✅

Working State: Can validate config without resolving secrets, can resolve when instance context provided

Tasks:

Implement internal/config/resolve.go:

func ValidateSecretRefs(cfg *Config) error  // Just validate syntax
func ResolveSecrets(cfg *Config, ctx *secrets.ResolutionContext, resolver secrets.Resolver) error

ValidateSecretRefs(): Parse all secret refs, ensure valid syntax
- Used during foundry config validate
- Doesn't actually resolve secrets
- Returns error if any secret reference is malformed
ResolveSecrets(): Actually resolve secrets
- Requires ResolutionContext (instance must be provided)
- Walk config structure recursively
- Find all string values that are secret references
- Resolve each using chain resolver
- Replace reference with actual value in config
- Handle resolution errors gracefully
Write unit tests with mock resolver

Test Criteria:

Can validate secret refs without resolution
Can resolve secrets when instance context provided
Handles missing secrets with clear error
Leaves non-secret values unchanged
Tests use fixtures with secret refs

Files Created:

v1/internal/config/resolve.go
v1/internal/config/resolve_test.go

Design Note: This allows foundry config validate to check syntax without needing actual secrets. Actual resolution happens when deploying/installing components, where instance context is known.

12. SSH Connection Types ✅

Working State: SSH connection data structures defined

Tasks:

Implement internal/ssh/types.go:

type Connection struct {
    Host       string
    Port       int
    User       string
    AuthMethod ssh.AuthMethod
    client     *ssh.Client
}

type KeyPair struct {
    Private []byte
    Public  []byte
}

Define connection options struct
Define auth method types (password, key)
Write basic validation

Test Criteria:

Structs compile
Validation works
Tests pass (100% coverage)

Files Created:

v1/internal/ssh/types.go
v1/internal/ssh/types_test.go

13. SSH Key Generation ✅

Working State: Can generate SSH key pairs

Tasks:

Implement internal/ssh/keygen.go:

func GenerateKeyPair() (*KeyPair, error)
func (kp *KeyPair) PublicKeyString() string  // OpenSSH format
func (kp *KeyPair) PrivateKeyPEM() []byte

Use crypto/ed25519 (modern, secure, fast)
Generate key pair
Encode private key as PEM
Format public key as OpenSSH authorized_keys format
Write unit tests

Test Criteria:

Files Created:

v1/internal/ssh/keygen.go
v1/internal/ssh/keygen_test.go

14. SSH Connection Establishment ✅

Working State: Can connect to SSH server with password or key

Tasks:

Implement internal/ssh/connect.go:

func Connect(host string, port int, user string, auth ssh.AuthMethod) (*Connection, error)
func (c *Connection) Close() error
func (c *Connection) IsConnected() bool

Use golang.org/x/crypto/ssh
Support password auth
Support public key auth
Set reasonable timeouts (30s connection, 60s overall)
Handle connection errors
Write unit tests (integration tests deferred)

Test Criteria:

Validation errors handled correctly
Network errors handled correctly
Connection closes cleanly
Unit tests pass

Files Created:

v1/internal/ssh/connect.go
v1/internal/ssh/connect_test.go

Note: Integration tests with testcontainers deferred to Task 26

15. SSH Command Execution ✅

Working State: Can execute commands over SSH and capture output

Tasks:

Implement internal/ssh/exec.go:

type ExecResult struct {
    Stdout   string
    Stderr   string
    ExitCode int
}

func (c *Connection) Exec(command string) (*ExecResult, error)
func (c *Connection) ExecWithTimeout(command string, timeout time.Duration) (*ExecResult, error)

Create session from connection
Capture stdout and stderr
Get exit code
Handle timeouts
Write unit tests

Test Criteria:

Validation errors handled correctly
ExecMultiple works correctly
Unit tests pass

Files Created:

v1/internal/ssh/exec.go
v1/internal/ssh/exec_test.go

Note: Integration tests with real SSH server deferred to Task 26

16. SSH Key Storage (OpenBAO Stub) ✅

Working State: Interface for storing SSH keys defined

Tasks:

Implement internal/ssh/storage.go:

type KeyStorage interface {
    Store(host string, key *KeyPair) error
    Load(host string) (*KeyPair, error)
    Delete(host string) error
}

type OpenBAOKeyStorage struct {
    // Stub for now
}

Define interface
Create stub implementation that returns errors
Document expected storage path: foundry-core/ssh-keys/<hostname>
Write stub tests

Test Criteria:

Interface is defined
Stub returns appropriate errors
Tests pass (100% coverage for stub)

Files Created:

v1/internal/ssh/storage.go
v1/internal/ssh/storage_test.go

17. Host Management Types ✅

Working State: Host data structures defined

Tasks:

Implement internal/host/types.go:

type Host struct {
    Hostname  string
    Address   string
    Port      int
    User      string
    SSHKeySet bool
}

type HostRegistry interface {
    Add(host *Host) error
    Get(hostname string) (*Host, error)
    List() ([]*Host, error)
    Remove(hostname string) error
}

Define host struct
Define registry interface
Write validation

Test Criteria:

Structs compile
Validation works
Tests pass (100% coverage)

Files Created:

v1/internal/host/types.go
v1/internal/host/types_test.go

18. Host Registry (In-Memory) ✅

Working State: Can store and retrieve host info (in-memory only for now)

Tasks:

Implement internal/host/registry.go:

type MemoryRegistry struct {
    hosts map[string]*Host
    mu    sync.RWMutex
}

func NewMemoryRegistry() *MemoryRegistry
// Implement HostRegistry interface

Implement all registry methods with in-memory storage
Thread-safe with mutexes
Write unit tests

Test Criteria:

Files Created:

v1/internal/host/registry.go
v1/internal/host/registry_test.go

Note: Real registry (backed by config or OpenBAO) comes later

19. CLI Command: `foundry config init` ✅

Working State: Can create a new config file interactively or with defaults

Tasks:

Implement cmd/foundry/commands/config/init.go
Register command with urfave/cli v3
Implement interactive prompts (for now, just use basic stdin)
- Cluster name
- Domain
- Single node or multi-node
Generate config struct with defaults
Write to ~/.foundry/<name>.yaml
Handle file already exists error
Add --force flag to overwrite
Write integration tests

Test Criteria:

foundry config init creates config file
Config file is valid YAML
Can load created config
Error if file exists without --force
Tests use temp directories

Files Created:

cmd/foundry/commands/config/init.go
cmd/foundry/commands/config/init_test.go

20. CLI Command: `foundry config validate` ✅

Working State: Can validate config file syntax and structure (including secret ref syntax)

Tasks:

Implement cmd/foundry/commands/config/validate.go
Load config from path
Run structural validation
Run secret reference validation (syntax only, no resolution)
Output success or detailed error
Exit with appropriate code
Write tests

Test Criteria:

Files Created:

cmd/foundry/commands/config/validate.go
cmd/foundry/commands/config/validate_test.go

21. CLI Command: `foundry config show` ✅

Working State: Can display current config with secrets redacted

Tasks:

Implement cmd/foundry/commands/config/show.go
Load config
Redact any secret references (replace with [SECRET])
Output as formatted YAML
Add --show-secret-refs flag to show secret ref syntax (not values)
Write tests

Test Criteria:

Shows config correctly
Secrets are redacted by default
--show-secret-refs shows ${secret:path:key} syntax
Tests pass

Files Created:

cmd/foundry/commands/config/show.go
cmd/foundry/commands/config/show_test.go

Note: Changed from --with-secrets since we can't resolve secrets without instance context. We can only show the reference syntax.

22. CLI Command: `foundry config list` ✅

Working State: Can list available config files

Tasks:

Implement cmd/foundry/commands/config/list.go
Find all *.yaml files in ~/.foundry/
Display list with indicators for active config
Show which config would be used by default
Write tests

Test Criteria:

Lists all configs
Shows active config
Works with empty directory
Tests use temp directories

Files Created:

cmd/foundry/commands/config/list.go
cmd/foundry/commands/config/list_test.go

23. CLI Command: `foundry host add` ✅

Working State: Can add a host via interactive prompts

Tasks:

Implement cmd/foundry/commands/host/add.go
Prompt for hostname, address, user
Test SSH connection with password
Generate SSH key pair
Install public key on host
Store private key (stub for now - just in-memory)
Add host to registry (in-memory)
Write unit tests

Test Criteria:

Can add host interactively
SSH key is generated and installed
Host appears in registry
Unit tests pass

Files Created:

cmd/foundry/commands/host/add.go
cmd/foundry/commands/host/add_test.go
cmd/foundry/commands/host/commands.go

24. CLI Command: `foundry host list` ✅

Working State: Can list all registered hosts

Tasks:

Implement cmd/foundry/commands/host/list.go
Query host registry
Display table with hostname, address, user, key status
Handle empty registry gracefully
Write tests

Test Criteria:

Lists hosts correctly
Shows empty state message
Table formatting works
Tests pass

Files Created:

cmd/foundry/commands/host/list.go
cmd/foundry/commands/host/list_test.go

25. CLI Command: `foundry host configure` ✅

Working State: Can run basic configuration on a host

Tasks:

Implement cmd/foundry/commands/host/configure.go
Connect to host
Run basic setup commands:
- Update packages (apt-get update)
- Install common tools (curl, git, vim, htop)
- Configure time sync
Show progress
Handle errors gracefully
Write unit tests

Test Criteria:

Can execute configuration steps
Error handling works
Progress is shown
Unit tests pass

Files Created:

cmd/foundry/commands/host/configure.go
cmd/foundry/commands/host/configure_test.go

26. Integration Tests - Full Workflow ✅

Working State: End-to-end test of Phase 1 functionality

Tasks:

Create test/integration/phase1_test.go
Test full workflow:
1. Create config
2. Validate config (including secret refs)
3. SSH connection with testcontainers
4. Host registry operations
5. SSH command execution
6. Secret resolution with instance context
Use testcontainers for SSH server
Clean up resources after test
Add testcontainers dependency

Test Criteria:

Full workflow completes successfully
All assertions pass
Resources cleaned up properly
Tests pass with -short flag

Files Created:

test/integration/phase1_test.go
Added testcontainers-go dependency

Note: Integration tests use build tag integration and skip in short mode

27. Documentation - Phase 1 ✅

Working State: Comprehensive documentation for Phase 1 features

Tasks:

Create docs/getting-started.md:
- Installation instructions
- Quick start guide
- Common commands
- Troubleshooting
Create docs/configuration.md:
- Config file format and schema
- Secret references with instance scoping
- Multi-config support
- Best practices
Create docs/secrets.md:
- Secret resolution order
- Instance context and namespacing
- Using ~/.foundryvars for development
- Environment variable format
- Security considerations
Create docs/hosts.md:
- Host management workflows
- SSH key handling
- Configuration steps
- Advanced usage examples
Update README.md with Phase 1 completion status

Test Criteria:

Documentation is clear and comprehensive
Examples are complete and correct
Links are valid
Instance context for secrets is well explained

Files Created:

docs/getting-started.md (173 lines)
docs/configuration.md (308 lines)
docs/secrets.md (464 lines)
docs/hosts.md (587 lines)
Updated README.md

Phase 1 Completion Checklist

Before considering Phase 1 complete, verify:

Phase 1 Status: ✅ COMPLETE

Key Design Decisions - Phase 1

Instance Context for Secrets

Secret references in config: ${secret:database/prod:password}
Resolution requires instance context: myapp-prod/database/prod:password
This allows same config to be used for multiple instances
Validation can check syntax without resolution
Actual resolution happens when instance is known (during deployment)

No `foundry host ssh` Command

Users can use their own SSH clients
Foundry manages keys and connection info
Focus on automation, not manual access

urfave/cli v3

Using latest version for modern CLI features
Check v3 docs for any API changes from v2

Dependencies for Phase 2

Phase 1 provides the foundation for Phase 2:

Configuration system ✓
Secret resolution (partial - OpenBAO stub) ✓
Instance-scoped secret paths ✓
SSH management ✓
Host registry ✓
CLI framework ✓

Phase 2 will implement:

OpenBAO installation and integration
Real OpenBAO secret resolution
K3s cluster management
Component deployment system

Estimated Working States: 26 testable states (removed foundry host ssh) Estimated LOC: ~3000-4000 lines (including tests) Timeline: Not time-bound - proceed at natural pace

FilesExpand file tree

phase-implementation-1.md

Latest commit

History

phase-implementation-1.md

File metadata and controls

Phase 1: Foundation - Implementation Tasks

Task Breakdown

1. Project Initialization ✅

Tasks:

2. Configuration Data Structures ✅

Tasks:

3. Configuration File Loading ✅

Tasks:

4. Configuration Path Resolution ✅

Tasks:

5. Secret Reference Parsing ✅

Tasks:

6. Secret Resolution Context ✅

Tasks:

7. Secret Resolution - Environment Variables ✅

Tasks:

8. Secret Resolution - ~/.foundryvars ✅

Tasks:

9. Secret Resolution - OpenBAO (Stub) ✅

Tasks:

10. Secret Resolution - Chain Resolver ✅

Tasks:

11. Configuration Secret Resolution ✅

Tasks:

12. SSH Connection Types ✅

Tasks:

13. SSH Key Generation ✅

Tasks:

14. SSH Connection Establishment ✅

Tasks:

15. SSH Command Execution ✅

Tasks:

16. SSH Key Storage (OpenBAO Stub) ✅

Tasks:

17. Host Management Types ✅

Tasks:

18. Host Registry (In-Memory) ✅

Tasks:

19. CLI Command: foundry config init ✅

Tasks:

20. CLI Command: foundry config validate ✅

Tasks:

21. CLI Command: foundry config show ✅

Tasks:

22. CLI Command: foundry config list ✅

Tasks:

23. CLI Command: foundry host add ✅

Tasks:

24. CLI Command: foundry host list ✅

Tasks:

25. CLI Command: foundry host configure ✅

Tasks:

26. Integration Tests - Full Workflow ✅

Tasks:

27. Documentation - Phase 1 ✅

Tasks:

Phase 1 Completion Checklist

Key Design Decisions - Phase 1

Instance Context for Secrets

No foundry host ssh Command

urfave/cli v3

Dependencies for Phase 2

19. CLI Command: `foundry config init` ✅

20. CLI Command: `foundry config validate` ✅

21. CLI Command: `foundry config show` ✅

22. CLI Command: `foundry config list` ✅

23. CLI Command: `foundry host add` ✅

24. CLI Command: `foundry host list` ✅

25. CLI Command: `foundry host configure` ✅

No `foundry host ssh` Command