Skip to content
BranchesTags

Repository files navigation

Taro

Taro is an experimental programming language that draws inspiration from Rust, Swift, and Golang. It features a strong static type system and automatic garbage collection, with a familiar syntax inspired by all three languages.

Prerequisites

  • Rust: Latest stable version
  • LLVM: Version 22.1.x (22.1.8 is the certified development version)
  • C++: A C++17 compiler for Taro's narrow LLVM ThinLTO shim

LLVM Setup

Repository scripts and Make targets resolve LLVM 22.1 automatically. They honor LLVM_SYS_221_PREFIX first, then check versioned llvm-config binaries, Homebrew, and common Unix installation prefixes. Confirm the selected toolchain before building:

make llvm-check

On macOS with Homebrew:

brew install llvm@22
export LLVM_SYS_221_PREFIX="$(brew --prefix llvm@22)"

On Linux, point the same variable at the installation prefix containing bin/llvm-config; a common packaged layout is:

export LLVM_SYS_221_PREFIX=/usr/lib/llvm-22

The export is optional for repository scripts when discovery succeeds, but is recommended when invoking cargo directly or when multiple LLVM releases are installed.

Quick Start

  1. Build the compiler, runtime, and standard library:

    python3 development/scripts/build_dist.py

  2. Create hello.tr:

    func main() {
    print("Hello, World!\n")
}

  3. Run the file with your local distribution:

    python3 development/scripts/run_dist.py hello.tr

  4. Run tests in a file:

    python3 development/scripts/run_dist.py --test std/src/tests/testing/testing_tests.tr

Status and Limitations

Taro is experimental. Syntax, compiler metadata, standard library APIs, and package tooling can change between commits.

  • Repository workflows currently support Unix-like hosts with LLVM 22.1.x; Windows remains out of scope.
  • The standard library is an attached toolchain artifact. Rebuild dist/ after compiler metadata changes or when attached std artifacts are missing.
  • Package management supports manifests, lockfiles, Git dependencies, and root-local path dependencies, but there is no public registry yet.
  • Incremental compilation reuses unchanged semantic and codegen artifacts for dependencies, root packages, and single-file commands. Executables are relinked for the current output path and linker inputs.
  • Operator overloading is expressed through standard library interfaces such as std.ops.Add, not operator declarations.
  • The language server supports package-wide diagnostics, navigation, references and highlights, package-scoped rename, hierarchical symbols, semantic tokens, inlay hints, signature help, and lexical/member completions. Formatting and code actions are not yet implemented.
  • .taro_meta files are binary internal compiler artifacts, not a stable interchange format.

Build and Run

Build Distribution

To build the compiler, runtime, and standard library from source, use build_dist.py. This creates a distribution directory with a sysroot-like structure (dist/ by default). For automation/bench workflows, build_dist.py also supports --profile, --dist-dir, and --target.

python3 development/scripts/build_dist.py

Run with Distribution Scripts

To compile and run a Taro program (script or package) using your locally built compiler, use run_dist.py. This script rebuilds the distribution before running and sets up the strict environment (TARO_HOME, etc.) for you.

python3 development/scripts/run_dist.py examples/hello.tr
python3 development/scripts/run_dist.py examples/hello.tr foo bar

To run a file's test suite instead, pass --test:

python3 development/scripts/run_dist.py --test std/src/tests/testing/testing_tests.tr

Manual CLI Usage

If you install Taro as a toolchain with binaries under <toolchain>/bin, both taro and taro-lsp can infer TARO_HOME from that layout automatically.

For a local repo dist/ or any other portable/custom layout, set TARO_HOME explicitly:

export TARO_HOME=$(pwd)/dist
dist/bin/taro build examples/hello.tr
dist/bin/taro run examples/hello.tr -- foo bar

When reading arguments in Taro, std.env.argv() / std.env.args() include argv[0], which is the actual generated executable path for taro run.

CLI Cheatsheet

Use dist/bin/taro with TARO_HOME=$(pwd)/dist for repo-local development. Use plain taro when running from an installed toolchain whose bin/ directory is on PATH.

Task Command
Type-check a file dist/bin/taro check examples/hello.tr --std-path std
Build a file dist/bin/taro build examples/hello.tr --std-path std
Run a file with arguments dist/bin/taro run examples/hello.tr --std-path std -- foo bar
Run tests in a file dist/bin/taro test std/src/tests/testing/testing_tests.tr --std-path std
Run package tests dist/bin/taro test std --std-path std
Run benchmarks dist/bin/taro bench path/to/package --std-path std
Create a package dist/bin/taro new github.com/acme/app

Common flags:

Flag Use
--std-path <PATH> Point the compiler at std sources when using repo-local layouts.
--build-std Rebuild and publish attached std artifacts into TARO_HOME.
--release Build with the release profile.
-O<0|1|2|3|s|z> Select LLVM's optimization pipeline independently of the build profile.
--target <TRIPLE> Compile for a target triple override.
--linker <PATH> Use a Clang-compatible linker driver for the selected target.
--sysroot <PATH> Use a target SDK/sysroot while linking.
--emit <link|llvm-bc> Select the final build artifact; defaults to a normal linked output.
--lto <off|full|thin> Apply the selected LTO mode to participating Taro packages for build/run; defaults to off.
--timings Print compiler phase timings.
--optimization-remarks <PASS_REGEX> Print LLVM passed, missed, and analysis remarks for matching pass names.
--dump-mir / --dump-llvm Dump intermediate compiler output for debugging.
--debug-info <none|line-tables> Select source debug metadata. Debug builds default to line tables; release builds default to none.
--no-incremental Disable dependency artifact reuse.
--locked Require package.lock to match dependency resolution exactly; use cached locked Git revisions without fetching when available.
--update-lock Refresh lockfile entries from current dependency sources.

An explicit -O selects LLVM's maintained module pipeline for that level. If it is omitted, debug builds retain the fast baseline pipeline and release builds use O2. --release also controls language defaults such as overflow checks; the profile and optimization level remain independently selectable.

LLVM 22 owns instruction-selector policy. On supported AArch64 targets, O0 uses GlobalISel with per-function SelectionDAG fallback; O1 and above retain SelectionDAG. Other architectures keep their LLVM target defaults. This gives debug builds the maintained AArch64 fast path without trading away optimized code quality or turning an incomplete lowering into a compiler crash.

Create a New Package

Use taro new to scaffold a package from a full package identifier:

taro new github.com/acme/app
taro new github.com/acme/lib --kind library
taro new github.com/acme/tool --kind both

This creates ./app or ./lib based on the repo segment of the package identifier.

Generated templates currently support:

  • --kind executable (default): writes src/main.tr
  • --kind library: writes src/lib.tr
  • --kind both: writes src/lib.tr and src/main/main.tr

VS Code Extension

The VS Code extension is designed for an external Taro toolchain install:

  • put the toolchain bin/ directory on your PATH
  • ensure the toolchain root contains attached std artifacts under lib/taro/std/<target-triple>/
  • use taro.languageServer.path only for custom taro-lsp locations
  • use taro.languageServer.env only for advanced overrides such as a custom TARO_HOME

Repo-local target/debug/taro-lsp and dist/ are still supported as a development fallback when working inside the Taro repository.

For the daily-driver repo workflow, build the local toolchain and language server together:

make lsp

This places both taro and taro-lsp under dist/bin/, with attached std artifacts under dist/lib/taro/std/<target-triple>/.

Language Server

taro-lsp currently provides:

  • package-wide diagnostics (parse/resolve/typecheck and related info) on open/change/save and watched source/manifest/lockfile changes
  • hover
  • go-to-definition
  • references
  • document highlights
  • package-scoped rename with prepare support
  • hierarchical document symbols
  • full-document semantic tokens
  • inferred-type and parameter-name inlay hints
  • signature help
  • completion for in-scope names plus probe-backed member/static-member contexts

Completion covers lexical names plus member/static-member candidates for identifier, dotted-path, call, parenthesized, indexed, and optional-chain receivers. VS Code should automatically request lexical completions when typing an identifier-start character (A-Z, a-z, or _), and the server runs an internal completion probe for incomplete member syntax, so point., point.m, and Heading.n can complete before the source is syntactically complete. Semantic tokens are full-document only, and inlay hints cover inferred local/closure-parameter types plus names for unlabeled call arguments. Formatting and code actions are not part of the current LSP surface.

Manual smoke fixture: open examples/lsp_smoke.tr from the repository root in the VS Code extension development host. Expected checks:

  • retyping l in lexicalProbe = localValue automatically opens lexical completions and offers localValue
  • point. offers x, y, and magnitude
  • point.m filters to magnitude
  • Heading. offers north, south, east, and west
  • Heading.n filters to north
  • describe( shows signature help
  • hover/go-to-definition work on SmokePoint, Heading, point.x, and Heading.south
  • references on point.x include the field declaration and all uses
  • prepare-rename and rename update package-owned references without touching dependencies

Compiler Timings

To print compiler phase timings (parse/typecheck/THIR/MIR/codegen/link), pass --timings:

taro build examples/hello.tr --timings
taro check examples/hello.tr --timings

benchmark_timings.py bootstraps per-profile distributions via build_dist.py and reports timing comparison tables. Each sample passes --no-incremental so the results measure cold compiler phases instead of cache reuse:

python3 development/scripts/benchmark_timings.py examples/hello.tr
python3 development/scripts/benchmark_timings.py examples/hello.tr --runs 10
python3 development/scripts/benchmark_timings.py examples/hello.tr --command run --runs 5

codegen_benchmarks.py compares the retained release baseline against O2. It alternates variant order to avoid thermal/order bias, disables incremental reuse for compile samples, warms executable runs, verifies identical output, and reports median compile time, median runtime, and executable size:

make codegen-benchmark
make codegen-benchmark RUNS=10
python3 development/scripts/codegen_benchmarks.py path/to/workload.tr --runs 10

To inspect why LLVM applied or rejected a transformation, filter optimization remarks by LLVM pass name. Add line-table debug information when source locations are useful:

taro build examples/arithmetic.tr --release \
  --optimization-remarks 'inline|loop-vectorize' \
  --debug-info line-tables

LLVM Bitcode Artifacts

taro build --emit llvm-bc emits the verified, post-optimization LLVM module without performing native object generation or linking. Runtime libraries, linkers, and sysroots are therefore not required for this output mode:

taro build examples/arithmetic.tr --release --emit llvm-bc -o arithmetic.bc

Without -o, package bitcode is written to target/<profile>/<package-name>.bc; a single source file is written as <current-directory>/<file-stem>.bc. The command prints the selected path. Dependency bitcode remains in the profile's internal artifact directory and is covered by incremental metadata. --emit llvm-bc is a terminal artifact mode: it does not additionally produce an executable.

Full Link-Time Optimization

taro build --lto full and taro run --lto full preserve each participating Taro package as LLVM bitcode, merge the complete user-package graph inside the compiler, run LLVM's full-LTO pipeline, and emit one native object for the normal platform linker. Release builds provide the useful default combination:

taro build . --release --lto full

Full LTO is opt-in. Attached std and the Rust runtime remain precompiled native libraries and form explicit optimization boundaries. This avoids requiring a system linker plugin that understands Taro's LLVM version while still enabling cross-package optimization for application and library code.

Incremental builds reuse each package's metadata and bitcode, then rerun the whole-program optimization and native link. --lto full cannot be combined with --emit llvm-bc: the latter intentionally requests one package-scoped bitcode artifact rather than a whole-program linked output.

Thin Link-Time Optimization

taro build --lto thin and taro run --lto thin retain the same package-level bitcode boundary as full LTO, but LLVM summarizes the modules, imports useful definitions across package boundaries, and generates native objects in parallel. It is the preferred LTO mode when compile-time and incremental-build cost matter:

taro build . --release --lto thin

ThinLTO uses a profile-scoped cache under target/<profile>/objects/thinlto-cache/. Unchanged package metadata and bitcode are reused first; LLVM then reuses cached native ThinLTO backends where their module summary, target CPU/features, optimization level, and preserved symbol set still match. --no-incremental bypasses both package reuse and the ThinLTO cache without deleting existing cache entries.

Generated linker inputs live under target/<profile>/objects/thinlto-objects/ and are refreshed for every final link. Externally visible Taro definitions remain exported because attached std, the runtime, and other native objects are opaque to ThinLTO. As with full LTO, no system linker plugin is required and --emit llvm-bc cannot be combined with an enabled LTO mode.

Incremental Compilation

Incremental dependency reuse is enabled by default for:

  • taro build
  • taro run
  • taro test
  • taro check

Per dependency package, the compiler emits:

  • target/<profile>/metadata/<package-identifier>.taro_meta
  • target/<profile>/objects/<package-identifier>.o (linked build/run/test paths)
  • target/<profile>/objects/<package-identifier>.bc (--emit llvm-bc and LTO inputs)
  • target/<profile>/objects/<root-identifier>.lto.o (full-LTO linked builds)
  • target/<profile>/objects/thinlto-objects/*.o (ThinLTO linker inputs)
  • target/<profile>/objects/thinlto-cache/llvmcache-* (incremental ThinLTO backends)

Reuse is mode-aware:

  • Linked build/run/test reuse dependency metadata + object artifacts.
  • Bitcode builds reuse dependency metadata + bitcode artifacts; artifact kinds cannot satisfy one another's cache entries.
  • Full-LTO builds reuse package bitcode, then regenerate the final LTO object.
  • ThinLTO builds reuse package bitcode and matching LLVM backend cache entries, then refresh their final linker-input objects.
  • check reuses dependency semantic metadata only (no object requirement).

Attached Std Artifacts (Strict)

std is treated as an attached toolchain artifact by default. The compiler expects prebuilt std artifacts in TARO_HOME and does not silently rebuild std on cache misses.

Expected attached std artifact layout:

  • TARO_HOME/lib/taro/std/<target-triple>/std.taro_meta
  • TARO_HOME/lib/taro/std/<target-triple>/std.o

For the repository-local distribution, this resolves to dist/lib/taro/std/<target-triple>/....

On missing/invalid std artifacts, the compiler errors with guidance to rebuild std explicitly.

Use --build-std to rebuild and publish attached std artifacts from source. Attached std is built in a canonical release-like configuration per target (shared across debug/release user builds):

TARO_HOME=$(pwd)/dist dist/bin/taro check examples/hello.tr --std-path std --build-std

Unix Cross-Compilation

--target controls LLVM object generation, attached std selection, runtime selection, and the final linker invocation. Target-specific runtime archives use:

  • TARO_HOME/lib/taro/runtime/libtaro_runtime.a for host builds
  • TARO_HOME/lib/taro/runtime/<target-triple>/libtaro_runtime.a

Every runtime archive has an adjacent .manifest.toml sidecar. Before linking, the compiler verifies its runtime ABI revision and fingerprint, exact target, object architecture, and archive SHA-256. This validation also applies to --runtime-path and TARO_RUNTIME_LIB, so custom archives must include the sidecar produced alongside them by the toolchain.

Distribution tooling generates sidecars automatically. For an advanced custom archive, use taro runtime-manifest /path/to/libtaro_runtime.a; add --target <TRIPLE> when the archive is not host-only.

Same-OS Darwin cross-architecture builds use the host macOS SDK automatically. Linux cross-architecture builds require --linker or --sysroot; cross-OS Darwin/Linux builds require both. An explicit runtime can always be supplied with --runtime-path.

Build a target-ready distribution with:

python3 development/scripts/build_dist.py --target x86_64-apple-darwin

Metadata reuse is guarded by format/version/compiler stamp/target/options/fingerprint/checksum validation for normal dependency caches.

Incremental compilation also reuses an unchanged root package's semantic and codegen artifacts. The current executable is still relinked so its output path and linker inputs are honored.

Use --no-incremental to force a cold path:

taro build my-package --no-incremental
taro run my-package --no-incremental
taro test my-package --no-incremental
taro check my-package --no-incremental

Metadata files use the .taro_meta extension and a binary internal ABI (not JSON).

Makefile Commands

For day-to-day development, you can use the root Makefile:

make help
make run FILE=examples/hello.tr
make check FILE=examples/hello.tr
make lsp
make test
make language-tests
make std-tests
make runtime-stress
make all-tests
make benchmark PACKAGE=std
make codegen-benchmark

Panic Stack Traces

Panic reports default to compact Taro-first output:

  • prefer language-level frames under taro stack:
  • otherwise render a filtered native backtrace (keeping Taro/std/synthetic entry frames)
  • if filtering would be empty, fall back to a short raw trace so output is never blank

When a symbol is synthetic and no source definition symbol exists, stack entries use a stable fallback name (missing_p{pkg}_d{idx}) instead of debug-formatted IDs.

Set TARO_BACKTRACE=full to print the full unfiltered native backtrace.

Language Basics

Here are a few examples to showcase the familiar yet distinct syntax.

Structs & Methods

import std.ops.Add

struct Point {
    x: int32
    y: int32
}

impl Point {
    // Static `new(...)` methods can also be called as `Point(x: ..., y: ...)`.
    func new(x: int32, y: int32) -> Point {
        return Point { x, y } // shorthand for { x: x, y: y }
    }

    func distance_squared(self) -> int32 {
        self.x * self.x + self.y * self.y // implicit return
    }
}

impl Add for Point {
    func add(self, rhs: Point) -> Point {
        Point { x: self.x + rhs.x, y: self.y + rhs.y }
    }
}

// Structs can also have immutable fields
struct Config {
    readonly id: int32
    debug: bool
}

Enums (Tagged Unions)

enum Message {
    case quit
    case move(int32, int32) // x, y
    case write(string)
}

func process(msg: Message) {
    match msg {
        case .quit => std.print("Quitting...\n")
        case Message.move(x, y) => std.print("Moving player\n") // fully qualified
        case .write(text) => std.print(text) // inferred
    }
}

Optional / Result Propagation

Postfix ! propagates Optional[T] and Result[T, E] values.

  • Optional[T]! extracts T or returns .none from the enclosing Optional context.
  • Result[T, E]! extracts T or returns .err(error) from the enclosing Result context. If the enclosing error type differs, it must implement From[E].
  • Propagation only works within the same container family.
  • For awaited values, write (await expr)!.
import std.io.Error
import std.prelude.Optional
import std.result.Result

func nextPort(raw: Optional[int32]) -> Optional[int32] {
    let port = raw!
    return .some(port + 1)
}

func readCount(input: Result[int32, Error]) -> Result[int32, Error] {
    let count = input!
    return .ok(count + 1)
}

func joinTask(task: std.task.Task[int32]) async -> Result[int32, std.task.TaskError] {
    let value = (await task.result())!
    return .ok(value)
}

Note

Additional examples are available in the examples directory.

Syntax Notes

  • Automatic Semicolon Insertion (ASI): Semicolons are optional at the end of statements.
  • Leading . on a New Line: A line that starts with . is parsed as postfix continuation of the previous expression unless the previous statement is explicitly terminated.
  • Trailing Commas: In multi-line sequences (like struct instantiation or lists), ensure you use explicit commas for the last element to prevent ASI from interpreting the newline as the end of the statement.
  • Integer Type Suffixes: Integer literals support suffixes in D_TY form:
    • Signed: _i8, _i16, _i32, _i64
    • Unsigned: _u8, _u16, _u32, _u64
    • Uppercase sign specifiers are accepted (_I32, _U64)
    • Examples: 1_u32, 200_i64, 0xFF_u16
let out = value
.some(out)     // parsed as: value.some(out)

let out = value;
.some(out)     // standalone inferred member expression
// Correct
let p = Point {
    x: 10,
    y: 20, // explicit comma required here if '}' is on next line
}

Benchmarking

Taro has a first-class benchmark harness. A benchmark is a synchronous, non-generic @bench function that accepts exactly one mutable Benchmark:

@bench
@tag("json", "smoke")
func parseSmall(benchmark: &mut std.bench.Benchmark) {
    let input = loadFixtureBeforeTiming()
    benchmark.setBytes(input.len())

    while benchmark.next() {
        let value = parse(input)
        std.hint.blackBox(value)
    }

    cleanupAfterTiming()
}

Code before the first next() call and after it returns false is outside the measured region. std.hint.blackBox(value) is an explicit, best-effort optimization barrier for inputs or results that LLVM could otherwise remove; it is not a lifetime, synchronization, security, or constant-time primitive. Use std.runtime.keepAlive for GC reachability.

taro bench my-package
taro bench my-package --list
taro bench my-package --filter json.parse --tag smoke
taro bench my-package --warmup 500ms --time 2s --samples 30
taro bench my-package --format json

Benchmarks compile as release/O2 by default; use --debug only when debugging the benchmark itself. The default measurement policy is 250 ms of warmup, one second divided over 20 measured samples, and a 30-second timeout per case. Each selected case runs in a fresh process to isolate GC and global state and to make timeouts enforceable. Name/tag selection and timing controls occur at runtime, so changing them reuses the same incremental benchmark artifact.

Human output reports median time/op, nearest-rank p95, MAD, iterations per sample, and throughput when setBytes is present. --format json adds raw samples and uses a versioned machine-readable schema; benchmark stdout is suppressed in that mode so it cannot corrupt the JSON document. Panics, timeouts, malformed harness results, and a benchmark that never calls next() are failures. Async, parallel, allocation-counting, sub-benchmark, and saved baseline APIs are intentionally not part of this first harness.

For repository development, make bench PACKAGE=path/to/package builds the local distribution and runs the language harness. The existing make benchmark PACKAGE=... target remains the cold compiler-timing tool.

Testing

Taro has a built-in test runner. Mark any () -> void function with @test and run it with taro test:

taro test my_file.tr
# or for a package:
taro test my-package/

Test Attributes

Attribute Description
@test Marks a function as a test case. Must be () -> void.
@tag Adds tags for test selection. Valid on @test functions and namespace declarations. Uses string literals: @tag("smoke", "slow").
@skip Skips the test. Accepts an optional reason string: @skip("not yet implemented").
@expectPanic Passes if the function panics, fails if it returns normally. An optional message is matched as a substring: @expectPanic("out of bounds"). Mismatches show the expected substring and actual panic report.

Filtering Tests

Use --filter to match qualified test names and --tag to select tagged tests:

taro test std --filter testing.testing_tests
taro test std --filter TESTING.TESTS
taro test std --tag smoke --tag slow
taro test std --filter testing --tag smoke

Rules:

  • --filter is a case-insensitive substring match against the qualified name.
  • . and :: are treated as equivalent separators when matching names.
  • --tag is case-insensitive and repeatable; multiple tags use OR semantics (any tag).
  • Combining --filter and --tag uses AND semantics (must satisfy both).
  • If nothing matches, the run succeeds with running 0 tests.

Test Example

import std.testing.{assertEqual, assertTrue, fail}

@test
func testAddition() {
    assertEqual(1 + 2, 3, "basic addition")
}

@test
@expectPanic
func testDivisionByZero() {
    let _ = 1 / 0
}

@test
@skip("pending implementation")
func testNotYetReady() {
    fail("not implemented")
}

@tag("smoke")
namespace CoreTests {
    @test
    func testNamespaceTagInheritance() {
        assertTrue(true, "namespace tags are inherited")
    }
}

@test
@tag("slow")
func testTaggedFunction() {
    assertTrue(true, "function tags are supported")
}

Running taro test on the above produces:

running 5 tests

test testAddition ... ok
test testDivisionByZero ... ok
test testNotYetReady ... SKIPPED
test CoreTests::testNamespaceTagInheritance ... ok
test testTaggedFunction ... ok

test result: ok. 4 passed; 0 failed; 1 skipped

Assertion Helpers

The standard library provides assertion helpers in std/testing:

Function Description
assertEqual(a, b, msg) Fails if a != b
assertTrue(cond, msg) Fails if cond is false
assertFalse(cond, msg) Fails if cond is true
fail(msg) Unconditionally fails the test

Repository Test Commands

To verify the compiler implementation, use the command that matches the test surface you want:

  • cargo test --workspace: Rust unit/integration/doctests for workspace crates.
  • make llvm-tests: Focused tests for LLVM 22.1 discovery, validation, and build environment setup.
  • python3 development/scripts/language_tests.py: Taro language E2E tests in language_tests/source_files. Runs in parallel by default using min(selected_tests, CPU core count) workers; --jobs is only needed to override (for example, --jobs 1 for serial mode). Bootstraps an isolated distribution via build_dist.py (release by default; pass --debug for debug bootstrap).
  • make codegen-matrix: Runs the high-risk LLVM codegen manifest with both debug and release generated-program profiles. On targets where LLVM selects GlobalISel, TARO_LLVM_STRICT_GLOBAL_ISEL=1 makes any per-function fallback fail the matrix.
  • make codegen-benchmark: Compares the retained release baseline with O2 for cold compile time, warmed runtime, executable size, and output equivalence.
  • make std-tests: Runs std package tests only (taro test std) via the test_all.py std stage.
  • python3 development/scripts/test_all.py: Unified fail-fast pipeline (development-script tests, cargo tests, dist build, std compile smoke, std package tests, language tests).
  • make all-tests: Shorthand for the unified pipeline.

Std package tests live under std/src/tests/<module>/<module_tests>.tr and run in the test_all.py std stage (or via make std-tests).

If you only want language tests with simple flags:

make language-tests            # JOBS auto-defaults to the language test runner default
make language-tests JOBS=4     # optional override
make language-tests FILTER=std_
make codegen-matrix JOBS=4     # focused backend coverage in both codegen profiles
make codegen-benchmark RUNS=10 # release baseline versus O2 measurements

Language Test Directives

Test files in language_tests/source_files/ can contain directive comments that control how the runner executes them:

Directive Description
// TEST Run with taro test instead of taro run. Passes if exit code is 0 (all tests pass). No output snapshot is compared.
// CHECK_ONLY Type-check only via taro check. No binary is produced or run.
// TARGET: <triple> Cross-compile for the given target triple.
// EXPECT_EXIT: <code> Expect the given exit code instead of 0.
// EXPECT_STDOUT_CONTAINS: <text> Assert that <text> appears in stdout.
// EXPECT_STDERR_CONTAINS: <text> Assert that <text> appears in stderr output.
// PACKAGE: <fixture> Copy and run language_tests/package_fixtures/<fixture>/app, including local dependency packages.

Use // TEST to write language tests that exercise the test harness itself:

// TEST
@test
func myTest() {
    std.testing.assertEqual(1 + 1, 2, "math works")
}

Packages

Package Management

Taro features a built-in package manager that feels familiar to users of Cargo or Go Modules.

  • Manifest: Packages are defined in a TOML manifest.
  • Dependencies: Supports Git-based dependencies (tags, branches, commits) and local paths.
  • Resolution: Selects one Git revision per package source, requiring that it satisfy every reachable semver request; incompatible requests fail with a conflict error.
  • Locking: Generates package.lock v2 with resolved revisions, dependency tree hashes, and the request list each locked package satisfies.
  • Integrity: Verifies installed Git dependencies against locked content hashes and fails on tampering.

Lockfile and Strict Mode

package.lock is generated automatically on dependency sync (build, check, run, test for package roots).

  • --locked: Require package.lock to be present and up to date; do not rewrite it. Locked Git revisions are used from the local cache when available, and Git is contacted only if the locked revision is missing.
  • --update-lock: Force lockfile refresh from current dependency sources.
  • CI=true: Enables strict lock behavior (same drift checks as --locked).

Security policy in this phase:

  • Transitive path dependencies are rejected. Only the root manifest may declare path dependencies.
  • Git cache identity is bound to canonical URL + package name, and cached repositories are origin-validated.
  • Existing v1 lockfiles must be regenerated; v2 lockfiles store requests = [...] for each locked package.

Package Structure

Taro packages follow a simple convention, similar to Cargo.

my-package/
├── package.toml   # Manifest file defining metadata and dependencies
├── src/           # Source code directory
│   └── main.tr    # Entry point
└── target/        # Build artifacts (automatically generated)

Manifest Format

The package.toml file must include a [package] section with a name field following the exact <host>/<author>/<project> convention. Extra path segments are not accepted yet.

kind is optional and defaults to executable. Supported values are library, executable, and both. Dependency packages may be library or both; both keeps its executable entry behavior when built as the root package.

[package]
name = "github.com/mantton/apple"
version = "0.1.0"
kind = "library"

Architecture and Features

Compiler Pipeline

Taro's compiler pipeline is deeply inspired by modern compiler designs (like Rust's rustc). It progresses through several distinct stages:

  • Parsing: Source code is parsed into an Abstract Syntax Tree (AST).
  • Name Resolution: Resolves identifiers to their definitions, linking usage to declaration.
  • HIR (High-level IR): The AST is lowered to a high-level intermediate representation where aggressive desugaring occurs.
  • Type Checking: A Bidirectional TypeChecker performs local expression type inference, handling function overloading and Swift-inspired optional coercions.
  • THIR (Typed HIR): The fully typed representation where intrinsic operations (like integer addition) are lowered distinctly from overloaded function calls.
  • MIR (Mid-level IR): A control-flow graph representation where significant optimizations happen (inlining, copy propagation, escape analysis).
  • Codegen: Handles monomorphization of generics and translates MIR to LLVM IR for final machine code generation.

Async Concurrency Runtime

Taro includes a multithreaded async runtime:

  • std.task.spawn runs async closures concurrently and returns Task[T]
  • Task.result() returns Result[T, TaskError] with cancellation/panic reporting
  • Awaited task panics are silent and inspectable through PanicPayload; detached or abandoned task panics are reported once as unobserved task panic
  • Task.cancel() and std.task.isCancelled() provide cancellation controls
  • std.task.dump() prints live task spawn chains and typed wait reasons; TARO_DEADLOCK_TIMEOUT_MS enables opt-in stuck-task and cycle diagnostics
  • An unconsumed Task is cancelled and reclaimed at scope exit. Use Task.detach() to transfer an existing handle, or std.task.detached(...) to launch explicit fire-and-forget work
  • withTaskGroup supports .cancelOnPanic and .independent policies
  • std.task.select races heterogeneous async operations while preserving the winning branch; std.task.race provides the same operation for one result type
  • std.task.withTimeout returns a distinct TimeoutError.timedOut and drains the cancelled operation before returning
  • std.task.sleep and std.io.task.AsyncStream provide timer and async I/O integration

The executor uses worker threads with work stealing. Worker count defaults to logical CPU count and can be overridden with the positive integer TARO_WORKERS; invalid values fail with a runtime configuration error. Use taro run --runtime-stats for a scheduler/I/O/GC summary and taro run --runtime-trace for a bounded, human-readable event trace. TARO_RUNTIME_TRACE_CAPACITY changes the default 4,096-event bound, up to 65,536 events. The runtime invariants are documented in docs/async-runtime.md, and make runtime-stress runs the async stress subset across multiple worker counts.

Memory Management

Taro uses a custom non-moving, mark-and-sweep garbage collector inspired by Golang's approach but tailored for simplicity and performance.

  • Structure: It uses a segregated-fit allocator with size classes and spans to minimize fragmentation.
  • Concurrency: Stop-the-world collection coordinates with runtime worker safepoints, with future plans for concurrent marking.
  • Safety: The compiler emits shadow stack frames and root slots to precisely identify stack roots.

std.weak.Weak(object) creates a typed, non-owning reference. weak.value() returns Optional[&T]: a live result is an ordinary strong reference snapshot, while .none means collection has cleared the target. Interior references are supported, and referenced locals are promoted to managed storage when needed. Weak references are cleared before cleanup callbacks for the same owner run.

Resources should still expose and prefer a deterministic close operation. std.runtime.addCleanup(&owner, state, callback) provides a fallback when an owner is abandoned: the synchronous, Sendable callback runs on a dedicated worker after the collector has resumed the world. Registration returns Result[Cleanup, CleanupError]; cleanup.cancel() prevents pending work and std.runtime.keepAlive(&owner) can make cancellation win a collection race. The cleanup state and callback must not retain the owner. Direct retention is reported as .ownerRetained, while indirect cycles remain the caller's responsibility. collect() queues eligible callbacks but does not wait for them; tests may use std.testing.waitForCleanups() when deterministic observation is required. Cleanup execution is not guaranteed before process exit.

Key Features

  • Enums: Tagged unions (sum types) allow for expressive invalid state modeling.
  • Generics: Full monomorphization support (like Rust/C++ templates) for zero-cost abstractions.
  • Move Semantics: Rust-style ownership and move semantics, with values moved by default and explicit copying for copyable types, but without a mutability uniqueness guarantee.
  • Async Concurrency: Multithreaded task runtime (std.task.spawn, cancellation, task groups, async sleep, async stream I/O).
  • Diagnostics: Rich, clear error messages to guide developers.
  • Basic LSP: Diagnostics, navigation, references/rename, symbols, semantic tokens, inlay hints, signature help, and lexical/member completions via taro-lsp.
  • Panic Reporting: Compact Taro-first panic stacks by default, with TARO_BACKTRACE=full for raw native traces.
  • Optimizations: Sophisticated MIR passes including inlining, escape analysis, and simplify-cfg.
  • Interoperability: C ABI compatibility for easy FFI.
  • Built-in Testing: First-class test support via taro test with @test, @tag, @skip, @expectPanic, plus --filter / --tag selection.

Repository Structure

  • compiler/: The core compiler source code (parsing, HIR, THIR, MIR, codegen).
  • compiler-cli/: The command-line interface implementation.
  • taro-bin/: The taro binary crate.
  • taro-lsp/: Language server implementation.
  • runtime/: Runtime components (garbage collector, async executor, panic/unwind support).
  • std/: The standard library implementation.
  • language_tests/: Comprehensive test suite for language features.
  • development/scripts/: Local build, test, and benchmark helper scripts.
  • docs/: Language and compiler internals documentation.
  • editors/: Editor integrations (VS Code, Zed).
  • tree-sitter-taro/: Tree-sitter grammar and editor syntax assets.

Roadmap

Taro is currently experimental.

  • Basic Compiler Pipeline (Parse -> Codegen)
  • Garbage Collection (Stop-the-world)
  • Generics and Monomorphization
  • Built-in Test Framework (taro test, @test, @tag, @skip, @expectPanic, --filter, --tag)
  • Async Concurrency Runtime (std.task, task groups, async timers, async I/O waits)
  • Basic LSP support and editor integration (taro-lsp, VS Code, Zed)
  • Package manager polish and registry
  • Standard library expansion

Troubleshooting

taro: command not found

Use dist/bin/taro from the repository root, or put an installed toolchain's bin/ directory on PATH.

Missing or invalid std artifacts

Build the local distribution again:

python3 development/scripts/build_dist.py

For a repo-local direct CLI call, make sure TARO_HOME points at dist/, then pass --std-path std --build-std when you want the compiler to rebuild std from the repository sources:

TARO_HOME=$(pwd)/dist dist/bin/taro check examples/hello.tr --std-path std --build-std

Runtime or linker path errors

Prefer the distribution scripts while debugging local layout problems:

python3 development/scripts/run_dist.py examples/hello.tr

For host builds, verify that TARO_HOME/lib/taro/runtime/libtaro_runtime.a and its .manifest.toml sidecar exist. For --target <TRIPLE>, verify both files under TARO_HOME/lib/taro/runtime/<TRIPLE>/; alternatively pass a manifested archive with --runtime-path. Cross-target linker setup can be supplied with --linker and --sysroot.

Unexpected cache or metadata behavior after compiler changes

Rebuild dist/, then retry with --no-incremental if a package-local cache is suspect. Metadata format bumps intentionally invalidate older .taro_meta files.

Lockfile drift in CI

CI=true behaves like strict lock mode. Run locally with --update-lock when dependency sources intentionally changed, then commit the updated v2 package.lock.

Language server does not start or completions are stale

Run make lsp, confirm dist/bin/taro-lsp exists, and make sure the editor extension is using the same toolchain layout as the CLI.

Contributing

Contributions are welcome. Check the docs directory for more information on the language internals.

About

The Taro Programming Language

Resources

Readme
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages