Contributing to Nytrix

Thanks for contributing to Nytrix.

Good contributions are **small, owned by the right layer, reproducible, and easy to verify**. Start from current source and observed behavior, make the narrowest correct change, and prove it with focused evidence.

---

1. Contribution Flow & Development Loop

For every contribution:

1. **Establish State** — Check git status, identify existing work, preserve unrelated changes.

2. **Find the Owning Layer** — Inspect the implementation that owns the behavior and its nearest callers.

3. **Reproduce Current Behavior** — Build the smallest reproducer, test fixture, or command.

4. **Design from First Principles** — Fix the owning layer directly. Migrate all affected callers. Avoid shims, fallbacks, or test-only shortcuts.

5. **Prove Correctness** — Validate with the original reproducer, add a narrow regression test, then broaden validation based on risk.

6. **Inspect & Tidy** — Run git diff --check, ./make tidy, and verify zero unexplained warnings or formatting defects.

> [!IMPORTANT]

> A skipped test, cache hit, fallback path, emitted IR artifact, or unexecuted platform is **never** proof that a change works.

---

2. Architecture & Design Principles

Nytrix development adheres to 17 core design principles across three axes:

Structural Principles

• **Foundational Thinking**: Identify the single layer that truly owns each invariant or data structure.
• **Redesign from First Principles**: Do not stack patches on top of structural defects; address the root cause.
• **Subtract Before You Add**: Simplify and remove obsolete paths before adding new machinery.
• **Exhaust the Design Space**: Evaluate at least 3 concrete alternative approaches before committing to large subsystem changes.
• **Outcome-Oriented Execution**: Define measurable, verifiable success criteria upfront.

Code Quality Principles

• **Laziness Protocol**: Do not compute or allocate data until strictly required.
• **Minimize Reader Load**: Keep functions focused, names precise, and interfaces clear.
• **Type System Discipline**:
• *Make illegal states unrepresentable* using sum types rather than collections of optional fields.
• *Brand semantic primitives* (UserId, ByteOffset, NYIRValue) so raw primitives are not accidentally interchangeable.
• *External data is untyped until parsed* at the boundary (RPC payloads, JSON, IPC messages, CLI flags, configuration).
• *Never lie to the type system* through unchecked casts or unsafe coercions.
• *Exhaustive matching* is mandatory when consuming enum and sum-type variants.
• **Boundary Discipline**: Keep subsystem interfaces explicit with typed enums/structs instead of magic string contracts.
• **Idempotent Operations**: Design transforms and passes such that repeated application produces stable fixed points.
• **Separate Shared State**: Eliminate global mutable state where it impedes parallelism, testing, or reasoning.

Process Principles

• **Fix Root Causes**: Solve issues at the owning source rather than masking symptoms downstream.
• **Prove It Works**: Validate using executable oracles, cold test runs, and absence of fallback blocks.
• **Guard Context**: Keep patches minimal, self-contained, and focused on the stated goal.
• **Build the Lever**: Create reusable diagnostics, oracles, and test fixtures that make future changes safer.
• **Experience First**: Preserve fast compiler turnaround, clean diagnostics, and intuitive developer tooling.

---

3. Repository Layout & Naming Conventions

Repository Structure

• src/ — Compiler frontend, NYIR optimizer, native code generation, runtime, and CLI tools.
• src/cmd/ — Standalone tool entry points (ny, test, fmt, fuzz, perf, doc, web, dap, lsp).
• src/code/ — AST, parser, semantic analysis, type inference, NYIR representation, optimization pipeline, and native backends.
• src/rt/ — Runtime amalgamation, memory allocator, value tagging, BigInt/GMP bridges, and platform primitives.
• src/base/ — Platform abstractions, common headers, and memory utilities.
• lib/ — Standard library modules (std.core, std.math, std.os, std.net, std.crypto, std.ui, etc.).
• etc/tests/ — Executable test fixtures, shape corpora, benchmarks, and error cases.
• etc/tests/native/ — Native shape tests (.nshape).
• etc/tests/runtime/ — Runtime integration tests (.ny).
• etc/tests/bench/ — Canonical benchmark shapes (.nshape).
• etc/tests/errors/ — Parser, lint, and semantic diagnostic test fixtures.
• etc/tests/native/web/ — Browser and WebAssembly tests.
• docs/ — Public specifications (docs/spec/), user guides (docs/learn/), and changelogs.

Naming Standards

• **Hyphens** for test fixtures and benchmarks: loop-unswitch.nshape, call-chain.nshape, float-cmp.ny.
• **Underscores** for C source and header files: loop_unswitch.c, alias_analysis.c, compat.h.
• **Module headers**: Keep ;; Keywords: ... first in all .ny modules.

Documentation & Markdown Metadata

Documentation pages in docs/learn/ and docs/spec/ begin with metadata headers for search indexing and portal rendering:

<!-- nytrix-doc: {"audience":"user","featured":true,"group":"learn","order":10,"summary":"One concise sentence for cards and search."} -->

• audience — Intended reader (user, contributor, internals).
• group / order — Manual grouping and display sequence.
• summary — Short factual description for card and fuzzy search indexing.
• featured — Boolean flag for curated documentation portal landing placement.

---

4. Build System & Toolchain Configuration

The canonical build interface is the ./make Python driver at repository root.

Driver Commands

./make --help          # List all available make commands
./make env             # Display resolved environment and configuration
./make doctor          # Diagnose toolchain, dependencies, and system configuration
./make targets         # List all CMake and custom targets
./make bin             # Build compiler and tool binaries (Release mode)
./make tidy            # Format code, run linter, and check header integrity
./make audit           # Run static bug audits
./make check           # Run full validation suite (build + audit + tests)

CMake Options Matrix

LLVM Discovery

• CMake detects llvm-config from version 22 down to 16 automatically.
• Override path if required:

bash

export LLVM_CONFIG=/usr/bin/llvm-config-21

export NYTRIX_LLVM_INCLUDE=/usr/lib/llvm21/include

Sanitizer Builds

Build sanitizers in dedicated debug build directories:

# AddressSanitizer + UndefinedBehaviorSanitizer
cmake -S . -B build/asan -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g"
cmake --build build/asan -j"$(nproc)"
ASAN_OPTIONS=detect_leaks=1:halt_on_error=1 \
  build/asan/ny-test --bin build/asan/ny_debug --failures-only --pattern=<fixture>

# ThreadSanitizer
cmake -S . -B build/tsan -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_FLAGS="-fsanitize=thread -fno-omit-frame-pointer -g"
cmake --build build/tsan -j"$(nproc)"

# MemorySanitizer (Clang only)
CC=clang cmake -S . -B build/msan -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_FLAGS="-fsanitize=memory -fno-omit-frame-pointer -g"
cmake --build build/msan -j"$(nproc)"

Valgrind Memory Validation

./make bin
valgrind --leak-check=full --show-leak-kinds=definite \
  --errors-for-leak-kinds=definite --error-exitcode=97 \
  build/release/ny-full path/to/reproducer.ny

---

5. Nytrix IR (NYIR) Contract & Optimization

Nytrix IR (NYIR) is the intermediate representation for SSA-based optimization and native machine lowering. The public interface is defined in src/code/native/ir.h; the verifier in src/code/native/ir/verify.c is the authoritative correctness boundary. See docs/spec/ir.md for the full specification.

Core IR Invariants

1. **Instruction Sequence & SSA Values**: A function owns a contiguous instruction array where len <= cap. Value identifiers are non-negative and strictly bounded by next_value.

2. **Single SSA Definition**: Every value is defined exactly once. Uses must refer to a dominating definition, except across block joins handled by PHIs.

3. **CFG & Block Structure**: Labels are unique, branch targets exist, and block terminators match successor edges in the CFG.

4. **PHI Ownership**: PHI nodes appear exclusively at block headers, with exactly one incoming (predecessor_label, value) pair per CFG predecessor.

5. **Deep Copy of Instruction State**: Instruction-owned payload arrays (extra_args, arg_sizes, phi_incoming) must be deep-copied and destroyed using nyir_inst_discard or nyir_erase_instruction.

6. **Raw Integer Representation**: Typed scalar integers are raw 64-bit integers (i64). Dynamic NyValue tagging is restricted to explicit lowering boundaries.

7. **Derived View Invalidation**: CFG, use-def chains, type maps, and range facts are derived data that must be updated or rebuilt after modifying instructions or edges.

Optimization Presets (Pipeline)

Strict Pass Ordering Constraints

• **CFG Simplification & DCE** run early so downstream passes operate on canonical, reachable blocks.
• **mem2reg** precedes scalar/loop analysis so SSA PHIs expose value flow to SCCP, CSE, and LICM.
• **Inlining** runs before scalar and loop optimization so newly exposed call boundaries and constants are simplified.
• **Loop Rotation** precedes SCEV and IRCE to ensure canonical headers and loop induction structure.
• **Vectorization** runs before Induction Variable (IV) elimination because vector patterns depend on canonical source induction variables and affine addresses.
• **Loop Unrolling** precedes SLP vectorization so unrolled independent operations can be packed.
• **PHI Elimination** is the final stage, executed only for backends requiring explicit local-memory form.

---

6. Native Code Generation & Executable Oracles

Native execution follows a structured pipeline:

source -> parser -> AST -> semantic analysis -> lowering -> NYIR -> machine form -> register allocation -> object/link -> execution

Verification Oracles

Native correctness is proven **only** by running the intended native execution path and verifying with executable oracles:

# Verify execution through native path with oracle assertion
./make ny --native-only --native-result-oracle test/fixture.ny

# Isolate pass failure with per-pass verification
./make ny --native-only --native-oracle-per-pass test/fixture.ny

# Inspect generated NYIR representation
./make ny --nyir-dump=/tmp/test.nyir test/fixture.ny

# Verify optimization fast paths and absence of slow fallback blocks
./make ny --jit --emit-ir=/tmp/test.ll test/fixture.ny
grep -c "fast_path_marker" /tmp/test.ll
grep -c "slow_fallback_block" /tmp/test.ll    # Must equal 0

Cross-Cutting Optimization Techniques

---

7. Performance Engineering & Benchmarking Discipline

Benchmark measurements must follow strict scientific controls:

1. **Fixed Execution Parameters**: Keep benchmark inputs, flags, cache modes, warmups, and measured runs identical across comparison targets.

2. **Comparable Baselines**: Compare Nytrix against C analogues only when both implement equivalent algorithms, data structures, and checksum verifications.

3. **Cold Runs for Definitive Proof**: Always confirm optimization gains with cold test runs:

bash

NYTRIX_TEST_COLD=1 ./make test --failures-only --color=never --pattern=<bench-name>

4. **Metric Integrity**: Separate compiler/startup wall time from timed workload execution.

5. **No False Proofs**: Cache hits, skipped engines, failed compilations, and 0µs durations are invalid as performance evidence.

---

8. Debugging & Runtime Tracing

Tracing Environment Flags

Nytrix includes targeted runtime tracing facilities:

# Call-stack execution trace
./make ny --trace test/fixture.ny

# Detailed tracing of calls, values, and VM states
NYTRIX_TRACE=1 NYTRIX_TRACE_CALLS=1 NYTRIX_TRACE_VALUES=1 ./make ny test/fixture.ny

# Trace specific module or function
NYTRIX_TRACE_FILTER=my_function_tail ./make ny test/fixture.ny

# Diagnose unresolved symbols
NYTRIX_DIAG_UNDEF=1 ./make ny test/fixture.ny

# Trace module import resolution
NYTRIX_TRACE_IMPORTS=1 ./make ny test/fixture.ny

# Bypass standard library compilation cache
NYTRIX_STD_CACHE=0 ./make ny test/fixture.ny

Systematic Issue Triage

1. **Reproduce**: Build a self-contained reproducer showing exact command, input, expected, and observed output.

2. **Isolate**: Identify the owning layer (frontend, NYIR pass, regalloc, runtime).

3. **Classify**: Categorize as bug, missing feature, unsupported syntax, or performance regression.

4. **Search**: Inspect existing tests and git history for related invariants.

5. **Fix**: Patch the owning layer and migrate affected callers.

6. **Verify**: Run the original reproducer, run cold tests, and broaden regression checks.

---

9. Standard Library & Facade Design

Module Header Convention

Every .ny source file in lib/ must begin with a structured header:

;; Keywords: text collections iteration
;; One concise summary for module cards and documentation portals.
;; References:
;; - std
;; Documentation:
;; ## Scope
;; Clear description of what this module owns.
;;
;; ## Namespaces
;; - **Core Operations:** `std.example.core`
module std.example

Source Converters

Source translators (c2ny and py2ny) live in src/cmd/fmt/init.c.

• When encountering unsupported syntax, mark explicitly with // c2ny: unsupported or # py2ny: unsupported accompanied by a clear diagnostic.
• Never silently drop unparsed source constructs.

---

10. Web & WebAssembly Testing

Web tests execute through the browser harness configured in etc/tests/native/web/tests.json:

# Run web/WASM test suite
./make web-test

# Build WebAssembly demos
./make web-demos

# Validate web contracts
./make web-check

• Maintain wasm-bare as the minimal browser runtime contract.
• Verify browser execution, WebGL2 capabilities, and virtual-time budgets under explicit headless checks.

---

11. Git Workflow & Commit Guidelines

Contribution Workflow

1. Create a feature or bugfix branch for your changes.

2. Ensure all changes are covered by focused regression tests or shapes.

3. Validate locally using ./make check and ./make tidy.

4. Open a pull request with a concise description of the problem, fix, and evidence.

Commit Rules

• Commit titles must be under **68 characters**.
• Squash related fixes together into logical, atomic commits.
• Stage explicit files only (git add <file>). Never commit build artifacts (build/), temporary files (tmp/), or compiled binaries (.so, .dll).
• Always run ./make tidy && git diff --check before committing.