# Mycel Fine-Tuning Datasets (v0.1)

Downloadable instruction-tuning, error-correction, and translation datasets for fine-tuning local open-weights language models (Llama, Qwen, Gemma, Mistral) on the Mycel programming language.

Every program in these datasets was executed and verified against the real Mycel compiler and runtime (`mycel` v0.1.0 on Node 24). There are no hand-written or fabricated outputs.

## Files & Checksums

| File | Rows | Size | SHA-256 Checksum | Description |
|---|---|---|---|---|
| [`mycel-instruct.jsonl`](./mycel-instruct.jsonl) | 364 | 127.6 KB | `dd69584b8ded604d3660a2fbd8f2c5b9da6fc399971a9daa1446fd956245be56` | Task instructions and verified Mycel programs covering beginner to app-level code. |
| [`mycel-fix.jsonl`](./mycel-fix.jsonl) | 76 | 52.7 KB | `23c178b1b1e138e7bbb4b85153e3467b361e6fa18a536c05bb2d7f4918482d29` | Broken code with real `mycel check --json` compiler diagnostics and verified fixes. |
| [`mycel-translate.jsonl`](./mycel-translate.jsonl) | 40 | 28.1 KB | `9218ce0c10cd35e36daca275a53db4609e2de846255a0baa533b9adf0a773a21` | Python programs and equivalent Mycel programs where both run and outputs match exactly. |

## How It Was Made & Verified

1. **mycel-instruct.jsonl**:
   - Covers 20+ language domains: primitives, 64-bit Int division (`/` vs `.div()`), strings, f-strings (`{x:.2f}`, `{x=}`), boolean logic, while/for loops, ranges, arrays/lists, comprehensions, maps, sets, tuples (`.0`, `.1`), functions (variadic, named args, defaults, closures), classes & inheritance, structs with `.with(...)`, enums with payload and exhaustive `match`, interfaces, extensions (`extend Int`), optionals (`T?`, `nil`, `??`, `!=`), Result error propagation (`?`), and standard libraries (`std.json`, `std.math`, `std.time`, `std.crypto`, `std.re`, `std.env`, `std.fs`).
   - Every program was executed using `lang/bin/mycel run --allow-all`. Only programs that exited with code 0 were retained, and their actual standard output is stored in `meta.output`. Programs reading interactive input have their inputs recorded in `meta.inputs`.

2. **mycel-fix.jsonl**:
   - Covers newcomer habits from Python, JavaScript, TypeScript, Java, Go, Rust, and Swift (from `lang/src/habits.ts`), plus common type errors, non-exhaustive matches, immutable let reassignments, and syntax mistakes.
   - Broken programs were verified by running `lang/bin/mycel check --json` to capture the real compiler diagnostic object (including `code`, `message`, `why`, `suggestion`).
   - Fixed programs were verified by compiling and running with `lang/bin/mycel run`.

3. **mycel-translate.jsonl**:
   - Seeded from `lang/compare/` and expanded with real-world algorithms (binary search, bubble sort, gcd, Caesar cipher, prime factors, Pascal's triangle, matrix operations, RLE, etc.).
   - Both the Python code (via CPython 3) and the Mycel code (via `mycel run`) were executed in isolated subshells; rows were kept only when both standard outputs matched character-for-character.

## Format Specification

Each row is a single-line JSON object adhering to the standard multi-turn chat format:

```json
{
  "messages": [
    { "role": "user", "content": "Write a Mycel program that computes the hypotenuse..." },
    { "role": "assistant", "content": "import std.math\nprint(math.hypot(3.0, 4.0))" }
  ],
  "meta": {
    "output": "5.0"
  }
}
```

Compatible with:
- **MLX LoRA** (`mlx_lm.lora`) on Apple Silicon
- **HuggingFace TRL** (SFTTrainer / dataset chat template)
- **Unsloth** / **Axolotl** / **Llama-Factory**
- **OpenAI fine-tuning JSONL format**

## Fine-Tuning Guide (Quickstart)

### Using MLX on Apple Silicon

Split `mycel-instruct.jsonl` into `train.jsonl` and `valid.jsonl` (e.g. 90/10 split):

```sh
# Shuffle and split
shuf mycel-instruct.jsonl > shuffled.jsonl
head -n 327 shuffled.jsonl > train.jsonl
tail -n 37 shuffled.jsonl > valid.jsonl

# Run LoRA training on Qwen2.5 or Gemma (4-bit)
python -m mlx_lm.lora \
  --model mlx-community/Qwen2.5-Coder-7B-Instruct-4bit \
  --train \
  --data . \
  --iters 600 \
  --batch-size 4 \
  --lora-layers 16
```

## Honest Limits

- **Language Stage**: Mycel is v0.1.0 in active development. Syntax and stdlib APIs reflect the November 2026 specification.
- **Single-File Scope**: The training rows represent single-file programs and modules. Multi-package dependency graphs (`ply.json` inter-package dependencies) are not covered in this dataset.
- **Foreign Ecosystem Interop**: Interop with native Python/npm packages is planned for Phase B and Phase C and is not included in this training set.

## License & Provenance

Licensed under the **Apache License, Version 2.0**.
Copyright (c) 2026 drky (same copyright and license as [github.com/drkfade/mycel-editors](https://github.com/drkfade/mycel-editors)).
Signed provenance: https://ol1x.xyz/provenance/