# Variant RCA Report: CVE-2026-54502 (Oj stack buffer overflow via large :indent)

## Summary

This variant analysis tested whether the `:indent` stack-buffer-overflow bug in the Oj Ruby gem could be triggered through a different entry point or bypassed on the patched version. The original vulnerability (CVE-2026-54502) is in `ext/oj/dump.h::fill_indent`, which writes `indent * depth` spaces into a stack-allocated output buffer when the caller supplies a large `:indent` value. The upstream fix (`ec368db`, v3.17.2) rejects any numeric `:indent` greater than 16 in the shared option parser (`ext/oj/oj.c::parse_options_cb`).

We tested four entry points on the vulnerable, fixed, and latest commits: `Oj.dump`, `Oj::StringWriter`, `Oj::StreamWriter`, and `Oj.default_options`. All four crash on the vulnerable version and are cleanly rejected on the fixed/latest versions. No bypass was found. The alternate entry points are not distinct security variants because they share the same option-parsing path and the same trust boundary as the original trigger.

## Fix Coverage / Assumptions

The upstream fix relies on a single invariant: **every dump option, including `:indent`, is parsed by `oj_parse_options()` and its `parse_options_cb` callback before it reaches `fill_indent`.** The fix adds:

```c
#define MAX_INDENT 16
if (MAX_INDENT < FIX2INT(v)) {
    rb_raise(rb_eArgError, "indent is limited to %d characters.", MAX_INDENT);
}
```

This explicitly covers all dump APIs because they all route through `oj_parse_options()`:

- `Oj.dump` / `Oj.to_json` (`ext/oj/oj.c`)
- `Oj::StringWriter.new(options)` (`ext/oj/string_writer.c:280`)
- `Oj::StreamWriter.new(io, options)` (`ext/oj/stream_writer.c:113`)
- `Oj.default_options = { ... }` (`ext/oj/oj.c:609`)
- All `oj_dump_*` mode-specific paths (Object, Strict, Compat, Rails, Custom, WAB)

The fix does not rely on callers being well-behaved; it validates at the library boundary. It does not cover direct memory corruption or C-level exploitation, which are outside the library's threat model.

## Variant / Alternate Trigger

We evaluated the following candidate variants/bypasses:

1. **Alternate entry point: `Oj::StringWriter.new(indent: 2147483647)`**
   - Vulnerable: SIGSEGV (reaches `fill_indent` via `oj_str_writer_push_value` / `fill_indent`).
   - Fixed: `ArgumentError` rejected by `parse_options_cb`.
   - Same root cause, but same option-parsing path; not a distinct bypass.

2. **Alternate entry point: `Oj::StreamWriter.new(io, indent: 2147483647)`**
   - Vulnerable: SIGSEGV / abort (reaches `fill_indent` via `oj_str_writer_push_value`).
   - Fixed: `ArgumentError` rejected by `parse_options_cb`.
   - Same root cause, same option-parsing path; not a distinct bypass.

3. **Alternate option path: `Oj.default_options = { indent: 2147483647 }; Oj.dump({a: 1})`**
   - Vulnerable: SIGSEGV (default options propagate to `Oj.dump`).
   - Fixed: `ArgumentError` rejected when setting default options.
   - Same root cause, same option-parsing path; not a distinct bypass.

4. **Bypass attempt: negative `:indent` (`indent: -1`)**
   - Vulnerable: no crash (`fill_indent` checks `0 < out->indent` before writing).
   - Fixed: no crash.
   - Not a bypass; the guard already prevents the overflow.

5. **Bypass attempt: Bignum `:indent` (`indent: 10**40`)**
   - Vulnerable: `RangeError` from `FIX2INT` (range-checked conversion).
   - Fixed: `RangeError`.
   - Not a bypass; Ruby's conversion rejects the value before it reaches the validation.

No bypass or materially distinct variant was found. The fixed and latest versions reject the same inputs that crash the vulnerable version.

## Impact

- **Package/component affected:** `ohler55/oj` C extension, specifically the `fill_indent` helper in `ext/oj/dump.h` and the option parser in `ext/oj/oj.c`.
- **Affected versions (as tested):**
  - Vulnerable: `4587e87` (v3.17.1)
  - Fixed: `ec368db` (v3.17.2)
  - Latest: `b0677dc` (post-v3.17.2, clang-format only)
- **Risk level:** Medium. On the vulnerable version, the alternate entry points cause the same deterministic native crash as the original `Oj.dump` trigger. On fixed/latest versions, the crash is prevented.

## Impact Parity

- **Disclosed/claimed maximum impact for the parent:** Memory corruption / stack buffer overflow / crash.
- **Reproduced impact from this variant run:**
  - Vulnerable: SIGSEGV on all four dump entry points (`Oj.dump`, `Oj::StringWriter`, `Oj::StreamWriter`, `Oj.default_options` + `Oj.dump`).
  - Fixed/Latest: Clean `ArgumentError` rejection for all four.
- **Parity:** `none` for the variant-as-bypass — no bypass was reproduced. The alternate triggers on the vulnerable version achieve `full` parity with the original crash impact.
- **Not demonstrated:** Arbitrary code execution was not attempted; only the crash was reproduced.

## Root Cause

The same root cause applies to all tested entry points: `fill_indent` in `ext/oj/dump.h` performs an unbounded `memset` whose size is the product of `out->indent` and the current depth. In the vulnerable version, `out->indent` can be set to `INT_MAX` through the public `:indent` option. The fixed version enforces `MAX_INDENT` at the single option-parsing choke point, so `out->indent` can never exceed 16 when it reaches `fill_indent`.

Because every tested entry point uses the same option parser, the fix closes the sink for all of them. There is no alternate code path that populates `out->indent` without going through `parse_options_cb`.

- **Fix commit:** `ec368dbe936ef0104b782e4b0f67b17d6c7276f7`
- **Vulnerable commit:** `4587e87e23adc9a4163834dc8c9ba9d7206c6501`
- **Latest tested commit:** `b0677dccb6d3e3dc260d19e1f1c2c3913f378afc`

## Reproduction Steps

Run the variant reproduction script:

```bash
bash bundle/vuln_variant/reproduction_steps.sh
```

The script:

1. Ensures/clones the Oj repository into three separate worktrees under `bundle/artifacts/`.
2. Checks out and builds the vulnerable (`4587e87`), fixed (`ec368db`), and latest (`b0677dc`) commits.
3. Runs six test cases on each commit:
   - `Oj.dump({a: 1}, indent: 2147483647)`
   - `Oj::StringWriter.new(indent: 2147483647).push_array.push_value(1).pop_all`
   - `Oj::StreamWriter.new(StringIO.new, indent: 2147483647).push_array.push_value(1).pop_all`
   - `Oj.default_options = {indent: 2147483647}; Oj.dump({a: 1})`
   - `Oj.dump({a: 1}, indent: -1)`
   - `Oj.dump({a: 1}, indent: 10**40)`
4. Classifies each run as `segfault`, `rejected`, `ok`, or `unknown`.
5. Prints a summary and exits `0` if a bypass is detected (segfault on fixed/latest), otherwise exits `1`.

### Expected evidence

- `bundle/logs/vuln_*`: logs showing SIGSEGV / abort for the large-indent tests.
- `bundle/logs/fixed_*` and `bundle/logs/latest_*`: logs showing `ArgumentError: indent is limited to 16 characters.` or `RangeError` for the large-indent tests.
- `bundle/logs/vuln_variant_reproduction_steps.log`: complete driver output with the final `BYPASS_FOUND: false` line.

## Evidence

### Environment

- Ruby 3.3.8 (x86_64-linux-gnu)
- Oj vulnerable commit `4587e87` (VERSION 3.17.1)
- Oj fixed commit `ec368db` (VERSION 3.17.2)
- Oj latest commit `b0677dc` (post-v3.17.2)
- C extension built directly with `extconf.rb` + `make` in each checkout

### Key excerpts from `bundle/logs/vuln_variant_reproduction_steps.log`

```
vuln_dump outcome: segfault
fixed_dump outcome: rejected
latest_dump outcome: rejected
vuln_string_writer outcome: segfault
fixed_string_writer outcome: rejected
latest_string_writer outcome: rejected
vuln_stream_writer outcome: segfault
fixed_stream_writer outcome: rejected
latest_stream_writer outcome: rejected
vuln_default_options outcome: segfault
fixed_default_options outcome: rejected
latest_default_options outcome: rejected
vuln_negative_indent outcome: ok
fixed_negative_indent outcome: ok
latest_negative_indent outcome: ok
vuln_bignum_indent outcome: rejected
fixed_bignum_indent outcome: rejected
latest_bignum_indent outcome: rejected
BYPASS_FOUND: false
```

### Sample log: `bundle/logs/vuln_string_writer.log`

```
--- vuln_string_writer ---
-e:1:in `push_array': [BUG] Segmentation fault
...
```

### Sample log: `bundle/logs/fixed_string_writer.log`

```
--- fixed_string_writer ---
-e:1:in `new': indent is limited to 16 characters. (ArgumentError)
...
```

## Recommendations / Next Steps

- **No bypass was found.** The upstream fix is sufficient for the `:indent` stack overflow. The recommended action is to apply the upstream patch (`ec368db`) and upgrade to Oj 3.17.2 or later.
- **Additional hardening:** Add a defensive bounds check inside `fill_indent` itself as a fail-safe. Even if the option parser is ever bypassed in the future, `fill_indent` should refuse to write more than `out->end - out->cur` bytes.
- **Regression coverage:** Extend the existing test suite to include `Oj::StringWriter`, `Oj::StreamWriter`, and `Oj.default_options` with `indent: 2147483647`, ensuring the validation remains effective for all entry points.
- **No code change required for this variant:** Because the fix already covers all tested entry points, no additional patch is needed to address the variant search.

## Additional Notes

- **Idempotency:** The script was run twice successfully from the same state. Both runs produced identical classifications: all vulnerable large-indent tests crashed, and all fixed/latest large-indent tests were rejected.
- **No trust-boundary change:** The alternate entry points (`StringWriter`, `StreamWriter`, `default_options`) are all library APIs controlled by the same caller. They do not cross a different trust boundary, so they are not independent security variants even though they reach the same sink.
- **Negative/Bignum results:** These attempts were deliberately included to test the robustness of the validation. Negative indent is harmless because of the existing `0 < out->indent` guard; Bignum indent is rejected by Ruby's `FIX2INT` conversion. Neither constitutes a bypass.