# Patch Analysis: CVE-2026-54502 (Oj stack buffer overflow via large :indent)

## Vulnerability Summary

The Oj Ruby gem (`ohler55/oj`) is vulnerable to a stack-based buffer overflow in `ext/oj/dump.h::fill_indent`. When a caller passes a large `:indent` option (e.g., `2147483647`) to `Oj.dump`, the native code computes `cnt *= out->indent` and calls `memset(out->cur, ' ', cnt)` without validating that the destination buffer can hold the requested bytes. The output buffer is only a few kilobytes, so a 2 GB write corrupts the stack and crashes the Ruby interpreter with SIGSEGV.

- **Vulnerable commit:** `4587e87e23adc9a4163834dc8c9ba9d7206c6501` (parent of the fix, matches v3.17.1)
- **Fix commit:** `ec368dbe936ef0104b782e4b0f67b17d6c7276f7` ("Fix stack limits (#1014)", released as v3.17.2)
- **Latest tested commit:** `b0677dccb6d3e3dc260d19e1f1c2c3913f378afc` (post-fix, only a clang-format change)

## What the Fix Changes

The fix is concentrated in `ext/oj/oj.c`:

```c
#define MAX_INDENT 16

// In parse_options_cb(), when handling the :indent option:
case T_FIXNUM:
    copts->dump_opts.indent_size = 0;
    *copts->dump_opts.indent_str = '\0';
    if (MAX_INDENT < FIX2INT(v)) {
        rb_raise(rb_eArgError, "indent is limited to %d characters.", MAX_INDENT);
    }
    copts->indent = FIX2INT(v);
    break;
```

The patch adds a hard upper bound of `16` on the numeric `:indent` value at the option-parsing layer (`parse_options_cb`), before the value ever reaches `fill_indent`. The string-based `:indent` path already limits the input to `sizeof(copts->dump_opts.indent_str)` (16 bytes including null), so it remains safe and is unaffected by this change.

The same commit also fixes several other "extreme size" issues (key length limits, parser stack-depth checks, regex match length typing, etc.), but those are separate sinks and do not change the `:indent` validation logic.

## Assumptions the Fix Makes

1. **Single option-parsing path:** The fix assumes that every code path that can set the dump `indent` goes through `oj_parse_options()` and its `parse_options_cb` callback. The `Out` struct field `out->indent` is always copied from `copts->indent` or `sw->opts->indent`, which are populated by this callback.
2. **No direct mutation of the C struct from Ruby:** The fix assumes attackers cannot directly set `Options.indent` or `Out.indent` from Ruby without passing through the option parser.
3. **MAX_INDENT is safe:** The fix assumes that an `indent` of at most 16 combined with the existing `MAX_DEPTH` of 1000 cannot produce a `memset` or `assure_size` request large enough to overflow or corrupt the buffer. The maximum indentation bytes requested by `fill_indent` would be `16 * 1000 = 16000`, which is small compared to the output buffer size.

## Code Paths Covered by the Fix

All of the following entry points call `oj_parse_options()` and therefore inherit the `:indent` validation:

- `Oj.dump` / `Oj.to_json` (`ext/oj/oj.c` dump functions)
- `Oj::StringWriter.new(options)` (`ext/oj/string_writer.c`)
- `Oj::StreamWriter.new(io, options)` (`ext/oj/stream_writer.c`)
- `Oj.default_options = { ... }` (`ext/oj/oj.c`)
- `Oj.load` / `Oj.parse` / `Oj.saj_parse` / `Oj::Doc` parsing paths (for parser options, though they do not reach `fill_indent`)
- `Oj.mimic_JSON` and `Oj.optimize_rails` flows that use the same option parser for dump formatting

All direct assignments to `out->indent` in the source are traceable to options that have passed through `parse_options_cb`:

```c
// ext/oj/rails.c:916
out.indent = copts.indent;

// ext/oj/string_writer.c:69, 284
sw->out.indent = sw->opts.indent;

// ext/oj/stream_writer.c:119
sw->sw.out.indent = sw->sw.opts.indent;
```

## What the Fix Does NOT Cover (and Why It Does Not Matter Here)

- **Negative `:indent` values:** The fix does not reject negative integers. However, `fill_indent` already has a guard `if (0 < out->indent)` that prevents any write when `indent` is zero or negative. Our tests confirm that `indent: -1` is harmless on both vulnerable and fixed versions.
- **Very large Bignum `:indent`:** Ruby's `FIX2INT` range-checks the value before converting it to a C `int` and raises a `RangeError` if it does not fit. Our tests confirm that `indent: 10**40` is rejected on both vulnerable and fixed versions.
- **Direct C extension exploitation:** The fix does not protect against an attacker who can already execute arbitrary C code or corrupt memory directly; that is outside the library's threat model.
- **Separate extreme-size bugs:** The same commit fixed other bugs (e.g., very long keys in the parser), but those are distinct sinks and are not bypasses of the `:indent` overflow fix.

## Comparison of Behavior Before and After the Fix

| Input | Vulnerable v3.17.1 | Fixed v3.17.2 / latest |
|-------|--------------------|------------------------|
| `Oj.dump({a:1}, indent: 2147483647)` | SIGSEGV (stack overflow) | `ArgumentError: indent is limited to 16 characters.` |
| `Oj::StringWriter.new(indent: 2147483647).push_array.push_value(1).pop_all` | SIGSEGV | `ArgumentError` |
| `Oj::StreamWriter.new(io, indent: 2147483647).push_array.push_value(1).pop_all` | SIGSEGV / abort | `ArgumentError` |
| `Oj.default_options = {indent: 2147483647}; Oj.dump({a:1})` | SIGSEGV | `ArgumentError` |
| `Oj.dump({a:1}, indent: -1)` | No crash (indent ignored) | No crash (indent ignored) |
| `Oj.dump({a:1}, indent: 10**40)` | `RangeError` (out of int range) | `RangeError` |

## Is the Fix Complete?

For the specific `:indent` stack-buffer-overflow sink, **the fix is complete** within the library's stated threat model. The validation is placed at the single choke point (`parse_options_cb`) that every documented dump option must pass through. We found no alternate path that could set `Out.indent` to a value larger than 16 without first being rejected by the parser.

## Implications for Variant Search

Because the fix is applied at the option parser, a true bypass would need to find:

1. A way to set `Options.indent` or `Out.indent` without going through `oj_parse_options()`, or
2. A way to coerce `parse_options_cb` into accepting a value larger than 16 (e.g., type confusion, integer truncation).

Neither path exists in the tested code:
- All Ruby-facing dump APIs route through `oj_parse_options()`.
- `FIX2INT` rejects out-of-range values rather than truncating them.
- The T_STRING/T_NIL branches either set `indent = 0` or raise on oversized strings.

Therefore, the variant search produced **no bypass**. The alternate entry points we tested (`Oj::StringWriter`, `Oj::StreamWriter`, `Oj.default_options`) do trigger the same sink on the vulnerable version, but they are the same trust boundary and the same option-parsing path, so they are not distinct security variants. The upstream fix covers them all.

## Target Threat Model / Security Policy Notes

The repository's `SECURITY.md` is a generic template and does not explicitly exclude local-file or library-API misuse. It states that supported versions receive security updates. The relevant security boundary here is "Ruby caller supplies options to the Oj dump API." The fix correctly enforces that boundary by rejecting unbounded `:indent` values regardless of which dump API receives them.

## Recommendations for Hardening

Although the fix is complete for the reported sink, the following additional hardening would be worthwhile:

1. **Defensive check in `fill_indent`:** Add a runtime assertion or bounds check inside `fill_indent` so that even if a future change bypasses `parse_options_cb`, the function refuses to write more than the buffer can hold. This would convert a potential stack overflow into a controlled error.
2. **Document `MAX_INDENT`:** Expose the limit in the public documentation so users know why large indent values are rejected.
3. **Regression tests:** Add tests covering `Oj::StringWriter`, `Oj::StreamWriter`, and `Oj.default_options` with `indent: 2147483647` to ensure the validation stays in place for all entry points.