Author: Ivan Oparin
Date: April 2026
Stack: llama-server + open-responses-server (patched) + Codex CLI v0.118.0
Hardware: Apple Silicon (tested on M3 Max 128 GB; principles generally apply across M-series, with different memory and throughput limits)

Version scope: This guide reflects the state of the ecosystem as of April 2026. Tool versions, API compatibility, and model support change frequently. Specific observations are tied to: Codex CLI v0.118.0, llama.cpp build b8670+, Ollama v0.13.3+, and LM Studio v0.3.39. Verify behavior on your build before drawing conclusions.

1. Which Backend Should You Choose?
2. Architecture Overview
3. Setup Guide
4. Verification
5. Configuration Reference
6. Troubleshooting
7. Why This Stack Needs a Shim
8. Benchmarks
9. Known Limitations
10. Appendix A: LM Studio Observations
11. Appendix B: Other Models
12. Appendix C: Implementation Notes
13. Appendix D: ORS Patch List
14. Appendix E: Web Search via ORS
15. Appendix F: Verifying Reasoning Effort
16. References

gpt-oss (20B and 120B) is OpenAI's open-weight reasoning model. Running it locally with Codex CLI on Apple Silicon requires a backend that supports the Responses API — Codex CLI v0.118.0 removed the chat wire API.

Why the choice matters for long sessions: For short-to-medium sessions, Ollama is a reliable low-friction option and llama-server + ORS is the most controllable. LM Studio is improving quickly but remains more build-dependent for tool-calling reliability (see Appendix A). For long multi-turn tool-calling (5+ turns), gpt-oss depends heavily on CoT passback — reasoning content from previous turns must be sent back to the model, or tool-calling quality degrades significantly. In the configurations evaluated here, llama-server + ORS is the only setup with directly verified CoT passback behavior. See Section 7 for the technical details.

Why use llama-server + ORS over Ollama?

• CoT passback: Verified reasoning preservation across tool-call turns. Ollama's behavior here is undocumented.
• Inference tuning: llama-server exposes batch sizes (-ub, -b), context size, flash attention, prompt caching, and slot management.
• Debugging: Detailed per-request timing logs (prompt eval, token generation, KV cache hits).

The working solution uses a three-layer stack. Codex speaks Responses, llama-server provides the most mature low-level inference control on Apple Silicon, and ORS bridges the protocol gap.

┌─────────────────────────────────────────────┐
│         Codex CLI v0.118.0                  │
│         wire_api = "responses"              │
│         POST /v1/responses                  │
└──────────────────┬──────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────┐
│    open-responses-server (patched)          │
│    Port 8081                                │
│    Translates /v1/responses ↔               │
│    /v1/chat/completions                     │
│    Handles:                                 │
│    - SSE event lifecycle                    │
│    - History reconstruction from input items│
│    - CoT (reasoning_content) passback       │
│    - function_call / function_call_output   │
│      input item parsing                     │
└──────────────────┬──────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────┐
│         llama-server (llama.cpp)            │
│         Port 8080                           │
│         --jinja (Harmony template)          │
│         /v1/chat/completions                │
│         Handles:                            │
│         - Harmony format rendering          │
│         - <|call|> / <|return|> stop tokens │
│         - reasoning_content extraction      │
│         - Prompt caching (KV cache reuse)   │
│         - Metal GPU acceleration            │
└─────────────────────────────────────────────┘

brew install ollama
ollama pull gpt-oss:20b
codex --oss -m gpt-oss:20b

This is the lowest-friction path — no config files, no shim layer, no extra terminals. However, as of April 2026, Ollama's gpt-oss support has limited observability (no access to per-request inference logs), partially contradictory documentation around reasoning handling, and undocumented CoT passback behavior. For quick experiments and short sessions this may be sufficient, but for reliable gpt-oss tool-calling workflows this guide recommends the llama-server + ORS stack below.

• macOS with Apple Silicon (M1 or later)
• Xcode Command Line Tools (xcode-select --install)
• CMake (brew install cmake)
• Python 3.12 (see Step 2)
• Node.js 22+ (for Codex CLI)

Why build from source? llama.cpp's gpt-oss and Gemma 4 support is actively evolving — critical fixes (parser improvements, template updates, EOG token handling) land in master regularly. Homebrew packages lag behind and may not include patches needed for correct tool-calling behavior. Build from the latest master for the most reliable experience.

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_METAL=ON
cmake --build build --config Release -j$(sysctl -n hw.logicalcpu)

After building, binaries are in build/bin/. Run llama-server and llama-bench directly from there:

./build/bin/llama-server -hf ggml-org/gpt-oss-20b-GGUF --ctx-size 0 --jinja -ub 2048 -b 2048

Optionally add build/bin/ to your PATH, or create symlinks. When a new fix lands in master, git pull && cmake --build build --config Release -j$(sysctl -n hw.logicalcpu) is enough to update.

The examples below assume llama-server and llama-bench are on your PATH. If not, replace them with ./build/bin/llama-server and ./build/bin/llama-bench.

Note: This is a patched fork with 9 fixes for Codex CLI compatibility and CoT passback. These changes have not yet been accepted upstream — two MRs are pending review (see Appendix D for the full patch list).

git clone https://github.com/relux-works/open-responses-server.git
cd open-responses-server
git checkout fix/codex-cli-compat

Install Python 3.12 using your preferred version manager. Python 3.12 is the tested version — earlier versions may lack required dependencies, and 3.14 fails to build pydantic-core (no prebuilt wheel). Stick to 3.12.

uv python install 3.12

npm install -g @openai/codex

Create or edit ~/.codex/config.toml:

[model_providers.llamacpp]
name = "llama.cpp"
base_url = "http://127.0.0.1:8081"   # ORS port (not llama-server's 8080)
api_key = "sk-local"
wire_api = "responses"

# gpt-oss-20b — low effort, almost no reasoning (10-30 tokens). Speed testing only.
[profiles.gpt-oss-local-low]
model_provider = "llamacpp"
model = "gpt-oss-20b-mxfp4"
model_context_window = 131072
model_reasoning_effort = "low"

# gpt-oss-20b — medium effort, recommended minimum (50-200 reasoning tokens per call)
[profiles.gpt-oss-local-med]
model_provider = "llamacpp"
model = "gpt-oss-20b-mxfp4"
model_context_window = 131072
model_reasoning_effort = "medium"

# gpt-oss-20b — high effort, thorough reasoning (200-1500 reasoning tokens per call)
[profiles.gpt-oss-local-high]
model_provider = "llamacpp"
model = "gpt-oss-20b-mxfp4"
model_context_window = 131072
model_reasoning_effort = "high"

# gpt-oss-120b (requires 96+ GB memory)
[profiles.gpt-oss-120b-med]
model_provider = "llamacpp"
model = "gpt-oss-120b-mxfp4"
model_context_window = 131072
model_reasoning_effort = "medium"

Port layout: Codex CLI connects to ORS on port 8081. ORS internally forwards to llama-server on port 8080. Codex CLI never talks to llama-server directly.

Terminal 1 — llama-server (gpt-oss-20b):

llama-server -hf ggml-org/gpt-oss-20b-GGUF \
  --ctx-size 0 --jinja -ub 2048 -b 2048

The model will be downloaded automatically on first run (~11.3 GB).

--ctx-size 0 sets context to the model's maximum (131072 tokens for gpt-oss-20b). Other key defaults that apply automatically on Apple Silicon and do not need to be set explicitly: Flash Attention (-fa auto, logs will show Flash Attention was auto, set to enabled), full GPU offloading (-ngl defaults to 99, all layers on Metal), and memory mapping (--mmap enabled by default). The -ub 2048 -b 2048 batch sizes are recommended by the llama.cpp team for Apple Silicon.

Why ggml-org GGUF models? The ggml-org HuggingFace organization publishes official quantizations from the llama.cpp team — guaranteed correct metadata, chat templates, special tokens, and tokenizer config. Alternatives (bartowski, unsloth) also work but may have subtle incompatibilities (e.g. mismatched add_bos_token between models can silently break speculative decoding).

Terminal 2 — open-responses-server:

cd /path/to/open-responses-server

API_ADAPTER_PORT=8081 \
OPENAI_BASE_URL_INTERNAL=http://127.0.0.1:8080 \
OPENAI_BASE_URL=http://127.0.0.1:8081 \
OPENAI_API_KEY=sk-local \
uv run --python 3.12 src/open_responses_server/cli.py start

OPENAI_BASE_URL_INTERNAL must not include /v1. ORS appends the path internally. If you write http://127.0.0.1:8080/v1, requests will go to /v1/v1/chat/completions and llama-server will return 404.

Terminal 3 — Codex CLI:

cd /your/project
codex --profile gpt-oss-local-med

After starting the stack, verify each layer:

llama-server logs — confirm Harmony stop tokens:

EOG token = 200002 '<|return|>'
EOG token = 200012 '<|call|>'

open-responses-server — confirm server started:

Uvicorn running on http://0.0.0.0:8081

Codex CLI header — confirm model and effort:

model:     gpt-oss-20b-mxfp4 medium

1. llama-server logs: n_tokens should increase between tool call turns (history is growing)
2. open-responses-server logs: reasoning_content should appear in streaming chunks
3. Codex CLI: Tool calls should appear as "Explored" items, final response should render
4. No repeated tool-call loops: Model should not repeat the same tool call for the same input (e.g. ls -R on the same directory)

Flash Attention is automatically enabled on Apple Silicon (M1+). llama-server logs will show Flash Attention was auto, set to enabled. FA accelerates prompt processing and reduces KV cache memory usage. It does not affect generation speed — that bottleneck is memory bandwidth.

model_context_window does not control the actual context size — that is set by llama-server's --ctx-size flag. This value tells Codex CLI how large the context is so it can correctly display the context fill indicator and trigger auto-compaction at the right threshold. It must match the --ctx-size value llama-server was started with (or the model's default if --ctx-size 0).

For slow local models, the patched ORS improves slow-stream robustness with SSE heartbeat events, but Codex CLI can still hit its own idle timeout during long prompt processing. The default (300s) is tuned for fast cloud models, not local inference. For dense models, increase on the client side:

[model_providers.llamacpp]
stream_idle_timeout_ms = 900000  # 15 minutes

Most of the issues below are resolved by building llama-server from current master and using the patched ORS (fix/codex-cli-compat branch) as described in the Setup Guide. This section documents specific error messages and their causes for reference when debugging.

Symptom: llama-server returns 404. Fix: Remove /v1 from OPENAI_BASE_URL_INTERNAL — see the note in Step 5 of the Setup Guide.

Symptom: couldn't bind HTTP server socket, port: 8080

Fix: Kill existing process or use different port:

pkill llama-server
# or
llama-server ... --port 8082

Symptom: Failed to build pydantic-core — requires Rust/maturin

Cause: No prebuilt wheel for Python 3.14. Use Python 3.12 as described in Setup Guide Step 2.

Symptom: System becomes unresponsive or llama-server crashes

Fix: Reduce context size:

llama-server ... --ctx-size 32768  # instead of 0 (full context)

These errors occur when using unpatched ORS or connecting Codex CLI directly to llama-server without ORS.

Symptom: Codex CLI shows error after streaming starts

Cause: Unpatched ORS SSE events don't match the lifecycle Codex CLI expects — missing output_item.added, output_item.done, wrong status values, or incomplete text output sequence.

Symptom: llama-server logs show successful tool call generation, but Codex CLI shows empty response

Cause: SSE event sequence doesn't match OpenAI Responses API spec. The patched ORS fixes all 9 event issues (see Appendix D).

Symptom: Model calls ls -R . repeatedly, reasoning says "Let's list files" every time

Cause: Conversation history not accumulating (input items not fully parsed), or CoT not passed back (model forgets prior decisions). Both are fixed in the patched ORS.

Symptom: Codex CLI disconnects with idle timeout waiting for SSE on large/dense models after several minutes

Cause: Two independent timeouts fire during long prompt processing:

1. Codex CLI idle timeout (default 300s) — counts only parsed SSE events; SSE comments do not reset the timer
2. ORS backend timeout (STREAM_TIMEOUT, default 120s) — httpx drops the connection

Fix (both required for dense models):

Client side — ~/.codex/config.toml:

[model_providers.llamacpp]
stream_idle_timeout_ms = 900000  # 15 minutes

Server side — ORS startup:

STREAM_TIMEOUT=600 ... uv run --python 3.12 src/open_responses_server/cli.py start

Symptom: Connecting Codex CLI directly to llama-server (without ORS) fails with:

{"error":{"code":400,"message":"'type' of tool must be 'function'","type":"invalid_request_error"}}

Cause: Codex CLI sends apply_patch as type: "custom" and web_search as type: "web_search". llama-server rejects non-function tool types.

Fix: Use ORS as the shim layer. This is one of the concrete reasons the shim is still needed.

Three issues prevent direct Codex CLI → llama-server connections today. ORS (open-responses-server) bridges all three.

gpt-oss relies on reasoning continuity across tool-call turns. The reasoning_content from a tool-call response must be sent back in the next request — without it, the model re-derives its reasoning from scratch. Experimentally verified impact: F1 score drops from 1.0 to ~0.3 by turn 5 without CoT passback (ref: aldehir's research).

ORS accumulates reasoning_content from streaming, saves it alongside each tool-call response, and reinjects it into subsequent requests. llama-server does not do this, Ollama does not document verified CoT preservation for this workflow, and LM Studio's tool-call streaming did not function correctly in the tested version (see Appendix A).

Codex CLI speaks the Responses API (/v1/responses), which differs from Chat Completions in input format (items vs messages), output format (strict SSE lifecycle), and tool-call structure. llama-server has a native /v1/responses endpoint, but it has open compatibility issues:

• Rejects non-function tool types (type: "custom", type: "web_search")
• Incomplete SSE event lifecycle for Codex CLI
• No CoT passback between turns

The patched ORS translates between the two protocols, fixing 9 specific SSE event issues (see Appendix D).

The shim becomes unnecessary when these gaps close:

1. Tool type rejection — llama-server accepts non-function tool types, or Codex CLI stops sending them
2. SSE event lifecycle — llama-server's /v1/responses produces the full event sequence Codex CLI expects
3. CoT passback — implemented server-side in llama-server, or client-side in Codex CLI

The shim is an engineering bridge, not a permanent requirement. As llama-server's Responses support matures, the need for it will diminish.

Ollama stateful Responses (PR #15404): Adds previous_response_id with in-memory ResponseStore. However, api.Message does not include reasoning_content — reasoning context would be lost during store/restore. See Appendix C for details.

Note: The benchmarks below are the author's measurements on M3 Max 128 GB and have not been independently reproduced. OpenAI publishes a compatibility test harness for objective verification.

Inference speed (from llama-server logs during Codex CLI sessions):

Long chain test (36 tool calls):

This test was run on a private repository (a small Codex CLI skill for YouTrack API access — a handful of Python scripts and supporting files). No public reproduction repo exists yet. Use this as a reference for what to expect on a small codebase, and test on your own repositories.

Prompt: "Read every Python file in scripts/ and tests/, find all functions that call the YouTrack API, and create a markdown table."

Result: 36 inference calls, 35 tool calls, 267.6s total. Model correctly identified 17 API endpoint mappings across 8 files. No repeated tool-call loops, no redundant scans.

In these measurements, MoE models delivered the best latency/throughput trade-off on Apple Silicon — gpt-oss-20b (MoE, ~87 tok/s) was comparable to Gemma 4 26B-A4B (~76 tok/s), while Gemma 4 31B dense was impractical at 9.4 tok/s. See Appendix B for the full model comparison table and speculative decoding results.

1. Non-function tool types: Codex CLI sends apply_patch as type: "custom" and web_search as type: "web_search". ORS skips these during tool conversion. apply_patch still works because Codex handles it client-side. web_search is not functional — but could be implemented as an ORS-level interceptor (see Appendix E).

2. Parallel tool calls: gpt-oss does not support parallel tool calling. Each response contains at most one tool call.

3. Reasoning effort consistency: The reasoning_effort parameter is passed through to llama-server via chat_template_kwargs. In the tested setup this works correctly — low produces 10–30 reasoning tokens, high produces 200–1500. However, behavior depends on llama-server's template implementation. Always verify your setup by running the same prompt at different effort levels and comparing reasoning token counts in llama-server logs (see Appendix F).

4. No server-side session state: Codex CLI does not use previous_response_id. Instead, it sends the full conversation history as input items on every request. ORS reconstructs history from these items request-by-request — it does not maintain persistent server-side sessions. Context grows with each turn, and very long sessions may hit the context window limit.

5. Performance vs cloud: Local inference on M3 Max (~87–90 tok/s generation for gpt-oss-20b) is respectable for local hardware but has not been benchmarked directly against OpenAI's cloud servers. Expect higher latency on prompt processing for long contexts compared to cloud, but generation speed is workable for agentic tasks.

6. Developer role handling: ORS converts developer role input items to system messages. This is the same pragmatic trade-off that LM Studio makes — it works in practice but is not a semantically perfect preservation of the Harmony role hierarchy.

7. ORS maturity: open-responses-server is still experimental. State is in-memory and non-durable, and its CoT handling is a practical chat-completions-style bridge rather than the canonical long-term Responses reasoning model.

8. No independent verification: The benchmarks and tool-call chain results in this guide are the author's measurements and have not been independently reproduced.

Use LM Studio only if you specifically want an MLX-native path and are prepared to validate your build against your workload.

The issues below were observed on LM Studio v0.3.39 (March 2026). LM Studio has been actively updating its /v1/responses support. Current versions may have addressed some or all of them.

Test configuration: Reasoning effort and model settings are controlled through Codex CLI's ~/.codex/config.toml, not through LM Studio UI. The profiles used during testing:

[model_providers.lmstudio]
name = "LM Studio"
base_url = "http://127.0.0.1:1234"
wire_api = "responses"

[profiles.oss]
model = "openai/gpt-oss-20b"
model_provider = "lmstudio"
model_reasoning_effort = "high"

[profiles.osslow]
model = "openai/gpt-oss-20b"
model_provider = "lmstudio"
model_reasoning_effort = "low"

The model ID (openai/gpt-oss-20b) must match the model loaded in LM Studio — verify with curl http://localhost:1234/v1/models.

Codex CLI end-to-end comparison (same prompt, same model):

Tool calls were silently dropped:

[WARN] No tool call streamer found for callId '1299435274720034'; ignoring arg delta
[WARN] Failed generating function tool request 'apply_patch' due to an invalid tool name

Developer role was replaced:

[WARN] Developer role detected - replacing it with system role

LM Studio's docs confirm this is a known design choice, not a bug.

Reasoning effort showed minimal effect (controlled via model_reasoning_effort in Codex CLI profiles above):

The 17% difference was smaller than expected.

KV cache reuse was not observed: Each turn started prompt processing from 0%.

The patched ORS can front other llama-server models beyond gpt-oss, though tool-calling quality remains model- and template-dependent. CoT passback benefits any model that emits reasoning_content; it is harmless for models that don't.

Gemma 4 has its own interleaved thinking requirement: "thoughts must NOT be removed between function calls" (Gemma 4 Prompt Formatting Guide). A specialized interleaved template was added in llama.cpp PR #21418.

Important: PR #21418 also added a dedicated Gemma 4 parser. Verify your llama.cpp version is b8665 or later.

curl -o gemma4-interleaved.jinja \
  https://raw.githubusercontent.com/ggml-org/llama.cpp/master/models/templates/google-gemma-4-31B-it-interleaved.jinja

llama-server -hf ggml-org/gemma-4-26b-a4b-it-GGUF \
  --ctx-size 0 --jinja \
  --chat-template-file gemma4-interleaved.jinja \
  -ub 2048 -b 2048

[profiles.gemma4]
model_provider = "llamacpp"
model = "gemma-4-26B-A4B-it-Q4_K_M"
model_context_window = 262144
model_reasoning_effort = "medium"

The "E" stands for "effective" parameters (per Google's model card) — these models use Per-Layer Embeddings (PLE) where large embedding lookup tables inflate total parameter count, but only a fraction is used for computation. They also feature shared KV layers and variable FFN widths. Fast on Apple Silicon, useful as standalone lightweight models.

llama-server -hf ggml-org/gemma-4-E2B-it-GGUF --ctx-size 0 --jinja -ub 2048 -b 2048
llama-server -hf ggml-org/gemma-4-E4B-it-GGUF --ctx-size 0 --jinja -ub 2048 -b 2048

All 31B parameters active on every token. Significantly slower than 26B-A4B (9.4 vs 75.5 tok/s on tg128). Use --ctx-size 65536 to reduce KV cache from ~25 GB to ~6 GB. Increase timeouts for Codex CLI (stream_idle_timeout_ms = 900000) and ORS (STREAM_TIMEOUT=600).

llama-server -hf ggml-org/gemma-4-31b-it-GGUF \
  --ctx-size 65536 --jinja \
  --chat-template-file gemma4-interleaved.jinja \
  -ub 2048 -b 2048

All measurements from llama-bench with -fa 1 (explicit for reproducibility), build b8670/b8796. Values in tokens per second.

llama-bench -hf ggml-org/<model>-GGUF -p 2048,8192 -n 128 -fa 1

*gpt-oss-20b numbers from llama-server logs, not llama-bench.

Tested with Gemma 4 31B (target) + E2B (draft) on M3 Max. Result: 0% acceptance rate — 0 accepted out of 4848 drafted tokens. Effective speed dropped to 2.86 tok/s (worse than baseline 9.4 tok/s).

Community data corroborates this: even with ~56% acceptance rate on Llama 70B (M3 Max), total speedup was only 2%. One plausible explanation is that Metal verification overhead cancels the savings for traditional autoregressive speculative decoding via llama.cpp. However, alternative approaches like DFlash (block diffusion-based speculative decoding) report strong results on Apple Silicon through MLX — 85 tok/s on Qwen3.5-9B (M5 Max, 3.3x speedup). DFlash was not tested here as its tooling is still immature and does not integrate with the llama-server pipeline used in this guide.

--no-mmproj is required for speculative decoding with models that include a vision encoder.

Universal: ORS as /v1/responses shim, SSE event fixes, input item parsing, --jinja template rendering.

Benefits from CoT passback: gpt-oss (required by Harmony spec), Gemma 4 (recommended by Google), any model emitting reasoning_content.

gpt-oss specific: Harmony template, reasoning_effort parameter.

Gemma 4 specific: Interleaved template (not in GGUF, must be passed via --chat-template-file), dedicated parser (PR #21418, llama.cpp b8665+).

Ollama PR #15404 adds previous_response_id support with an in-memory ResponseStore. It stores input/output messages, walks the chain to rebuild history, handles TTL expiry (30 min default), max capacity (1024), and circular reference detection.

However, StoredResponse stores InputMessages and OutputMessages as []api.Message. The api.Message structure does not include a reasoning_content field — reasoning context would be lost during store/restore. When merged, Ollama's status changes from "non-stateful Responses" to "stateful Responses without reasoning_content preservation." For most models and short sessions this is sufficient; for gpt-oss on long tool-call chains, the CoT passback gap remains.

PR #20393 — rework gpt-oss parser (merged): Tightened Harmony grammar, fixed response_format enforcement, removed invalid parallel-call logic. ggml-org/llama.cpp#20393

PR #20289 — --skip-chat-parsing (merged): Allows apps to bypass the autoparser. Fixed a parsing regression breaking gpt-oss. ggml-org/llama.cpp#20289

PR #21418 — Gemma 4 parser + interleaved template (merged): Added dedicated Gemma 4 parser and <|tool_response> as EOG token. ggml-org/llama.cpp#21418

• #21087 — WebUI drops role: tool messages: ggml-org/llama.cpp#21087
• #20650 — 500 errors during tool-calling flows: ggml-org/llama.cpp#20650

┌──────────────────────────────────────────────┐
│  llama-server (inference layer)              │
│  ✅ Harmony template rendering               │
│  ✅ reasoning_content extraction             │
│  ✅ KV cache reuse / prompt caching          │
│  ✅ Metal GPU acceleration                   │
│  ⚠️ /v1/responses (exists, Codex compat WIP) │
│  ❌ Rejects non-function tool types          │
│  ❌ CoT passback between turns               │
│  ❌ Complete SSE event lifecycle for Codex   │
└──────────────────────────────────────────────┘

┌──────────────────────────────────────────────┐
│  open-responses-server (protocol layer)      │
│  ✅ /v1/responses → /v1/chat/completions     │
│  ✅ SSE event sequence (9 fixes)             │
│  ✅ Input items parsing                      │
│  ✅ Filters non-function tool types          │
│  ✅ CoT (reasoning_content) passback         │
│  ⚠️ developer → system conversion            │
└──────────────────────────────────────────────┘

Source:

• Patched fork: https://github.com/relux-works/open-responses-server/tree/fix/codex-cli-compat
• Upstream MR 1 (patches 1–9): teabranch/open-responses-server#63
• Upstream MR 2 (slow stream reliability): teabranch/open-responses-server#64

Codex CLI sends web_search as a built-in tool (type: "web_search"). On OpenAI's servers, this is handled server-side. When running locally, nobody handles it: Codex CLI doesn't execute it client-side (unlike apply_patch), llama-server doesn't know what it is, and ORS currently drops it during tool conversion.

apply_patch works locally because Codex CLI processes it entirely client-side — it patches files itself and never sends it to the backend. web_search is different: Codex CLI expects the server to execute the search and return results. With local backends, there is no server-side search implementation.

ORS already intercepts the tool-call flow between Codex CLI and llama-server. A web search interceptor would work as follows:

1. Convert type: "web_search" to a standard function tool with a description (e.g. "search the web for the given query and return results") before forwarding to llama-server
2. Forward to llama-server as a regular function tool — the model sees it alongside other tools and can decide to call it
3. Intercept the function call when it comes back — instead of forwarding to Codex CLI, ORS executes the search itself
4. Execute the search against a configurable backend (SearXNG, Brave Search API, Tavily, or DuckDuckGo)
5. Return results as a tool response, injected back into the conversation before the next llama-server request

This is architecturally clean — ORS already rewrites tool calls and manages conversation history. The search interceptor would be a new middleware in the same pipeline.

This is a feature, not a patch — estimated at several days of implementation. The main design decisions are: which search backend to use (self-hosted SearXNG for privacy, or a commercial API for quality), how to format results for the model, and whether to support search configuration via environment variables.

After setting up the stack, verify that model_reasoning_effort actually affects model behavior. Run the same prompt with different profiles and compare reasoning token output.

Step 1: Send a simple prompt at different effort levels:

# Terminal 3 — run with medium effort
codex --profile gpt-oss-local-med
# > what is this project about?

# Then restart with high effort
codex --profile gpt-oss-local-high
# > what is this project about?

Step 2: Check llama-server logs (Terminal 1) for each request. Look for the eval time line:

prompt eval time =   5808.73 ms /  7993 tokens
       eval time =    787.62 ms /    69 tokens    ← total generated tokens (reasoning + response)

The eval time line reports total generated tokens, not pure reasoning tokens. Use it as a coarse proxy: high should materially increase generated output relative to medium or low. To confirm reasoning specifically, inspect reasoning_content in ORS logs (Step 3).

Step 3: In ORS logs (Terminal 2), look for reasoning_content in streaming output. This is the definitive check. With low, reasoning may be nearly empty. With medium or high, you should see substantive analysis text.

If reasoning token count does not change between effort levels, check that:

• llama-server was built from current master (older builds may not support chat_template_kwargs)
• The GGUF contains a Harmony template that reads the reasoning_effort parameter (ggml-org GGUFs include this)

• gpt-oss model page: https://openai.com/index/introducing-gpt-oss/
• Harmony format spec: https://developers.openai.com/cookbook/articles/openai-harmony
• OpenAI Responses API: https://developers.openai.com/api/reference#responses-streaming
• CoT handling guide: https://developers.openai.com/cookbook/articles/gpt-oss/cot-handling
• Raw CoT handling: https://developers.openai.com/cookbook/articles/gpt-oss/handle-raw-cot
• Provider verification guide: https://developers.openai.com/cookbook/articles/gpt-oss/verifying-implementations
• Run gpt-oss locally with Ollama: https://developers.openai.com/cookbook/articles/gpt-oss/run-locally-ollama
• Run gpt-oss locally with LM Studio: https://developers.openai.com/cookbook/articles/gpt-oss/run-locally-lmstudio
• Codex CLI configuration: https://developers.openai.com/codex/config-reference
• Codex CLI reference: https://developers.openai.com/codex/cli/reference

• OpenAI compatibility docs: https://docs.ollama.com/api/openai-compatibility
• PR #15404 — previous_response_id: ollama/ollama#15404

• Official gpt-oss guide: ggml-org/llama.cpp#15396
• Responses API feature request: ggml-org/llama.cpp#19138
• Responses API Codex issues: ggml-org/llama.cpp#20607
• Gemma 4 interleaved template: https://github.com/ggml-org/llama.cpp/blob/master/models/templates/google-gemma-4-31B-it-interleaved.jinja
• Gemma 4 Prompt Formatting Guide (function calling + thinking): https://ai.google.dev/gemma/docs/core/prompt-formatting-gemma4

• Responses API docs: https://lmstudio.ai/docs/developer/openai-compat/responses

• CoT passback experiments (aldehir): https://aldehir.com/posts/gpt-oss-tool-calling/
• Open Responses specification: https://www.openresponses.org/specification
• open-responses-server (patched fork): https://github.com/relux-works/open-responses-server/tree/fix/codex-cli-compat
• DFlash MLX: https://github.com/Aryagm/dflash-mlx
• Apple Silicon performance discussion: ggml-org/llama.cpp#4167