Skip to content

fix(voice-call): avoid OpenAI realtime double greeting#86285

Merged
steipete merged 19 commits into
openclaw:mainfrom
giodl73-repo:fix-85846-openai-realtime-double-greeting
Jul 5, 2026
Merged

fix(voice-call): avoid OpenAI realtime double greeting#86285
steipete merged 19 commits into
openclaw:mainfrom
giodl73-repo:fix-85846-openai-realtime-double-greeting

Conversation

@giodl73-repo

@giodl73-repo giodl73-repo commented May 25, 2026

Copy link
Copy Markdown
Contributor

Fixes #85846

What Problem This Solves

OpenAI Realtime outbound calls could greet twice when caller audio arrived before session.updated: the provider drained that audio into server VAD before the explicit queued greeting had disabled automatic response creation.

Why This Change Was Made

  • Signal provider readiness before draining pre-ready caller audio, so the existing explicit greeting path establishes its response policy first.
  • Suppress only server-VAD create_response while a manual response is active; retain interruption handling.
  • Keep suppression through a queued retry and restore it only after the final response or terminal error.
  • Correlate manual create/cancel failures with client event IDs so unrelated or stale Realtime errors cannot release suppression, and flush queued work after a correlated rejection.
  • Share the turn-detection session-update builder across GA and Azure paths, without moving greeting content into session instructions.
  • Remove redundant voice-call wrapper logic and tests now covered at the provider boundary.

No config, plugin API, or non-OpenAI realtime-provider behavior changes.

User Impact

Outbound OpenAI Realtime calls produce one intended initial greeting even when the caller speaks immediately. Later caller interruptions still cancel the active response as before.

Evidence

  • Blacksmith Testbox: 58 OpenAI provider tests and 9 Talk runtime tests passed.
  • Exact changed-surface gate passed: https://github.com/openclaw/openclaw/actions/runs/28752407321
  • Live Twilio/OpenAI outbound call completed with one greeting.
  • Direct authenticated OpenAI Realtime probe reached ready state, emitted three expected session updates, exactly one manual response.create, exactly one server response, zero errors, and returned audio plus transcript.
  • Fresh autoreview found no actionable issues.
  • Exact-head hosted CI passed at 53b9efa0a7118df9906c66999fc0580809c45baa: https://github.com/openclaw/openclaw/actions/runs/28753065715

Known proof gaps: none for the changed surface.

@openclaw-barnacle openclaw-barnacle Bot added channel: voice-call Channel integration: voice-call size: S maintainer Maintainer-authored PR labels May 25, 2026
@clawsweeper

clawsweeper Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed July 5, 2026, 4:06 PM ET / 20:06 UTC.

Summary
The PR updates the OpenAI realtime voice provider to suppress server-VAD auto responses around manual greeting/response turns, restore them on terminal/error paths, correlate create/cancel errors by event_id, and adds focused provider tests plus a changelog entry.

PR surface: Source +92, Tests +427, Docs +1. Total +520 across 3 files.

Reproducibility: yes. at source level: current main combines a ready-triggered explicit greeting with OpenAI server VAD create_response defaulting true, and the linked issue reports repeated live Twilio/OpenAI reproductions. I did not place a fresh phone call in this read-only review.

Review metrics: 1 noteworthy metric.

  • Release-owned changelog entry: 1 added. Normal PRs should keep release-note context in the PR body or squash message while release generation owns CHANGELOG.md.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #85846
Summary: This PR is the active fix candidate for the canonical OpenAI realtime outbound double-greeting issue; earlier same-bug PRs are closed unmerged.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Remove the PR-added CHANGELOG.md entry while keeping the PR-body release-note context.
  • Let the latest exact-head checks finish after the cleanup before maintainer landing.

Risk before merge

  • [P1] The PR intentionally changes OpenAI realtime response timing; if a suppress/restore edge case is wrong, outbound calls could still duplicate a greeting or suppress a later spoken response.
  • [P1] The PR still carries a release-owned CHANGELOG.md entry, so merging as-is would bypass the repository's release-generation ownership for changelog content.

Maintainer options:

  1. Remove changelog and proceed (recommended)
    Drop the release-owned CHANGELOG.md entry, keep the PR-body evidence, and let exact-head CI plus maintainer review gate the OpenAI realtime timing change.
  2. Request final-head live proof
    If maintainers want extra assurance for the final error-correlation commits, ask for a short redacted current-head Twilio/OpenAI call or authenticated Realtime probe before merge.
  3. Hold the provider timing change
    Keep the PR open if maintainers are not ready to accept a timing-sensitive OpenAI realtime message-delivery change in this release train.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Remove the CHANGELOG.md entry added by this PR; keep the PR-body evidence and release-note context intact, do not change OpenAI runtime or tests, then run git diff --check.

Next step before merge

  • [P2] A narrow automated repair can remove the PR-added changelog line without changing runtime or tests; maintainer review still owns final landing.

Security
Cleared: No concrete security or supply-chain regression was found; the diff changes provider runtime/test behavior and a changelog entry without dependencies, workflows, permissions, secrets handling, or package metadata changes.

Review findings

  • [P3] Remove the release-owned changelog edit — CHANGELOG.md:22
Review details

Best possible solution:

Merge the provider-owned fix after removing the changelog entry, preserving the PR-body release-note context, and letting exact-head CI plus maintainer review gate the timing-sensitive message-delivery behavior.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main combines a ready-triggered explicit greeting with OpenAI server VAD create_response defaulting true, and the linked issue reports repeated live Twilio/OpenAI reproductions. I did not place a fresh phone call in this read-only review.

Is this the best way to solve the issue?

Yes, with cleanup. The provider-owned suppress/restore approach is the narrowest maintainable fix because it avoids new config surface and preserves explicit greeting behavior for other realtime providers; the changelog entry should be removed before merge.

Full review comments:

  • [P3] Remove the release-owned changelog edit — CHANGELOG.md:22
    This previously raised cleanup is still present. CHANGELOG.md is release-owned by repo policy, so normal PRs should keep release-note context in the PR body or squash message and let release generation write the final entry.
    Confidence: 0.93

Overall correctness: patch is correct
Overall confidence: 0.89

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a04b6ced4f96.

Label changes

Label justifications:

  • P2: This is a normal-priority user-visible fix for duplicated spoken output in the optional OpenAI realtime voice-call path.
  • merge-risk: 🚨 message-delivery: The PR changes when OpenAI realtime auto responses are suppressed and restored, which can duplicate or suppress spoken call responses if the timing is wrong.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body reports a live Twilio/OpenAI outbound call with one greeting and a direct authenticated OpenAI Realtime probe that reached ready state, produced one manual response.create and one server response, returned audio/transcript, and had zero errors.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body reports a live Twilio/OpenAI outbound call with one greeting and a direct authenticated OpenAI Realtime probe that reached ready state, produced one manual response.create and one server response, returned audio/transcript, and had zero errors.
Evidence reviewed

PR surface:

Source +92, Tests +427, Docs +1. Total +520 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 1 117 25 +92
Tests 1 464 37 +427
Docs 1 1 0 +1
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 582 62 +520

Acceptance criteria:

  • [P1] git diff --check.
  • [P1] Verify the only repair diff is removal of the PR-added CHANGELOG.md line.
  • [P1] Verify extensions/openai/realtime-voice-provider.ts and extensions/openai/realtime-voice-provider.test.ts are unchanged by the repair.

What I checked:

  • Repository policy read and applied: Root AGENTS.md was read completely; the release-owned changelog rule applies directly to this PR's CHANGELOG.md edit. (AGENTS.md:238, a04b6ced4f96)
  • Scoped extension policy read: extensions/AGENTS.md was read completely; the OpenAI provider change stays inside the bundled plugin boundary and uses plugin SDK surfaces. (extensions/AGENTS.md:3, a04b6ced4f96)
  • No matching maintainer notes found: The checkout has no matching files under .agents/maintainer-notes for this OpenAI/voice-call review.
  • Current main source repro: Current main defaults OpenAI realtime server VAD create_response to true, so caller speech can create an automatic response. (extensions/openai/realtime-voice-provider.ts:857, a04b6ced4f96)
  • Current main explicit greeting path: Current main still calls triggerGreeting from the realtime session ready hook when initialGreetingInstructions are present. (src/talk/session-runtime.ts:147, a04b6ced4f96)
  • PR implementation: The PR sends response.create with a client event_id after suppressing auto response creation for the manual response turn, then restores auto-response when the turn settles. (extensions/openai/realtime-voice-provider.ts:1265, 53b9efa0a711)

Likely related people:

  • steipete: Git history/blame ties Peter Steinberger to the OpenAI realtime provider, voice-call handler, shared session runtime lines, PR assignment, and the final current-head error-correlation commits. (role: feature history owner and recent area contributor; confidence: high; commits: a23ab9b906dc, 286d0b9fb340, 53b9efa0a711; files: extensions/openai/realtime-voice-provider.ts, extensions/openai/realtime-voice-provider.test.ts, extensions/voice-call/src/webhook/realtime-handler.ts)
  • vincentkoc: Recent current-main history shows Vincent Koc carrying release/current-main work across the OpenAI realtime provider, voice-call handler, and shared session runtime paths. (role: recent area contributor; confidence: medium; commits: e085fa1a3ffd; files: extensions/openai/realtime-voice-provider.ts, extensions/voice-call/src/webhook/realtime-handler.ts, src/talk/session-runtime.ts)
  • omarshahine: Omar Shahine provided the public proof-gate review that shaped the later live Twilio/OpenAI proof requirement for this PR. (role: reviewer; confidence: medium; files: extensions/openai/realtime-voice-provider.ts, extensions/openai/realtime-voice-provider.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (5 earlier review cycles)
  • reviewed 2026-06-24T16:35:31.846Z sha 0641d1f :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-05T18:04:12.116Z sha a24fe0d :: needs real behavior proof before merge. :: [P3] Remove the release-owned changelog edit
  • reviewed 2026-07-05T19:13:31.089Z sha 2927dff :: needs real behavior proof before merge. :: [P3] Remove the release-owned changelog edit
  • reviewed 2026-07-05T19:20:48.929Z sha 2927dff :: needs changes before merge. :: [P3] Remove the release-owned changelog edit
  • reviewed 2026-07-05T19:52:53.689Z sha b1180fc :: needs changes before merge. :: [P3] Remove the release-owned changelog edit

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels May 25, 2026
@clawsweeper

clawsweeper Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@giodl73-repo giodl73-repo force-pushed the fix-85846-openai-realtime-double-greeting branch 3 times, most recently from 34dcae8 to 4f41219 Compare May 25, 2026 16:38

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Current-head refresh for #86285:

  • Rebases giodl73-repo:fix-85846-openai-realtime-double-greeting onto current upstream main; new head 4f4121990c8.
  • This picks up the upstream QA-lab WhatsApp type fix that was making unrelated CI lanes fail.
  • Focused WSL validation passed after rebase:
    • node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts --reporter=dot -> 1 file, 20 tests passed
    • ./node_modules/.bin/oxfmt --check extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
    • ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.core.json extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
    • git diff --check refs/remotes/upstream-safe/main...HEAD
  • The refreshed commit is SSH-signed.

No live Twilio/OpenAI outbound call proof was run in this environment.

@clawsweeper

clawsweeper Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 25, 2026
@giodl73-repo giodl73-repo force-pushed the fix-85846-openai-realtime-double-greeting branch from 4f41219 to 8ec995e Compare May 25, 2026 17:04

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Current-head refresh for #86285:

  • Rebases giodl73-repo:fix-85846-openai-realtime-double-greeting onto current upstream main; new head 8ec995ee33fc2c6e66944838c4fd686ca4d36f67.
  • Focused WSL validation passed:
    • node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts --reporter=dot -> 1 file, 20 tests passed
    • ./node_modules/.bin/oxfmt --check extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
    • ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.core.json extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
    • git diff --check refs/remotes/upstream-safe/main...HEAD
  • The refreshed commit is SSH-signed with the giodl73 key.

I still have not run the requested live Twilio/OpenAI outbound-call proof on this machine.

@clawsweeper

clawsweeper Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels May 25, 2026
@omarshahine

Copy link
Copy Markdown
Contributor

Thanks for the PR. Before this can move forward, please add live proof from the affected surface, not just unit tests, mocked tests, or source inspection.

A useful proof update should include:

  • the exact build/SHA tested
  • the real environment used
  • the command, UI flow, channel flow, provider request, or other live path exercised
  • before/after symptom evidence where applicable
  • the observed result after the patch
  • any remaining proof gaps

Please redact secrets, tokens, phone numbers, and private message content from logs or screenshots.

@giodl73-repo giodl73-repo force-pushed the fix-85846-openai-realtime-double-greeting branch from 8ec995e to 3de30fb Compare May 25, 2026 18:17

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Current-head refresh for #86285:

  • Rebases giodl73-repo:fix-85846-openai-realtime-double-greeting onto current upstream main; new head 3de30fb486939fc5f19ba81be4ca14dc53010fc5.
  • Focused WSL validation passed after rebase:
    • node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts --reporter=dot -> 1 file, 20 tests passed
    • ./node_modules/.bin/oxfmt --check extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
    • ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.core.json extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
    • git diff --check refs/remotes/upstream-safe/main...HEAD
  • The refreshed commit is SSH-signed with the giodl73 key.

I still have not run live Twilio/OpenAI outbound-call proof on this machine. No merge performed.

@clawsweeper

clawsweeper Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@giodl73-repo giodl73-repo force-pushed the fix-85846-openai-realtime-double-greeting branch from 3de30fb to 9b03cf8 Compare May 25, 2026 18:44

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Refreshed onto upstream main e844d1d6e56b31fa574b4c2970fdd6e59a9f2e83, which includes the upstream cron-service CI fix. New signed head is 9b03cf869ea.

Focused validation after rebase passed:

  • node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts --reporter=dot -> 1 file, 20 tests passed

Remaining known gap: I still do not have live Twilio/OpenAI outbound call proof on this machine. No merge performed.

@openclaw-barnacle openclaw-barnacle Bot added channel: matrix Channel integration: matrix channel: whatsapp-web Channel integration: whatsapp-web app: web-ui App: web-ui size: XL and removed size: L labels Jul 5, 2026
@steipete steipete force-pushed the fix-85846-openai-realtime-double-greeting branch from 7415e3b to 53b9efa Compare July 5, 2026 19:58
@openclaw-barnacle openclaw-barnacle Bot added size: L and removed channel: matrix Channel integration: matrix channel: whatsapp-web Channel integration: whatsapp-web app: web-ui App: web-ui size: XL labels Jul 5, 2026
@steipete

steipete commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Exact-head land-ready proof for 53b9efa0a7118df9906c66999fc0580809c45baa:

  • Maintainer rewrite keeps response-source arbitration inside the OpenAI Realtime provider and leaves other realtime providers unchanged.
  • Blacksmith Testbox: 58 OpenAI provider tests and 9 Talk runtime tests passed.
  • Exact changed-surface gate passed: https://github.com/openclaw/openclaw/actions/runs/28752407321
  • Live Twilio/OpenAI outbound call completed with one intended greeting and preserved caller interruption.
  • Direct authenticated OpenAI Realtime probe produced one manual request, one server response, audio plus transcript, and zero errors.
  • Client event correlation now protects suppression from unrelated create errors, queued-request rejection, and stale cancellation errors.
  • Fresh autoreview found no actionable issues after those repairs; confidence 0.91.
  • Exact-head hosted CI passed: https://github.com/openclaw/openclaw/actions/runs/28753065715
  • Repository OPENCLAW_TESTBOX=1 scripts/pr prepare-run 86285 passed and verified remote head equality.

Known proof gaps: none for the changed surface.

@steipete steipete merged commit 85ad4cb into openclaw:main Jul 5, 2026
105 of 110 checks passed
@steipete

steipete commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: openai maintainer Maintainer-authored PR merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: L status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[voice-call] OpenAI realtime: outbound calls greet caller twice — triggerGreeting + server_vad.create_response both fire response.create

4 participants