feat(agent): make capture-session state reentrant for multi-app#4283
Merged
Conversation
Make the userspace capture pipeline able to run multiple independent recording sessions in one process, so an enterprise multi-app agent can serve several apps concurrently without cross-contamination. This is a generic reentrancy change only — no app/tenant/session concept enters OSS, and single-session behaviour is preserved byte-for-byte through explicit fallbacks. - syncMock: add New() and NewDedupQueue() so a caller can own an independent manager + dedup queue; Get()/GetDedupQueue() remain the process-global default. Add a context carrier (NewContext/FromContext/FromContextOrGlobal) and resolve the manager at the HTTP/generic/MySQL emit sites from the context, falling back to the global when none is set. - syncMock: add a per-instance test-id counter (NextTestID) and use it from conn.Capture; deprecate the package-global GlobalTestCounter. - proxy: add SetSessionResolver/GetSessionFor(tgid) so an external composer can route a connection to its owning session; a nil resolver (the OSS default) returns the single session. - proxy: delete the SrcPortToDstURL entry on connection close (after the parser errgroup joins) to bound the map and stop a recycled source port from reading a previous connection's stale TLS destination. - mockdb: add a per-instance MockFormat so spec.mockFormat is honoured per session instead of via the process-global default. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
Add GetSessionFor and SetSessionResolver to the agent.Proxy interface so external composers can install per-TGID session routing through the interface, not just the concrete type. The single-session default is unchanged (nil resolver returns the one session). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
Extend the reentrancy seam so a per-session manager also carries its own dedup queue and an optional static-deduplication hook, keeping multi-app callers' dedup state fully isolated. Single-session behaviour is unchanged: the package-global manager falls back to the global dedup queue, and the static-deduper context value is simply unset. - SyncMockManager now owns a dedupQueue (New() allocates one); DedupQueue() returns it, or the package-global globalDedupQueue for the default instance, so callers can switch from GetDedupQueue() to mgr.DedupQueue() for per-session dedup ordering. - Add the StaticDeduper interface plus WithStaticDeduper / StaticDeduperFromContext, a context carrier defined here so a per-app static deduper can ride the parser context without the proxy and the consuming hook importing each other. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
Add unit tests for the per-session seam additions: New() managers own an independent dedup queue while the global instance falls back to the global queue, and the StaticDeduper context carrier round-trips (nil when unset). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
EmitMock (the V2 mock-emit path used by all supervisor-based parsers) routed unconditionally through the package-global syncMock.Get(), so a multi-app caller could not isolate V2-parser mocks per app the way the legacy mgr.AddMock parsers now can. - supervisor.Session gains an optional Mgr field; EmitMock prefers it and falls back to the package-global when nil (single-session default — no behaviour change). - recordViaSupervisor sets Mgr from the parser context (syncMock.FromContext(ctx)), so when a multi-app caller carries a per-app manager on ctx, every V2 parser emits into that app's manager via the one EmitMock chokepoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
The pause decision stays global (pod-level cgroup memory), but the buffered mocks that consume that memory can live in many sync-mock managers: the multi-app agent runs one manager per app and the package-global Get() manager is then unused, so SetMemoryPressure on it relieves nothing. Add RegisterPressureHook so a composer can register a fan-out over its live managers; applyPausedState invokes the global manager and every registered hook. Single-app behaviour is unchanged (no hook registered → only the global manager, as before). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
…ibution Reentrancy seam for the enterprise DaemonSet agent's per-pod test-case attribution. TestCase gains a SourcePod field tagged json/yaml/bson "-", so it is never persisted to stored test-case files or serialized into the upload body — it is purely an in-memory routing tag. Capture stamps it from the context (WithSourcePod / SourcePodFromContext) for both the HTTP and gRPC capture paths. OSS single-app callers never set the context value, so SourcePod stays empty and behaviour is unchanged. The enterprise reader sets it per connection from the owning pod so the uploader can carry a per-pod source to the control plane. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
The sys_enter_socket tracepoint auto-registers any process calling socket() that passes the in-eBPF namespace check into the shared target_namespace_pids map. On a hostPID+hostNetwork DaemonSet agent that check matches host processes (init/PID 1, node daemons, short-lived forks), polluting the map so the proxyless capture records non-target host traffic — the test-set-0 leak. In DaemonSet mode the CRD-scoped SessionReconciler is the sole owner of target_namespace_pids and arms exactly the recorded pods' TGIDs, so the tracepoint's auto-detection is redundant. Gate it off when KEPLOY_DAEMONSET_ENABLED=true. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
Resolve syncMock.go by unioning the per-session dedup fields (testCounter, dedupQueue) with main's memory-pressure accounting fields (pressureDropped, totalAdded, outChanClosedDrops, pressureRanges). Both are additive struct fields; no logic overlap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ayush Sharma <kshitij3160@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
… recycle handleConnection registers `defer SrcPortToDstURL.Delete(sourcePort)`, but that defer runs (LIFO) AFTER the defer that calls srcConn.Close(), which releases the client source port at the OS level. The kernel can then recycle that port to a new connection that Stores its own dst mapping before the older connection's Delete fires — so the older connection clobbers the new one's fresh entry, and the new connection's no-SNI ClientHello fails to recover its destination. Track per-port ownership (port -> connection token) and delete only via CompareAndDelete on the token: whichever connection currently owns the port is the only one allowed to delete its mapping. Correct under any interleaving and touches no readers or the stored value type. Adds a regression test that fails under the old unconditional Delete. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
pressureHooks was an append-only slice with no way to remove a hook, so a multi-app composer that registers one hook per app/session would leak: every closed-over SyncMockManager (and its buffers) stays pinned for the life of the process and is re-invoked on every pressure transition. The 'register once' contract was convention-only with no enforcement mechanism. RegisterPressureHook now returns an idempotent unregister func (hooks keyed by token in a map). Callers that ignore the return value still compile, so this is backward compatible. Adds a test covering register/fire/unregister/no-leak and the nil-hook no-op. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
…s consumer-driven
Two seam-honesty fixes on the multi-app reentrancy surface:
- New()'s doc claimed it gives the manager 'its own output channel', but it
wires none — a New() manager whose owner forgets SetOutputChannel buffers
every mock and silently emits nothing. Correct the doc and emit a one-time
warning the first time a mock is buffered while outChan was never wired.
- The per-app dedup carrier (mgr.DedupQueue()) and the WithStaticDeduper context
seam have zero OSS consumers; m.dedupQueue is read only by its own getter. The
comments overstated this ('per-session dedup for free'). Reword to state the
isolation is opt-in and only materializes when the multi-app consumer threads
DedupQueue() into ResolveJob. Add a test pinning the getter contract (private
per New() instance; global fallback for the package manager and nil receiver)
so a refactor that drops the per-instance queue fails CI.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the changes that are made
Makes the userspace capture pipeline able to run multiple independent recording sessions in one process, so an enterprise multi-app agent can serve several apps concurrently without cross-contamination. This is a generic reentrancy change only — no app/tenant/session concept enters OSS, and single-session behaviour is preserved byte-for-byte via explicit fallbacks.
New()+NewDedupQueue()give a caller its own manager + dedup queue;Get()/GetDedupQueue()stay the process-global default. New context carrier (NewContext/FromContext/FromContextOrGlobal); the HTTP/generic/MySQL emit sites resolve the manager from the context and fall back to the global when none is set.NextTestID);conn.Captureuses it; package-globalGlobalTestCounterdeprecated.SetSessionResolver/GetSessionFor(tgid)let an external composer route a connection to its owning session; a nil resolver (the OSS default) returns the single session.SrcPortToDstURLon connection close (after the parser errgroup joins) to bound the map and stop a recycled source port from reading a previous connection's stale TLS destination.MockFormatsospec.mockFormatis honoured per session instead of via the process-global default.All changes keep the existing single-session path identical; the new seams are inert unless a caller opts in.
Links & References
Closes: NA
🔗 Related PRs
🐞 Related Issues
📄 Related Documents
What type of PR is this? (check all applicable)
Added e2e test pipeline?
Added comments for hard-to-understand areas?
Added to documentation?
Are there any sample code or steps to test the changes?
New unit tests assert: independent
New()managers/queues share no state;FromContextOrGlobalresolves per-session vs global;GetSessionForroutes by TGID and falls back to the single session;NextTestIDnumbers per-instance; per-instanceMockFormatselects gob vs structured independently.Self Review done?
Any relevant screenshots, recordings or logs?