Skip to content

fix(topology): reattach fanout on cross-type producer key swap#25725

Open
pront wants to merge 2 commits into
masterfrom
fix/enrichment-table-source-key-reattach
Open

fix(topology): reattach fanout on cross-type producer key swap#25725
pront wants to merge 2 commits into
masterfrom
fix/enrichment-table-source-key-reattach

Conversation

@pront

@pront pront commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary

Fixes a topology reload bug where downstream components whose input key stayed the same across reload would silently stop receiving events if the upstream producer changed kind (e.g. an enrichment-table-derived source at key X replaced by a regular source at key X, or a transform at X replaced by a source at X).

get_changed_outputs previously only iterated to_change sets, missing cross-type respawns. It now also considers keys that appear in (sources.to_remove ∪ transforms.to_remove ∪ enrichment_tables.to_remove) intersected with (sources.to_add ∪ transforms.to_add ∪ enrichment_tables.to_add).

Vector configuration

Reproducer config A (pre-reload):

sources:
  in:
    type: demo_logs
    format: json
    interval: 1

enrichment_tables:
  memory_table:
    type: memory
    ttl: 60
    inputs: ["in"]
    source_config:
      source_key: "shared"
      export_interval: 2

sinks:
  observer:
    type: console
    inputs: ["shared"]
    encoding:
      codec: json

Reloaded config B:

sources:
  shared:
    type: demo_logs
    format: json
    interval: 1

sinks:
  observer:
    type: console
    inputs: ["shared"]
    encoding:
      codec: json

How did you test this PR?

Manual end-to-end reproduction with ./target/debug/vector --config live.yaml, swapping A -> B via SIGHUP:

  • On master: observer receives 0 events after reload (bug confirmed).
  • On this branch: observer continues receiving demo_logs events after reload.

Also ran cargo check with the topology feature set. A unit-test-only regression could not be reliably reproduced in the current test harness (the reattach path fires but events don't flow via basic_source/basic_sink after reload — a separate test-infra issue), so the branch relies on the manual reproducer above. Happy to add a test if a reviewer knows a working scaffold.

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Changelog fragment added.
  • No. A maintainer will apply the no-changelog label to this PR.

References

@github-actions github-actions Bot added the domain: topology Anything related to Vector's topology code label Jul 1, 2026
pront added 2 commits July 1, 2026 10:43
get_changed_outputs only iterated to_change sets, missing keys that
disappear as one kind of source and reappear as another (e.g. an
enrichment-table-derived source removed while a regular source with
the same key is added). Downstream components whose input stays the
same would silently stop receiving events after such a reload.

Extend the set to also include cross-type respawns:
(sources.to_remove | enrichment_tables.to_remove) intersected with
(sources.to_add | enrichment_tables.to_add). Enrichment-table sink
keys can appear in enrichment_tables.* but never match
output_ids.component, so their inclusion in the union is harmless.

Note: this addresses the codex-flagged reattach gap by inspection.
A minimal isolated repro of the end-to-end symptom (unchanged sink
loses events post-reload) was attempted but hit other reload-path
issues that need separate investigation.
@pront pront force-pushed the fix/enrichment-table-source-key-reattach branch from 6b7e070 to 037d160 Compare July 1, 2026 14:44
@pront pront marked this pull request as ready for review July 1, 2026 15:16
@pront pront requested a review from a team as a code owner July 1, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: topology Anything related to Vector's topology code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant