Skip to content

Add support for scheduled queries to run at startup#8554

Merged
zwass merged 7 commits into
osquery:masterfrom
Micah-Kolide:micah/allow_scheduled_queries_to_run_at_startup
Mar 25, 2025
Merged

Add support for scheduled queries to run at startup#8554
zwass merged 7 commits into
osquery:masterfrom
Micah-Kolide:micah/allow_scheduled_queries_to_run_at_startup

Conversation

@Micah-Kolide

@Micah-Kolide Micah-Kolide commented Feb 21, 2025

Copy link
Copy Markdown
Contributor

This PR adds a new field for scheduled queries startup_priority which if set to a non-default value will run queries in ascending order once osqueryd starts the SchedulerRunner.

I've tested this a fair amount on MacOS, and I've attached time table of my tests showing when osqueryd initialized and when it executed the defined scheduled queries.
Time Table.csv

Here are some logs also showing the example change:

I0219 17:35:35.824323 107137088 dispatcher.cpp:78] Adding new service: SchedulerRunner (0x60000121c558) to thread: 0x17039f000 (0x6000029293c0) in process 25893
I0219 17:35:35.824326 1881698304 eventfactory.cpp:410] Event publisher fsevents run loop terminated for reason: Publisher disabled via configuration
I0219 17:35:35.824347 1882845184 scheduler.cpp:120] Executing scheduled query test_startup_priority_0_int_400: select * from processes
I0219 17:35:35.890591 1882845184 scheduler.cpp:201] Found results for query: test_startup_priority_0_int_400
I0219 17:35:35.890978 1882845184 scheduler.cpp:120] Executing scheduled query test_startup_priority_1_int_120: select * from processes
I0219 17:35:35.942234 1882845184 scheduler.cpp:201] Found results for query: test_startup_priority_1_int_120
I0219 17:35:35.942560 1882845184 scheduler.cpp:120] Executing scheduled query test_startup_priority_1_int_300_1: select * from processes
I0219 17:35:35.990871 1882845184 scheduler.cpp:201] Found results for query: test_startup_priority_1_int_300_1
I0219 17:35:35.991300 1882845184 scheduler.cpp:120] Executing scheduled query test_startup_priority_1_int_300_2: select * from processes
I0219 17:35:36.050416 1882845184 scheduler.cpp:201] Found results for query: test_startup_priority_1_int_300_2
I0219 17:35:36.050863 1882845184 scheduler.cpp:120] Executing scheduled query test_startup_priority_2_int_30_1: select * from processes
I0219 17:35:36.099247 1882845184 scheduler.cpp:201] Found results for query: test_startup_priority_2_int_30_1
...
I0219 17:36:05.949177 1882845184 scheduler.cpp:120] Executing scheduled query test_startup_priority_max_int_60: select * from processes
I0219 17:36:06.005627 1882845184 scheduler.cpp:201] Found results for query: test_startup_priority_max_int_60

@Micah-Kolide Micah-Kolide requested review from a team as code owners February 21, 2025 19:55
@Micah-Kolide Micah-Kolide force-pushed the micah/allow_scheduled_queries_to_run_at_startup branch from f0d188e to cb8c10d Compare February 21, 2025 20:08
zwass
zwass approved these changes Mar 11, 2025
View reviewed changes

@zwass zwass left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved assuming no issue with my comment. Please take a look at that.

Comment thread osquery/dispatcher/scheduler.cpp
Comment on lines +285 to +294
Config::get().scheduledQueries(([&i, &first_query_runs](
const std::string& name,
const ScheduledQuery& query) {
bool query_has_not_run =
first_query_runs.find(name) == first_query_runs.end();
if ((query.splayed_interval > 0 && i % query.splayed_interval == 0) ||
(query.startup_priority != UINT64_MAX && query_has_not_run)) {
if (query_has_not_run) {
first_query_runs.insert(name);
}

@zwass zwass Mar 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is name here going to be the full name prefixed by the pack name? If not, is it possible that there's a bug in which two queries with the same name in different packs cause a collision in this code?

@Micah-Kolide Micah-Kolide Mar 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name here is the full name with the prefix as well.

osquery/osquery/config/config.cpp

Lines 481 to 488 in 9e3535a

void Config::scheduledQueries(
std::function<void(std::string name, const ScheduledQuery& query)>
predicate,
bool denylisted) const {
RecursiveLock lock(config_schedule_mutex_);
for (PackRef& pack : *schedule_) {
for (auto& it : pack->getSchedule()) {
std::string name = getQueryName(pack->getName(), it.first);

osquery/osquery/config/config.cpp

Lines 472 to 479 in 9e3535a

std::string getQueryName(const std::string& packName, const std::string& name) {
// The query name may be synthetic.
if (packName != "main") {
return "pack" + FLAGS_pack_delimiter + packName + FLAGS_pack_delimiter +
name;
}
return name;
}

@Micah-Kolide Micah-Kolide Mar 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A pack structure similar to:

"packs": {
  "test_pack_2": {
    "queries": {
      "test_startup_2": {...}
    }
  }
}

outputs: pack_test_pack_2_test_startup_2

@zwass zwass Mar 25, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay great! Thank you :)

@zwass zwass merged commit 60342ca into osquery:master Mar 25, 2025
@zwass zwass mentioned this pull request Jul 29, 2025
Open
@directionless directionless mentioned this pull request Mar 9, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zwass zwass zwass approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Micah-Kolide @zwass