New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow SQL warnings to be reported. #46690
Allow SQL warnings to be reported. #46690
Conversation
7051422
to
66d3b34
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few comments but generally LGTM.
Not too sure about the naming of the two configuration options, and whether we should scope them under MySQL or not. Do other DBs even have warnings? @matthewd since you're the PG expert in my eyes
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Outdated
Show resolved
Hide resolved
|
The Postgres equivalent is https://github.com/ged/ruby-pg/blob/1dd1039983b0f5c15e3cb23e3bf6475dc0b9ee6a/ext/pg_connection.c#L2786 It looks like SQLite has a similar mechanism in https://www.sqlite.org/errlog.html, though that's perhaps too limited for us by being process-global (and, AFAICS, not currently exposed by the gem?) |
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Outdated
Show resolved
Hide resolved
This makes sense to me -- maybe let's start with passing the DB name to the proc for now, unless we see some other use case where having the full pool config would be necessary? |
You mean connection name? Because the DB name would change between environments. I think passing a rich object is better than just a name. That is assuming the object is public. |
Yes, sorry I meant the db config names
Yeah, I guess it makes more sense to pass an object here rather than a magic string with the name |
|
|
0071e93
to
07f799f
Compare
|
Still need to implement Postgres support, feel free to hold off on reviewing until that's up. I'll re-request reviews when it's ready |
07f799f
to
8e0200d
Compare
b65192b
to
81f501a
Compare
|
I talked about this with Adrianna today and if there are no strong objections I think we should implement this for application level only in this PR and in a followup PR figure out how we want to do it per-connection (either waiting until someone needs / requests it or ensuring it gets in the next release). |
Wasn't the idea that you could filter that from the callback using the connection that is passed as argument? Even in multi-db scenarios, 99% of the time you won't have different handling for different DBs. It's nice that you can do it, but I see no need to streamline it more than it already is. |
@eileencodes and I discussed reverting back to the original API and foregoing passing a second argument to the callback (either a connection / db config) until we were more convinced of a per-connection use case. In other words, If we're concerned about the use case Matthew presented (which seems the most viable) of ignoring warnings on secondary databases, we could introduce an |
|
Fine by me. |
2657b63
to
d6b9431
Compare
activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb
Outdated
Show resolved
Hide resolved
a7438e7
to
3e45a37
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some very minor stuff.
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Outdated
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
Outdated
Show resolved
Hide resolved
60f618e
to
5ad6136
Compare
activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
Outdated
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb
Outdated
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb
Outdated
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb
Outdated
Show resolved
Hide resolved
5ad6136
to
fe335e2
Compare
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Outdated
Show resolved
Hide resolved
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
Show resolved
Hide resolved
fe335e2
to
c1951a3
Compare
|
Appreciate the feedback, folks |
8436d48
to
e49fd9c
Compare
| ->(warning) do | ||
| warning_message = "[#{warning.class}] #{warning.message}" | ||
| warning_message += " (#{warning.code})" if warning.code | ||
| ActiveRecord::Base.logger.warn(warning_message) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will load ActiveRecord::Base when this config is set, and we don't want that. But it inside a on_load block.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think so. We're in a callback here, it won't be invoked until a query is performed and it emits warnings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, right.
| ->(warning) do | ||
| warning_message = "[#{warning.class}] #{warning.message}" | ||
| warning_message += " (#{warning.code})" if warning.code | ||
| ActiveRecord::Base.logger.warn(warning_message) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, right.
e49fd9c
to
dc4bdf8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@matthewd @jhawthorn if you don't have any other concerns I'd like to merge this. |
cbb5fe7
to
dbc5432
Compare
|
Was wondering if we should consider adding an API on top of this one to override the behavior per block. Similar to I can imagine someone wanting to perform a query despite of warnings but without adding the warning to the ignore list so they could rescue it like: def update_record(params)
ActiveRecord::Base.connection.execute("query...")
rescue SQLWarning => e
# no-op
# I know my manual query issues a warning but I'm okay with that
# however I don't want to add the warning to the ignore list as this is the only place which I expect to raise a warning. Every other part of the app should still raise
endBut this example won't work for a read query as we don't get to use the result if we raise. def update_record(params)
warnings = ActiveRecord.capture_warnings do
# we know this query issues warnings but this is a specific use-case so we want to ignore it but only here
result = ActiveRecord::Base.connection.execute("query...")
end
end |
I think this is a valid use case for sure |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One non-blocking question, otherwise looks good to me!
dbc5432
to
a64aebe
Compare
`ActiveRecord.db_warnings_action` can be used to configure the action to take when a query produces a warning. The warning can be logged, raised, or trigger custom behaviour provided via a proc. `ActiveRecord.db_warnings_ignore` allows applications to set an allowlist of SQL warnings that should always be ignored, regardless of the configured action. Co-authored-by: Paarth Madan <paarth.madan@shopify.com>
a64aebe
to
56c5685
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Thank you
Motivation / Background
At Shopify, we use https://github.com/Shopify/activerecord-pedantmysql2-adapter to raise SQL warnings as errors so they can be identified and fixed. We believe this provides value upstream, so we'd like to propose an API in Rails that provides warning reporting.
Activerecord::PedantMysql2provides a more comprehensive API, including the ability to capture / silence warnings within certain scopes. We've reduced the API to what we believe is most critical: allowing users to raise, log, or ignore warnings. Users can also specify a custom proc to be called when a warning is encountered, which provides some of the flexibility that the pedant gem was offering.Detail
We've introduced two configurations to Active Record:
config.active_record.db_warnings_actioncan be used to configure the action to take when a query produces a warning. The warning can be ignored, logged, raised, or trigger custom behaviour provided via a proc.config.active_record.db_warnings_ignoreallows applications to set an allowlist of SQL warnings that should always be ignored, regardless of the configured action.For now, this is only scoped to MySQL adapters, internally relying on MySQL's
SHOW WARNINGSstatement, and the PostgreSQL adapter, relying on#set_notice_processor.Checklist
Before submitting the PR make sure the following are checked:
[Fix #issue-number]