Skip to content

/ airflow

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make warnings more visible #12204

Merged
merged 2 commits into from Nov 9, 2020
Merged

Make warnings more visible #12204

merged 2 commits into from Nov 9, 2020

Conversation

@turbaszek
Copy link
Contributor

@turbaszek turbaszek commented Nov 9, 2020

This PR proposes using custom showwarning function that provides users with better information about warnings using rich library to highlight the warning.

I think this makes harder to miss a warning especially the deprecation ones. I decided to use rich as it may be used in future for enhancing other outputs (see #11259 (comment)).

Before

Screenshot 2020-11-09 at 16 34 47

After

Screenshot 2020-11-09 at 16 34 22

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
@potiuk
potiuk reviewed Nov 9, 2020
View changes
Copy link
Member

@potiuk potiuk left a comment

I like it. I am just wondering if those color-control-characters will not show up in the logs in Airflow? I remember we had some problem with that when we introduced colors to log in general.
@turbaszek
Copy link
Contributor Author

@turbaszek turbaszek commented Nov 9, 2020

@potiuk suggested that it would be good to consult the colours with colour-blind users. In general I agree with that, however, the colour will depend on terminal settings so I'm not sure how much we can do around that.

CC: @dimberman @kaxil @TobKed @ryanahamilton
@turbaszek
Copy link
Contributor Author

@turbaszek turbaszek commented Nov 9, 2020 •
edited

I like it. I am just wondering if those color-control-characters will not show up in the logs in Airflow? I remember we had some problem with that when we introduced colors to log in general.

By default they are streamed to stderr and it look like it works as expected:

root@4cec474e47b3:/opt/airflow# airflow scheduler -D
/opt/airflow/airflow/settings.py:109 DeprecationWarning: AAAAA
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
root@4cec474e47b3:/opt/airflow# ^C
root@4cec474e47b3:/opt/airflow# cat /root/airflow/airflow-scheduler.out
[2020-11-09 16:09:51,464] {scheduler_job.py:1249} INFO - Starting the scheduler
[2020-11-09 16:09:51,465] {scheduler_job.py:1254} INFO - Processing each file at most -1 times
[2020-11-09 16:09:51,600] {dag_processing.py:250} INFO - Launched DagFileProcessorManager with pid: 2674
[2020-11-09 16:09:51,602] {scheduler_job.py:1761} INFO - Resetting orphaned tasks for active dag runs
[2020-11-09 16:09:51,659] {settings.py:51} INFO - Configured default timezone Timezone('UTC')
root@4cec474e47b3:/opt/airflow# cat /root/airflow/airflow-scheduler.err
/opt/airflow/airflow/settings.py:109 DeprecationWarning: AAAAA

The problem with control character was that we piped the logs from tty-like source to file (webserver) and that befuddled the colourful logging mechanism .
@potiuk
potiuk approved these changes Nov 9, 2020
View changes
@github-actions
Copy link

@github-actions github-actions bot commented Nov 9, 2020

The PR needs to run all tests because it modifies core of Airflow! Please rebase it to latest master or ask committer to re-run it!
@turbaszek
Make warnings more visible
Loading status checks…
c1c599e 
This PR proposes to use custom showwarning function that
provides users with better information about warnings using
rich library to highlight the warning.
@turbaszek turbaszek force-pushed the PolideaInternal:visible-warnings branch from c4acd3b to c1c599e Nov 9, 2020
@kaxil
kaxil reviewed Nov 9, 2020
View changes
setup.py Outdated Show resolved Hide resolved
@turbaszek
fixup! Make warnings more visible
Loading status checks…
7e1c34d
@turbaszek turbaszek requested a review from kaxil Nov 9, 2020
@kaxil kaxil merged commit cc12db7 into apache:master Nov 9, 2020
54 checks passed
54 checks passed
Build info
Details
Selective checks
Details
Label PRs when reviewed
Details
Test OpenAPI client generation
Details
Analyze (python)
Details
Analyze (javascript)
Details
Wait for CI images
Details
Static checks: basic checks only
Details
Wait for PROD images
Details
Static checks
Details
Pylint
Details
Build docs
Details
Spell check docs
Details
Backport packages
Details
Provider packages
Details
Python unit tests for helm chart
Details
Postgres9.6,Py3.7: Always Core Other API CLI Providers WWW Integration Heisentests
Details
Postgres13,Py3.7: Always Core Other API CLI Providers WWW Integration Heisentests
Details
Postgres9.6,Py3.8: Always Core Other API CLI Providers WWW Integration Heisentests
Details
Postgres13,Py3.8: Always Core Other API CLI Providers WWW Integration Heisentests
Details
MySQL5.7, Py3.6: Always Core Other API CLI Providers WWW Integration Heisentests
Details
MySQL8, Py3.6: Always Core Other API CLI Providers WWW Integration Heisentests
Details
MySQL5.7, Py3.8: Always Core Other API CLI Providers WWW Integration Heisentests
Details
MySQL8, Py3.8: Always Core Other API CLI Providers WWW Integration Heisentests
Details
Sqlite Py3.6: Always Core Other API CLI Providers WWW Integration Heisentests
Details
Sqlite Py3.7: Always Core Other API CLI Providers WWW Integration Heisentests
Details
Quarantined tests (mysql)
Details
Quarantined tests (postgres)
Details
Quarantined tests (sqlite)
Details
Constraints
Details
K8s 3.6 v1.18.6 image K8s 3.6 v1.18.6 image
Details
K8s 3.6 v1.17.5 image K8s 3.6 v1.17.5 image
Details
K8s 3.6 v1.16.9 image K8s 3.6 v1.16.9 image
Details
K8s 3.7 v1.18.6 image K8s 3.7 v1.18.6 image
Details
K8s 3.7 v1.17.5 image K8s 3.7 v1.17.5 image
Details
K8s 3.7 v1.16.9 image K8s 3.7 v1.16.9 image
Details
K8s 3.8 v1.18.6 image K8s 3.8 v1.18.6 image
Details
K8s 3.8 v1.17.5 image K8s 3.8 v1.17.5 image
Details
K8s 3.8 v1.16.9 image K8s 3.8 v1.16.9 image
Details
Upload coverage
Details
Push PROD images
Details
Push CI images
Details
Constraints push
Details
Tag repo nightly
Details
Selective build check Selective build check
Details
Status of image build CI: 3.6 Status of image build CI: 3.6
Details
Status of image build CI: 3.7 Status of image build CI: 3.7
Details
Status of image build CI: 3.8 Status of image build CI: 3.8
Details
Status of image build PROD: 3.6 Status of image build PROD: 3.6
Details
Status of image build PROD: 3.7 Status of image build PROD: 3.7
Details
Status of image build PROD: 3.8 Status of image build PROD: 3.8
Details
CodeQL No new alerts
Details
Mergeable Mergeable Run has been Completed!
Details
WIP Ready for review
Details
Acehaidrey pushed a commit to Acehaidrey/incubator-airflow that referenced this pull request Nov 12, 2020
@turbaszek Ace Haidrey
Make warnings more visible (apache#12204)
d44e9e5 
This PR proposes to use custom showwarning function that
provides users with better information about warnings using
rich library to highlight the warning.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk

@ryanahamilton ryanahamilton

@dimberman dimberman

@kaxil kaxil

@mik-laj mik-laj

Assignees
No one assigned
Labels
full tests needed
Projects
None yet
Milestone
No milestone
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
@turbaszek @potiuk @kaxil
You can’t perform that action at this time.