Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upCover arbitrary filtering in the Scrapy logging documentation #4216
Comments
|
@Gallaecio, i would like to try to contribute to this issue. it is my first time contributing, would this be a good fit for me? if Yes, any pointers on where to start on this issue?. i have read the contributing to scrappy documentation. any other pointers are welcome. Thank you |
|
For this specific issue, I would recommend to:
|
|
@Gallaecio, Thank you for getting back to me. |
I think substrings should be fine, it should be trivial for users to go from that to regular expressions in needed. You could alternatively mention that something other than substrings may be used, and mention regular expressions linking to https://docs.python.org/3/library/re.html |
|
@Gallaecio, i created a pull request (#4257 ) for this. could you review and get back to me with your feedback. Thank you |
|
@Gallaecio |
|
There are no additional things, although @akamanzi may be out of time to complete his proposal. If so, you could see if you can address the issue yourself, maybe build on top of his work so far. |
|
Is this issue still open ? If Yes, then can I would like to contribute to it. I am starting my journey as a open source contributor I hope that's fine. |
|
@yash-sethia , i haven't looked at this issue for a while. currently busy with school dissertation. you can give it a try, i suggest looking at the recommendations @Gallaecio made in the pull request i initially created (#4257), review them and build on top of that. |
|
Is This still open, Can I take this up . |
It should be clear, from reading the documentation, how to filter out a specific log message that we wish to ignore.
This is specially important for warnings that depend on input, like the one introduced in #4214. Since you seldom have the power to fix the issue that triggers the warning message, caused by the content or behavior of the website you are scraping, you may need to simply ignore those warning messages.
Exposing a setting or a LogFormatter method for each of those warnings does not seem scalable to me, specially when such warnings can come from third-party Scrapy extensions.