Scrapy seems to use logging inside.

You can have a look: https://docs.python.org/3/library/logging.handlers.html#timedrotatingfilehandler

Also, you could use an external unix tool to directly rotate the logs files without having to make changes on how Scrapy works with logs: https://www.tutorialspoint.com/unix_commands/logrotate.htm

If that's possible to do using stdlib logging module, it would be nice to document how. Pull requests are welcome!

I am working on this, will raise a pull request

Is it worked on? If not, I'm going to work on it.

@Roman519 Go for it! 😃

I've shoe horned a TimedRotatingFileHandler into root logger by way of log.configure_logging function. Do I need to add unit tests for something as simple as this. I'm asking because the default root filehandler doesn't have a unit test that I can see.

We are trying for new patches to provide complete coverage of their changes, so that we eventually reach that sweet 100% test coverage. So unless the tests are really difficult to write, it would be great to have them.

Is this being worked on? Looks interesting, would like to contribute if theres not a pr I missed :D

There is no pull request at the moment, I believe.

@Gallaecio in my scrapy projects i configure log handlers in settings.py like this:

from logging.handlers import TimedRotatingFileHandler
from scrapy.utils.log import configure_logging

logHandler = TimedRotatingFileHandler('crawl.log', when='midnight', interval=1)
configure_logging(install_root_handler=False)
logging.basicConfig(
    format='%(asctime)s [%(name)s] %(levelname)s: %(message)s',
    level=logging.INFO,
    handlers=[logHandler])

Is this right approach?

It does look like the recommended approach in the documentation.

@Gallaecio is there pull request for this enhancement? I am interested to contribute.

No pull request at the moment.

@kmike @Gallaecio, could you check the modification proposed in #4465? I think this will provide the requested enhancement.
If it looks suitable, I will go-ahead and prepare the documentation and tests accordingly.