Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mention DOWNLOADER_CLIENT_TLS_METHOD tweaking to avoid some bans #5232

Open
Gallaecio opened this issue Aug 18, 2021 · 3 comments
Open

Mention DOWNLOADER_CLIENT_TLS_METHOD tweaking to avoid some bans #5232

Gallaecio opened this issue Aug 18, 2021 · 3 comments
Labels
docs enhancement

Comments

Assignees
No one assigned
Labels
docs enhancement
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@Gallaecio @umairnsr87 @sang721
@Gallaecio
Copy link
Member

@Gallaecio Gallaecio commented Aug 18, 2021

Changing the value of that setting has been seen to work around some bans, so it may be worth mentioning in https://docs.scrapy.org/en/master/topics/avoiding-bans.html#bypassing-web-browser-filters
@umairnsr87
Copy link
Contributor

@umairnsr87 umairnsr87 commented Aug 18, 2021

Should I update it @Gallaecio ?

Loading

@Gallaecio
Copy link
Member Author

@Gallaecio Gallaecio commented Aug 24, 2021

Thanks, but we reverted the creation of that page of the documentation for now, so this needs to wait a bit.

Loading

@sang721
Copy link

@sang721 sang721 commented Oct 19, 2021

In my case, the requests method works flawlessly and DOWNLOADER_CLIENT_TLS_METHOD also works but only for the first try. As i checked scrapy request middlewares, the request headers seems to hold a different value than what requests library holds. I wonder if that was the case ?

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment