Skip to content

/ requests

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set-Cookie - violating rfc6265 #2576

Open
henadzit opened this issue Apr 28, 2015 · 9 comments
Open

Set-Cookie - violating rfc6265 #2576

henadzit opened this issue Apr 28, 2015 · 9 comments

Comments

@henadzit
Copy link

@henadzit henadzit commented Apr 28, 2015

Hey,

I found that requests (version 2.6.0) violates one paragraph of rfc6265 - http://tools.ietf.org/html/rfc6265#section-4.1.2.3

"If the server omits the Domain attribute, the user agent will return the cookie only to the origin server."

It means that if Set-Cookie is set without domain at test.com, the cookie shouldn't be visible at subdomain.test.com.

However, the RFC warns that some user agents behave that way. Have this been discussed before? I haven't found anything. I encountered that issue when I was requesting endpoint which sent me 301 to subdomain of the initial endpoint. The service didn't handle that properly.

Below the code that shows the issue.

"""
Use twoliner to add hosts to /etc/hosts

echo "127.0.0.1       test.com
127.0.0.1       subdomain.test.com" |  sudo tee -a /etc/hosts
"""

import BaseHTTPServer
import requests
import threading


# http server
class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def do_GET(self):
        l = self.headers.get('Host').lower()
        print "Server: processing {}".format(l)

        if l == 'test.com:8000':
            self.send_response(301)
            self.send_header('Set-Cookie', 'Test=True')
            self.send_header('Location', 'http://subdomain.test.com:8000')
        elif l == 'subdomain.test.com:8000':
            print "---> Cookie: {} <---".format(self.headers.get('Cookie'))
            self.send_response(200)
        else:
            assert False, "Not supported"


httpd = BaseHTTPServer.HTTPServer(('', 8000), RequestHandler)


def server_callback():
    while True:
        httpd.handle_request()

server_thread = threading.Thread(target=server_callback)
server_thread.daemon = True
server_thread.start()

# http client
requests.get('http://test.com:8000')

The code outputs

Server: processing test.com:8000
127.0.0.1 - - [28/Apr/2015 23:47:55] "GET / HTTP/1.1" 301 -
Server: processing subdomain.test.com:8000
---> Cookie: Test=True <---
127.0.0.1 - - [28/Apr/2015 23:47:55] "GET / HTTP/1.1" 200 -
@sigmavirus24
Copy link
Contributor

@sigmavirus24 sigmavirus24 commented Apr 29, 2015

So the code that handles cookies for us in the standard library. We rely on the add_cookie_header method on a CookieJar. We create a mock request object but as far as I can tell we're doing everything we should. If we look at that, we'll see that the add_cookie_header method calls _cookies_for_domain which calls domain_return_ok on DefaultCookiePolicy. The check explicitly says that it is a liberal check.

With that information, I think I'm comfortable saying that we should start providing a way for users to enable a stricter policy. I don't think we can start doing that by default, though, until 3.0 since it will be a breaking change. Does that sound reasonable?
@sigmavirus24
Copy link
Contributor

@sigmavirus24 sigmavirus24 commented Apr 29, 2015

Also, I wasn't previously aware of this so I'd like to sincerely thank you for bringing this to our attention.
@Lukasa
Copy link
Member

@Lukasa Lukasa commented Apr 29, 2015

@sigmavirus24 Can users not provide a stricter policy by replacing the cookiejar on the Session object?
@sigmavirus24
Copy link
Contributor

@sigmavirus24 sigmavirus24 commented Apr 29, 2015

@Lukasa they can. I'm proposing that we provide a stricter policy by default in the near future to make requests comply better with 6265
@henadzit
Copy link
Author

@henadzit henadzit commented Apr 29, 2015

With that information, I think I'm comfortable saying that we should start providing a way for users to enable a stricter policy. I don't think we can start doing that by default, though, until 3.0 since it will be a breaking change. Does that sound reasonable?

I very agree. I am not sure how many people this change could break and whether the behavior should be configurable. I'm also wondering how this is handled in requests analogues.
@Lukasa
Copy link
Member

@Lukasa Lukasa commented Apr 29, 2015

Yeah, I want to know what browsers do and what curl does. I'll test this at some stage.
@sigmavirus24
Copy link
Contributor

@sigmavirus24 sigmavirus24 commented Apr 29, 2015

I'm pretty sure we can use the script in @henadzit's original report to test browsers and curl both. It's well done.
@henadzit
Copy link
Author

@henadzit henadzit commented Apr 30, 2015

I updated the test script to test curl and wget. I also tried to use Ruby httpclient but it failed on my server answers. Probably, it requires some headers or something like that. I will look into it later. Anyway, the script is at https://github.com/henadzit/henadzit-various/blob/master/requests-show-rfc6265.py

Its output

Client: curl
Server: processing test.com:8000
127.0.0.1 - - [30/Apr/2015 01:02:01] "GET / HTTP/1.1" 301 -
Server: processing subdomain.test.com:8000
---> Cookie: None <---
127.0.0.1 - - [30/Apr/2015 01:02:01] "GET / HTTP/1.0" 200 -

Client: requests
Server: processing test.com:8000
127.0.0.1 - - [30/Apr/2015 01:02:01] "GET / HTTP/1.1" 301 -
Server: processing subdomain.test.com:8000
---> Cookie: Test=True <---
127.0.0.1 - - [30/Apr/2015 01:02:01] "GET / HTTP/1.1" 200 -

Client: wget
Server: processing test.com:8000
127.0.0.1 - - [30/Apr/2015 01:02:01] "GET / HTTP/1.1" 301 -
Server: processing subdomain.test.com:8000
---> Cookie: None <---
127.0.0.1 - - [30/Apr/2015 01:02:01] "GET / HTTP/1.1" 200 -

As you can see, both wget and curl follow the RFC and don't store the cookie.

I still want to test browser behavior.
@sigmavirus24
Copy link
Contributor

@sigmavirus24 sigmavirus24 commented Aug 8, 2015

So I've had this bug nagging at me for no particular reason. I started digging into it tonight and I found that the problem is that the standard library's cookie handling library actually always adds a domain if it's absent to a cookie. It also, helpfully (and I literally just realized this while writing this comment) adds domain_specified which is a bool. If domain isn't in the parsed cookie then that attribute will be false. This makes making our own policy really easy.
sigmavirus24 added a commit to sigmavirus24/requests that referenced this issue Aug 8, 2015
@sigmavirus24
Add a custom requests cookie policy
e318c3e 
This ensures that we follow RFC 6265 Section 4.1.2.3 appropriately. If a
cookie is returned without a domain attribute, we do not want to send it
to subdomains.

Closes psf#2576
@sigmavirus24 sigmavirus24 mentioned this issue Aug 8, 2015
Closed
0 of 4 tasks complete
sigmavirus24 added a commit to sigmavirus24/requests that referenced this issue Aug 9, 2015
@sigmavirus24
Add a custom requests cookie policy
628f1fc 
This ensures that we follow RFC 6265 Section 4.1.2.3 appropriately. If a
cookie is returned without a domain attribute, we do not want to send it
to subdomains.

Closes psf#2576
@sigmavirus24 sigmavirus24 mentioned this issue Apr 18, 2016
Closed
@tirkarthi tirkarthi mentioned this issue Dec 21, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@sigmavirus24 @henadzit @Lukasa
You can’t perform that action at this time.