How to use Python to login to a webpage and retrieve cookies for later usage?

Each Answer to this Q is separated by one/two green lines.

I want to download and parse webpage using python, but to access it I need a couple of cookies set. Therefore I need to login over https to the webpage first. The login moment involves sending two POST params (username, password) to /login.php. During the login request I want to retrieve the cookies from the response header and store them so I can use them in the request to download the webpage /data.php.

How would I do this in python (preferably 2.6)? If possible I only want to use builtin modules.

Here’s a version using the excellent requests library:

from requests import session

payload = {
    'action': 'login',
    'username': USERNAME,
    'password': PASSWORD

with session() as c:'', data=payload)
    response = c.get('')

import urllib, urllib2, cookielib

password = 'mypassword'

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
login_data = urllib.urlencode({'username' : username, 'j_password' : password})'', login_data)
resp ='')
print is the straight html of the page you want to open, and you can use opener to view any page using your session cookie.

The answers/resolutions are collected from stackoverflow, are licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0 .

Leave a Reply

Your email address will not be published.