Python - Web crawling exercises

Hello Python developer!

How are you? Hope you've been good and had wonderful time with Python.

There are 10 Python requests module exercises.

Please read and type and go for better understanding of requests module.

Preparation : Python requests module installation

$ pip install requests 

Exercise 1. importing requests module

import requests as rq

Exercise 2. Getting web page data with URL

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

rq.get(url)

rq.post(url) 

Exercise 3. Getting Web server response code

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

Exercise 4. Getting Web server status code(Available page)

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

print(res.status_code)

Exercise 5. Getting Web server status code(Unavailable web page)

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today//errors"

res = rq.get(url)

print(res)

print(res.status_code)

Exercise 6. Conditional cases by web server status coderror code

import requests as rq

def url_check(url):

    res = rq.get(url)

   

    print(res)

   

    sc = res.status_code

   

    if sc == 200:

        print("%s Request succeeded." %(url))

    elif sc == 404:

        print("%s page was not found." %(url))

    else :

        print("%s had unknown error" %(url))

url_check("https://github.com/Cevastian/Python-Developer-start-today")

url_check("https://github.com/Cevastian/Python-Developer-start-today//a")

Exercise 7. Getting web page header data

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

print(res.headers)

Exercise 8. Getting web page set cookies 

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

headers = res.headers

print(headers['Set-Cookie'])

Exercise 9. Getting all web page header data

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

headers = res.headers

for header in headers:

    print(headers[header])

Exercise 10. Getting cookies by cookies attribute

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

cookies = res.cookies

print(cookies)

Hope these requests code exercises are useful for you.

See you next topics and exercises! Have a nice day!