Installing requests:

Install the package requests in a virtual environment (recommended):

$ pip install requests

Alternatively, you can install it for the current user (or globally):

$ python3 -m pip install --user requests

Verify the installation:

import requests
# Check installed version
print(requests.__version__)

Output:

2.31.0

HTTP Methods:

• GET requests:

  requests.get(url, headers=headers, params=params)

• POST requests:

  # FORM data
  requests.post(url, data=data)
  
  # JSON data
  requests.post(url, json=data)
  
  # Upload files
  requests.post(url, files=files)

• PUT requests:

  requests.put(url, json=data)

• PATCH requests (partial update):

  requests.patch(url, json=data)

• DELETE requests:

  requests.delete(url)

Basic GET request:

# import requests module
import requests

from requests.exceptions import HTTPError, ConnectionError, Timeout, RequestException

url = 'http://www.mtitek.com/index.htm'

# adding headers
headers = {'Accept': 'text/html'}

# adding parameters
params = {'param1': 'value1'}

try:
  # GET request
  response = requests.get(url, headers=headers, params=params)

  print(response) # Output: <Response [200]>
except HTTPError as e:
  print(f"HTTP Error: {e}")
except ConnectionError as e:
  print(f"Connection Error: {e}")
except Timeout as e:
  print(f"Timeout Error: {e}")
except RequestException as e:
  print(f"Request Error: {e}")

Response object properties:

import requests

response = requests.get('http://www.mtitek.com/index.htm')

# response status
print(f"status code: {response.status_code}") # Output: status code: 200
print(f"reason: {response.reason}") # Output: reason: OK

# response headers
print(f"headers keys: {list(dict(response.headers))}") # Output: headers keys: ['Date', 'Server', 'X-Powered-By', 'Connection', 'Transfer-Encoding', 'Content-Type']
print(f"headers keys/values: {dict(response.headers)}") # Output: headers keys/values: {'Date': 'Tue, 03 Jun 2023 01:49:09 GMT', 'Server': 'Apache', 'X-Powered-By': 'PleskLin', 'Connection': 'close', 'Transfer-Encoding': 'chunked', 'Content-Type': 'text/html'}

# request info
print(f"request URL: {response.url}") # Output: request URL: http://www.mtitek.com/index.htm
print(f"request headers keys: {list(dict(response.request.headers))}") # Output: request headers keys: ['User-Agent', 'Accept-Encoding', 'Accept', 'Connection']
print(f"request headers keys/values: {dict(response.request.headers)}") # Output: request headers keys/values: {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}

# response content
print(f"response content: {response.text}")
print(f"response raw content: {response.content}")

Authentication:

import requests
from requests.auth import HTTPBasicAuth

url = 'http://www.mtitek.com/index.htm'

# basic authentication
response = requests.get(url, auth=HTTPBasicAuth('user', 'password'))
print(response) # Output: <Response [200]>

# basic authentication (shorthand)
response = requests.get(url, auth=('user', 'password'))
print(response) # Output: <Response [200]>

# bearer token authentication
headers = {'Authorization': 'Bearer jwttoken'}
response = requests.get(url, headers=headers)
print(response) # Output: <Response [200]>

Working with JSON data:

import requests
from requests.exceptions import JSONDecodeError

url = 'http://mtitek.com/tutorials/ml/python/file.json'

response = requests.get(url)

try:
  dict_keys = response.json()
  print(list(dict_keys.keys())) # Output: ['a', 'b', 'c', 'd']
except JSONDecodeError as e:
  print(f"Response is not valid JSON: {e}")