Python Requests

Learning Objectives

Modern Python applications frequently communicate with websites, APIs, cloud services and external systems. The Requests library makes HTTP communication much easier to work with.

What Is HTTP?

HTTP stands for HyperText Transfer Protocol.

It is one of the main protocols used for communication between clients and web servers.

Python Program
      ↓
   HTTP Request
      ↓
    Web Server
      ↓
   HTTP Response
      ↓
Python Program

When you visit a website, your browser sends requests to a server. The server sends responses back.

What Is Requests?

Requests is a popular Python library for making HTTP requests.

Python's standard library can also communicate over HTTP, but Requests provides a much simpler interface for many common tasks.

import requests

response = requests.get("https://example.com")

print(response.status_code)

Installing Requests

Requests is an external package, so install it with pip.

python -m pip install requests

If you are working inside a virtual environment, make sure the environment is activated before installing the package.

python -m pip show requests

This can help confirm that Requests is installed in the Python environment you are using.

GET Requests

A GET request is commonly used to retrieve information from a server.

import requests

response = requests.get(
    "https://example.com"
)

print(response.text)

The response contains information returned by the server.

<!doctype html>
<html>
...
</html>

When working with APIs, the response may instead contain JSON data.

Understanding Responses

A Requests response object provides useful information about the server's reply.

import requests

response = requests.get(
    "https://example.com"
)

print(response.status_code)
print(response.headers)
print(response.text)

Important response attributes include:

• response.status_code
• response.headers
• response.text
• response.content
• response.url

HTTP Status Codes

HTTP status codes tell you how a server handled a request.

if response.status_code == 200:
    print("Success!")
else:
    print("Request failed.")

Response Headers

HTTP headers contain additional information about a request or response.

import requests

response = requests.get(
    "https://example.com"
)

print(response.headers)

You can access a particular header:

print(response.headers.get("Content-Type"))

Using .get() is convenient when a header may not exist.

Query Parameters

Query parameters are values placed in the URL to provide additional information to a server.

Requests allows you to provide them using the params argument.

import requests

params = {
    "search": "python",
    "page": 1
}

response = requests.get(
    "https://example.com/search",
    params=params
)

print(response.url)

Requests handles URL encoding for the parameters.

POST Requests

POST is commonly used to send data to a server, often when creating a new resource.

import requests

data = {
    "name": "Ali",
    "email": "ali@example.com"
}

response = requests.post(
    "https://example.com/users",
    data=data
)

print(response.status_code)

The exact format expected by a server depends on its API.

Sending JSON

Many modern APIs accept JSON data.

import requests

payload = {
    "name": "Ali",
    "age": 25
}

response = requests.post(
    "https://example.com/users",
    json=payload
)

print(response.status_code)

The json= parameter is usually more convenient than manually converting a Python dictionary into a JSON string.

Request Headers

You can send HTTP headers with a request.

import requests

headers = {
    "Accept": "application/json"
}

response = requests.get(
    "https://example.com/api/products",
    headers=headers
)

print(response.status_code)

APIs may use headers to specify content formats, authentication information or other request metadata.

PUT Requests

PUT is commonly used when replacing or updating a resource.

import requests

data = {
    "name": "Ali",
    "age": 26
}

response = requests.put(
    "https://example.com/users/1",
    json=data
)

print(response.status_code)

The exact semantics depend on the API's design, so always consult its documentation.

PATCH Requests

PATCH is commonly used for partial updates.

import requests

data = {
    "age": 27
}

response = requests.patch(
    "https://example.com/users/1",
    json=data
)

print(response.status_code)

Unlike a full replacement, a PATCH request can communicate only the fields that need changing.

DELETE Requests

DELETE is commonly used to remove a resource.

import requests

response = requests.delete(
    "https://example.com/users/1"
)

print(response.status_code)

A successful DELETE may return a status such as 204 with no response body.

Using Timeouts

Network requests should not normally be allowed to wait forever.

import requests

response = requests.get(
    "https://example.com",
    timeout=10
)

print(response.status_code)

A timeout tells Requests how long to wait before stopping the operation.

Handling Exceptions

Network communication can fail because of connection problems, timeouts and other request-related errors.

import requests

try:

    response = requests.get(
        "https://example.com",
        timeout=10
    )

    response.raise_for_status()

    print(response.text)

except requests.exceptions.RequestException as error:

    print("Request failed:", error)

raise_for_status() raises an HTTP error for unsuccessful HTTP status codes.

Authentication

Many APIs require authentication before they allow access to protected resources.

A common approach is to send a token in a header.

import requests

token = "YOUR_API_TOKEN"

headers = {
    "Authorization": f"Bearer {token}"
}

response = requests.get(
    "https://example.com/api/profile",
    headers=headers
)

print(response.status_code)

Using Sessions

A Session can be useful when making multiple requests to the same service.

import requests

session = requests.Session()

session.headers.update({
    "Accept": "application/json"
})

response1 = session.get(
    "https://example.com/api/products"
)

response2 = session.get(
    "https://example.com/api/orders"
)

print(response1.status_code)
print(response2.status_code)

session.close()

Sessions can maintain settings such as headers and cookies across requests and can reuse connections.

Practical API Example

A common real-world pattern is retrieving JSON data from an API and converting it into Python objects.

import requests

url = "https://example.com/api/products"

try:

    response = requests.get(
        url,
        timeout=10
    )

    response.raise_for_status()

    products = response.json()

    for product in products:
        print(product["name"])

except requests.exceptions.RequestException as error:

    print("Network error:", error)

except ValueError:

    print("The server returned invalid JSON.")

Notice the complete flow:

Send request
     ↓
Check HTTP status
     ↓
Parse JSON
     ↓
Process Python data
     ↓
Display or store results

Common Mistakes

1. Forgetting to Install Requests

ModuleNotFoundError: No module named 'requests'

Install it in the Python environment you are actually using.

2. Ignoring Status Codes

A response object existing does not automatically mean the server returned the data you expected.

3. No Timeout

Network operations can take longer than expected.

4. Assuming Every Response Is JSON

A server may return HTML, text, an empty body or malformed data instead.

5. Exposing API Keys

Never commit secrets directly into public source code.

6. Ignoring Exceptions

Networks are unreliable. Handle failures gracefully.

7. Sending the Wrong Data Format

An API may expect query parameters, form data or JSON. Follow its documentation.

Practice Exercises

Mini Challenge — API Product Fetcher

Build a Python program that retrieves product information from an API and prepares it for an ecommerce report.

Starter Structure

import requests

url = "YOUR_API_URL"

try:

    response = requests.get(
        url,
        timeout=10
    )

    response.raise_for_status()

    products = response.json()

    for product in products:
        print(
            product["name"],
            product["price"]
        )

except requests.exceptions.RequestException as error:

    print("Request failed:", error)

except ValueError:

    print("Invalid JSON response.")

Professional Extension

After retrieving the products, store them in the SQLite database you created in Lesson 60.

API
 ↓
Requests
 ↓
JSON
 ↓
Python dictionaries
 ↓
SQLite
 ↓
Ecommerce report

Knowledge Check

Lesson Summary

Requests provides a clean and practical interface for HTTP communication in Python.

You now have the foundation for communicating with external web services. In the next lesson, we will take this concept further and learn how to work with REST APIs in a structured way.