XEIZE
/Vulnerability documentation/Python/Deserialization of untrusted data

Deserialization of untrusted data

Deserialization of untrusted data

Description

Deserialization reconstructs application objects from stored or transmitted data. Python's serialization formats have different security properties:

  • pickle.load, pickle.loads, and Unpickler.load() may locate and call functions or classes while reconstructing objects. Malicious pickle data can execute arbitrary code during loading; only unpickle trusted data.
  • PyYAML's yaml.unsafe_load, Loader, CLoader, UnsafeLoader, and CUnsafeLoader allow tags that construct Python objects. Do not use them with untrusted YAML.
  • marshal is an internal format mainly used for Python's .pyc files and is not secure against malicious data. allow_code=True can return code objects, but unmarshalling does not itself execute those objects. Corrupt input, incompatible versions, or subsequent use can cause exceptions, undefined behavior, or other security problems.

Do not conflate code execution during pickle or unsafe YAML loading with the integrity, availability, and compatibility risks of marshal.

Potential impact

  • Code execution: Malicious pickle data or unsafe YAML tags can invoke functions or commands with the application's privileges.
  • State tampering: Unexpected objects or values may alter application state and stored data.
  • Information disclosure: Executed code or subsequent processing may access files, secrets, or environment information.
  • Denial of service: Malicious or oversized input can exhaust CPU, memory, or recursion limits, or trigger repeated exceptions.
  • Compatibility failures: marshal code objects are not portable across Python versions; loading them with the wrong version can produce undefined behavior.

Choosing a serialization format

Prefer data-only formats such as JSON or yaml.safe_load for untrusted input. Keeping an Unpickler in a variable or using a custom loader does not remove the need to establish that the input is trustworthy.

Remediation

  1. Replace pickle and marshal on external-input paths with a data-only format such as JSON.
  2. Limit input size and validate the parsed object type, required fields, and value types, lengths, and ranges. Perform schema validation after a data-only parser; validation after unsafe deserialization cannot prevent code execution during loading.
  3. For untrusted YAML, use yaml.safe_load or yaml.load(..., Loader=yaml.SafeLoader). Where available, CSafeLoader uses the same restricted constructors.
  4. If a legacy object format cannot be removed immediately, restrict its producers to trusted, authenticated parties and verify the exact bytes' authenticity and integrity before deserialization. HMAC can prevent tampering by parties without the key; it does not make a malicious authorized producer's payload safe.
  5. Do not rely solely on marshal.loads(..., allow_code=False), HTTP Content-Type, authentication and authorization, post-deserialization validation, or an overly permissive custom Unpickler.
  6. Add size, time, and memory limits for availability, while removing unsafe deserialization functions from untrusted-input boundaries.

Examples

Before

python
import pickle

from flask import Flask, request

app = Flask(__name__)


@app.post("/profile/import")
def import_profile():
    raw = request.get_data(cache=False)
    profile = pickle.loads(raw)  # Malicious pickle data can execute code during this call
    return {"name": str(profile.get("name", ""))}

pickle.loads runs before the request body is validated. Checking the returned object cannot undo code execution that occurred during loading.

After

Change the external API contract to accept JSON and validate its schema after parsing.

python
from flask import Flask, abort, request

app = Flask(__name__)
app.config["MAX_CONTENT_LENGTH"] = 64 * 1024


@app.post("/profile/import")
def import_profile():
    profile = request.get_json()
    if not isinstance(profile, dict):
        abort(400, "profile must be an object")

    name = profile.get("name")
    if not isinstance(name, str) or not 1 <= len(name) <= 100:
        abort(400, "invalid name")

    return {"name": name}

If YAML is required, use a safe loader to construct basic values and then validate their schema:

python
import yaml
from flask import abort, request


def read_yaml_profile():
    profile = yaml.safe_load(request.get_data(cache=False))
    if not isinstance(profile, dict):
        abort(400, "profile must be a mapping")
    return profile

yaml.safe_load does not validate input size or the application's schema. Apply size limits and post-parse validation as well.

References

XEIZE DocumentationTechnical support