I recently wrote an article on “Converting a Python Dictionary to an Object to Simplify Nested Data Structures — A Utility”. That article covered how to convert dictionaries into an object.

A follower mention that Pydnatic can also be used for conversion plus he points out: “You define explicitly what you expect, and you get autocompletion as a bonus 🙂”.

Let’s take a look at Pydantic and its use in simplfying nested JSON.

What is Pydantic?

Pydantic is a data validation and settings management library for Python. It parses raw data into structured Python objects, ensuring the data adheres to the expected types and formats. It’s widely used for defining configurations, parsing API responses, and managing structured data in web frameworks.

Pydantic shines when dealing with nested JSON data. By defining data models explicitly, developers gain better control over the data structure, making it easier to work with complex datasets.

Why Use Pydantic for Nested JSON?

1. Explicit Structure: You define what you expect. Each field and its type are clearly laid out.
2. Validation: Pydantic validates the input data, ensuring it matches the defined schema.
3. Autocompletion: IDEs provide autocompletion when working with Pydantic models, improving developer productivity.
4. Ease of Conversion: Pydantic converts dictionaries into structured objects effortlessly.

Example: Parsing Nested JSON

Consider a library catalog represented as a nested JSON structure:

library_data = {
    "fiction": [
        {
            "title": "1984",
            "author": "George Orwell",
            "details": {"year": 1949, "genre": "Dystopian"}
        },
        {
            "title": "The Great Gatsby",
            "author": "F. Scott Fitzgerald",
            "details": {"year": 1925, "genre": "Classic"}
        }
    ],
    "non_fiction": [
        {
            "title": "Sapiens",
            "author": "Yuval Noah Harari",
            "details": {"year": 2011, "genre": "History"}
        },
        {
            "title": "Educated",
            "author": "Tara Westover",
            "details": {"year": 2018, "genre": "Memoir"}
        }
    ]
}

Accessing the genre of “1984” requires indexing:

genre = library_data["fiction"][0]["details"]["genre"]

Pydantic can be used to define models for this data and work with it in a structured way.

Defining Models with Pydantic

Here’s how to create Pydantic models for the normalized library data:

from pydantic import BaseModel
from typing import List

class BookDetails(BaseModel):
    year: int
    genre: str

class Book(BaseModel):
    title: str
    author: str
    details: BookDetails

class Library(BaseModel):
    fiction: List[Book]
    non_fiction: List[Book]

With these models, the raw dictionary can be parsed into a structured object:

library = Library(**library_data)

Now, accessing the genre of “1984” becomes:

genre = library.fiction[0].details.genre
print(genre)  # Output: Dystopian

Validation in Action

Pydantic ensures the data conforms to the defined schema. If the input data is invalid, Pydantic raises an error with detailed messages:

invalid_data = {
    "fiction": [
        {"title": "1984", "author": "George Orwell", "details": {"year": "1949", "genre": "Dystopian"}}
    ],
    "non_fiction": []
}

try:
    Library(**invalid_data)
except Exception as e:
    print(e)

This example would raise a validation error because the year is a string instead of an integer.

Autocompletion

One of the standout features of Pydantic is autocompletion. After defining the models, most IDEs can suggest attributes and methods, reducing the chance of errors:

library.fiction[0].details.
# IDE suggests: year, genre

Additional Features of Pydantic

1. Aliases: Pydantic supports custom field aliases, allowing you to map JSON keys to Python-friendly attribute names.
2. Default Values: Fields can have default values, making them optional.
3. Custom Validation: Add custom validation logic using @validator decorators.
4. Serialization: Convert Pydantic models back to JSON or dictionaries using dict() or json() methods.

Thank you for reading this article. I hope you found it helpful and informative. If you have any questions, or if you would like to suggest new Python code examples or topics for future tutorials, please feel free to reach out. Your feedback and suggestions are always welcome!

Happy coding!
Py-Core.com Python Programming

You can also find this article at Medium.com