`pyyaml` can’t parse `pydantic` object if `typing` module is used

Let me start off by saying I wanted to open an issue in pydantic repo. Once I started rubber duck debugging I came up to the conclusion it’s actually pyyaml that isn’t working right but I’m not so sure anymore.

from dataclasses import dataclass
from functools import partial
from typing import List, Type

import yaml
from pydantic import BaseModel

yaml_input = """
!Foo
name: foo
bar:
    - !Bar
      name: bar
  """


def get_loader():
    loader = yaml.SafeLoader
    for tag_name, tag_constructor in tag_model_map.items():
        loader.add_constructor(tag_name, tag_constructor)
    return loader


def dynamic_constructor_mapping(model_class: Type[BaseModel], loader: yaml.SafeLoader,
                        node: yaml.nodes.MappingNode) -> BaseModel:
    return model_class(**loader.construct_mapping(node))


def get_constructor_for_mapping(model_class: Type[BaseModel]):
    return partial(dynamic_constructor_mapping, model_class)


class Bar(BaseModel):
    name: str


class Foo1(BaseModel):
    name: str
    bar: list


class Foo2(BaseModel):
    name: str
    bar: List


class Foo3(BaseModel):
    name: str
    bar: List[Bar]


@dataclass
class Foo4:
    name: str
    bar: List[Bar]


foos = [Foo1, Foo2, Foo3, Foo4]

for foo_cls in foos:
    tag_model_map = {
        "!Foo": get_constructor_for_mapping(foo_cls),
        "!Bar": get_constructor_for_mapping(Bar),
    }
    print(f"{foo_cls.__qualname__} loaded {yaml.load(yaml_input, Loader=get_loader())}")

which prints

Foo1 loaded name='foo' bar=[Bar(name='bar')]
Foo2 loaded name='foo' bar=[]
Foo3 loaded name='foo' bar=[]
Foo4 loaded Foo4(name='foo', bar=[Bar(name='bar')])

list of pydantic objects is parsed correctly if list is used in static typing
list of pydantic objects is NOT parsed correctly if List is used in static typing
list of pydantic objects is NOT parsed correctly if List[Bar] is used in static typing
list of dataclass objects is always parsed correctly

The constructor seems to be returning the correct object in all examples so I don’t understand where the problem lies.

pydantic==1.8.2
Python 3.8.10

Answer

I had the same exact issue as you.

The thing that solved the issue for me was setting the deep to true in the construct_mapping method.

Example:

fields = loader.construct_mapping(node, deep=True)

Advertisement

Answer