Discriminate between element not provided and element provided with empty string

[skewty]

Example 1

<foo><next></next></foo>

Example 2

<foo></foo>

Sample Code

from pydantic_xml import BaseXmlModel, element

class Foo(BaseXmlModel, tag='foo'):
    next: str | None = element("next", default=None)

If next is provided with empty string (example 1) I need it to equal '', if it not provided (example 2) I need it to equal None.

How do I accomplish this?

[dapper91]

@skewty Hi,

you may define your own field validator:

from pydantic_xml import BaseXmlModel, element, xml_field_validator
from pydantic_xml.element import XmlElementReader

class Foo(BaseXmlModel, tag='foo'):
    next: str | None = element("next", default=None)

    @xml_field_validator('next')
    @classmethod
    def as_empty_string(cls, element: XmlElementReader, field_name: str) -> str | None:
        if (sub_element := element.pop_element(field_name, search_mode=cls.__xml_search_mode__)):
            return sub_element.pop_text() or ''

        return None


print(repr(Foo.from_xml("<foo><next>bar</next></foo>")))  # Foo(next='bar')
print(repr(Foo.from_xml("<foo><next></next></foo>")))  # Foo(next='')
print(repr(Foo.from_xml("<foo></foo>")))  # Foo(next=None)

[skewty]

That works! Big thank you. I have to use the above trick in around 50 different models for various fields so I was looking into clean, concise ways to include it without too much boilerplate. It wasn't as easy or clean as on might expect. DRY was difficult.

I looked into using Annotated and XmlFieldValidator which could work but made the code for those fields quite different and much longer than others. Preserving field order is a pain in the but. I prefer JSON :)

I settled on this function:

def allow_empty_string(
    cls: type[BaseXmlModel], element: XmlElementReader, field_name: str
) -> str | None:
    """pydantic-xml treats empty string and null as the same. This allows us to have
    both and differentiate between the two cases. Use this function in a
    xml_field_validator as seen below."""
    tag = field_name.replace("_", "")  # undo snake case to get xml tag name
    sub_element = element.pop_element(tag, search_mode=cls.__xml_search_mode__)
    return (sub_element.pop_text() or "") if sub_element else None

class SomeModel(BaseXmlModel):
    ...
    reference_number: str | None = element("referencenumber", default=None)
    ...
    status_info: str | None = element("statusinfo", default=None)
    ...

    @xml_field_validator("reference_number", "status_info")
    @classmethod
    def allow_empty(cls, element: XmlElementReader, field_name: str) -> str | None:
        return allow_empty_string(cls, element, field_name)

    # I tried to find some python syntax that would allow me to use the allow_empty_string
    # function directly without defining a new class method each time but failed.

For the models in my file, the little tag name hack using replace() works fine but begs the question, how would I get the element name? Is there a better / safer way?

Ideally, in a perfect world, I could:

class Foo(BaseXmlModel):
    next: str | None = element("next", empty_string=True, default=None)

or

class Foo(BaseXmlModel, empty_string=True):
    # affects all optional strings fields in model
    next: str | None = element("next", default=None)

I take it the coalescing of empty string to null is due to something that occurs upstream in the XML libraries being used. This behavior was very surprising to me. And not being able to easily get out from it was equally surprising. This would be a nice feature to add in my opinion.

[skewty]

Is this the best we can do?

def allow_empty_string(
    cls: type[BaseXmlModel], element: XmlElementReader, field_name: str
) -> str | None:
    """pydantic-xml treats empty string and null as the same. This allows us to have
    both and differentiate between the two cases. Use this function in a
    xml_field_validator as seen below."""
    tag = field_name  # fallback
    for xei in cls.__pydantic_fields__[field_name].metadata:
        if isinstance(xei, XmlEntityInfo):
            if xei.path:
                tag = xei.path
            break
    sub_element = element.pop_element(tag, search_mode=cls.__xml_search_mode__)
    return (sub_element.pop_text() or "") if sub_element else None

[skewty]

Perhaps a few more auto-generated class methods in XmlBaseModel would make sense?

@classmethod
def __xml_entity_info__(cls, field_name: str) -> XmlEntityInfo | None:
    for xei in cls.__pydantic_fields__[field_name].metadata:
        if isinstance(xei, XmlEntityInfo):
            return xei
    return None

@classmethod
def __xml_tag__(cls, field_name: str) -> str:
    xei = cls.__xml_entity_info(field_name)
    return xei.path if xei and xei.path else field_name