Skip to content
Advertisement

Serialising with cattrs and want to omit field x1 string field

Using cattrs to structure data and I want to omit x1 string field.

I want to perform a trivial cleanup on strings that have been passed in except for the password field.

I can get it to work on all strings

from attrs import define
from cattrs import Converter

MYDATA = {
           "hostname": "MYhostNAme ", 
           "port": 389, 
           "adminuser": " cn=admin, dc=acme, dc=com", 
           "adminpass": " ADmin "
          }

@define
class _LDAP:
  hostname: str                                                                                                          
  port: int                                                                                                              
  adminuser: str                                                                                                         
  adminpass: str                                                                                                         
                  
def tidystr(text):
  return text.lower().translate(str.maketrans("", "", " ntr"))                                                        
                                                                                                                         
class _Vars:
  converter = Converter()                                                                                                
  converter.register_structure_hook(str, lambda x, cls: tidystr(x))                                                      
  ldap = converter.structure(MYDATA, _LDAP)         
  
app = _Vars()
assert app.ldap.hostname == "myhostname"  # True
assert app.ldap.adminpass == "admin"      # True !!Not what I want!!

I can fool cattrs by passing in the adminpass field as Any

@define
class _LDAP:
  adminpass: Any

but this is a bit clunky.

The docs show how to omit individual fields – but I can’t figure out how then I would call the tidystr function. Following the docs closely I would do

class Vars:                                                                                                                
  converter = Converter()                                                                                                
  hook = make_dict_structure_fn(_LDAP, converter, adminpass=override(omit=True))                                         
  converter.register_structure_hook(_LDAP, hook)                                                                       
  ldap = converter.structure(MYDATA, _LDAP)

which obviously won’t work because tidystr() isn’t being called.

I’ve tried various ways and am lost. The docs also show something like what I’m trying to do but the example is changing the keys not the values.

Advertisement

Answer

I’m the author of cattrs. Let’s see how we can solve this.

First we need a way to recognize which fields you want to tidy up and which you don’t. Looks like you’d like to apply tidying to all strings by default and opt-out for some fields.

Option #1: NewType

Use a NewType for the password field.

from typing import NewType

from attrs import define

from cattrs import Converter

NonTidyStr = NewType("NonTidyStr", str)

MYDATA = {
    "hostname": "MYhostNAme ",
    "port": 389,
    "adminuser": " cn=admin, dc=acme, dc=com",
    "adminpass": " ADmin ",
}


@define
class _LDAP:
    hostname: str
    port: int
    adminuser: str
    adminpass: NonTidyStr


def tidystr(text):
    return text.lower().translate(str.maketrans("", "", " ntr"))


class _Vars:
    converter = Converter()
    converter.register_structure_hook(str, lambda x, cls: tidystr(x))
    converter.register_structure_hook(NonTidyStr, lambda x, _: str(x))
    ldap = converter.structure(MYDATA, _LDAP)


app = _Vars()
assert app.ldap.hostname == "myhostname"  # True
assert app.ldap.adminpass == " ADmin "

(Any function taking a string will happily take a NewType based on it instead, and it’ll actually be a string at runtime.)

Option 2: Annotated

We can use typing.Annotated to build our own little mini system for structuring.

The class becomes:

@define
class _LDAP:
    hostname: str
    port: int
    adminuser: str
    adminpass: Annotated[str, "notidy"]

Then, we need to create the appropriate hook:

from cattrs._compat import is_annotated

def is_no_tidy(t: Any) -> bool:
    return is_annotated(t) and "notidy" in t.__metadata__

converter.register_structure_hook_func(is_no_tidy, lambda x, _: str(x))

The implementation is a little gnarly since Python doesn’t have super good type inspection capabilities, but we can use an internal cattrs function instead.

This approach is, in general, very powerful.

Option 3: wait for 22.3.0

I’m adding an option to override the structure and unstructure functions for individual fields in the next version. Then, you’ll be able to do something like:

hook = make_dict_structure_fn(_LDAP, converter, adminpass=override(structure_fn=str))                                         
converter.register_structure_hook(_LDAP, hook)

Can’t promise a release date though ;)

Advertisement