cardinal_pythonlib.json.serialize


Original code copyright (C) 2009-2022 Rudolf Cardinal (rudolf@pobox.com).

This file is part of cardinal_pythonlib.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.


Functions to make it easy to serialize Python objects to/from JSON.

See notes_on_pickle_json.txt.

The standard Python representation used internally is a dictionary like this:

{
    __type__: 'MyClass',
    args: [some, positional, args],
    kwargs: {
        'some': 1,
        'named': 'hello',
        'args': [2, 3, 4],
    }
}

We will call this an InitDict.

Sometimes positional arguments aren’t necessary and it’s convenient to work also with the simpler dictionary:

{
    'some': 1,
    'named': 'hello',
    'args': [2, 3, 4],
}

… which we’ll call a KwargsDict.

class cardinal_pythonlib.json.serialize.JsonClassEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

Provides a JSON encoder whose default method encodes a Python object to JSON with reference to our TYPE_MAP.

Constructor for JSONEncoder, with sensible defaults.

If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.

If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.

If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.

If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.

If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.

If specified, separators should be an (item_separator, key_separator) tuple. The default is (’, ‘, ‘: ‘) if indent is None and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.

If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a TypeError.

default(obj: Any) → Any[source]

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)
class cardinal_pythonlib.json.serialize.JsonDescriptor(typename: str, obj_to_dict_fn: Callable[[Any], Dict[KT, VT]], dict_to_obj_fn: Callable[[Dict[KT, VT], Type[object]], Any], cls: Type[object], default_factory: Callable[[], Any] = None)[source]

Describe how a Python class should be serialized to/from JSON.

cardinal_pythonlib.json.serialize.args_kwargs_to_initdict(args: List[Any], kwargs: Dict[str, Any]) → Dict[str, Any][source]

Converts a set of args and kwargs to an InitDict.

cardinal_pythonlib.json.serialize.dict_to_enum_fn(d: Dict[str, Any], enum_class: Type[enum.Enum]) → enum.Enum[source]

Converts an dict to a Enum.

cardinal_pythonlib.json.serialize.dict_to_pendulum(d: Dict[str, Any], pendulum_class: Type[object]) → pendulum.datetime.DateTime[source]

Converts a dict object back to a Pendulum.

cardinal_pythonlib.json.serialize.dict_to_pendulumdate(d: Dict[str, Any], pendulumdate_class: Type[object]) → pendulum.date.Date[source]

Converts a dict object back to a pendulum.Date.

cardinal_pythonlib.json.serialize.dump_map(file: TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None[source]

Prints the JSON “registered types” map to the specified file.

cardinal_pythonlib.json.serialize.enum_to_dict_fn(e: enum.Enum) → Dict[str, Any][source]

Converts an Enum to a dict.

cardinal_pythonlib.json.serialize.initdict_to_instance(d: Dict[str, Any], cls: Type[object]) → Any[source]

Converse of simple_to_dict(). Given that JSON dictionary, we will end up re-instantiating the class with

d = {'a': 1, 'b': 2, 'c': 3}
new_x = SimpleClass(**d)

We’ll also support arbitrary creation, by using both *args and **kwargs.

cardinal_pythonlib.json.serialize.instance_to_initdict_simple(obj: Any) → Dict[str, Any][source]

For use when object attributes (found in obj.__dict__) should be mapped directly to the serialized JSON dictionary. Typically used for classes like:

class SimpleClass(object):
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c

    Here, after

        x = SimpleClass(a=1, b=2, c=3)

    we will find that

        x.__dict__ == {'a': 1, 'b': 2, 'c': 3}

and that dictionary is a reasonable thing to serialize to JSON as keyword arguments.

We’ll also support arbitrary creation, by using both *args and **kwargs. We may not use this format much, but it has the advantage of being an arbitrarily correct format for Python class construction.

cardinal_pythonlib.json.serialize.instance_to_initdict_stripping_underscores(obj: Any) → Dict[str, Any][source]

This is appropriate when a class uses a '_' prefix for all its __init__ parameters, like this:

class UnderscoreClass(object):
    def __init__(self, a, b, c):
        self._a = a
        self._b = b
        self._c = c

Here, after

y = UnderscoreClass(a=1, b=2, c=3)

we will find that

y.__dict__ == {'_a': 1, '_b': 2, '_c': 3}

but we would like to serialize the parameters we can pass back to __init__, by removing the leading underscores, like this:

{'a': 1, 'b': 2, 'c': 3}
cardinal_pythonlib.json.serialize.json_class_decoder_hook(d: Dict[KT, VT]) → Any[source]

Provides a JSON decoder that converts dictionaries to Python objects if suitable methods are found in our TYPE_MAP.

cardinal_pythonlib.json.serialize.json_decode(s: str) → Any[source]

Decodes an object from JSON using our custom decoder.

cardinal_pythonlib.json.serialize.json_encode(obj: Any, **kwargs) → str[source]

Encodes an object to JSON using our custom encoder.

The **kwargs can be used to pass things like 'indent', for formatting.

cardinal_pythonlib.json.serialize.kwargs_to_initdict(kwargs: Dict[str, Any]) → Dict[str, Any][source]

Converts a set of kwargs to an InitDict.

cardinal_pythonlib.json.serialize.make_instance_to_initdict(attributes: List[str]) → Callable[[Any], Dict[KT, VT]][source]

Returns a function that takes an object (instance) and produces an InitDict enabling its re-creation.

cardinal_pythonlib.json.serialize.obj_with_no_args_to_init_dict(obj: Any) → Dict[str, Any][source]

Creates an empty InitDict, for use with an object that takes no arguments at creation.

cardinal_pythonlib.json.serialize.pendulum_to_dict(p: pendulum.datetime.DateTime) → Dict[str, Any][source]

Converts a Pendulum or datetime object to a dict.

cardinal_pythonlib.json.serialize.pendulumdate_to_dict(p: pendulum.date.Date) → Dict[str, Any][source]

Converts a pendulum.Date object to a dict.

cardinal_pythonlib.json.serialize.register_class_for_json(cls: Type[object], method: str = 'simple', obj_to_dict_fn: Callable[[Any], Dict[KT, VT]] = None, dict_to_obj_fn: Callable[[Dict[KT, VT], Type[object]], Any] = <function initdict_to_instance>, default_factory: Callable[[], Any] = None) → None[source]

Registers the class cls for JSON serialization.

  • If both obj_to_dict_fn and dict_to_obj_fn are registered, the framework uses these to convert instances of the class to/from Python dictionaries, which are in turn serialized to JSON.

  • Otherwise:

    if method == 'simple':
        # ... uses simple_to_dict and simple_from_dict (q.v.)
    
    if method == 'strip_underscore':
        # ... uses strip_underscore_to_dict and simple_from_dict (q.v.)
    
cardinal_pythonlib.json.serialize.register_enum_for_json(*args, **kwargs) → Any[source]

Class decorator to register Enum-derived classes with our JSON system. See comments/help for @register_for_json, above.

cardinal_pythonlib.json.serialize.register_for_json(*args, **kwargs) → Any[source]

Class decorator to register classes with our JSON system.

  • If method is 'provides_init_args_kwargs', the class provides a function

    def init_args_kwargs(self) -> Tuple[List[Any], Dict[str, Any]]
    

    that returns an (args, kwargs) tuple, suitable for passing to its __init__() function as __init__(*args, **kwargs).

  • If method is 'provides_init_kwargs', the class provides a function

    def init_kwargs(self) -> Dict
    

    that returns a dictionary kwargs suitable for passing to its __init__() function as __init__(**kwargs).

  • Otherwise, the method argument is as for register_class_for_json().

Usage looks like:

@register_for_json(method=METHOD_STRIP_UNDERSCORE)
class TableId(object):
    def __init__(self, db: str = '', schema: str = '',
                 table: str = '') -> None:
        self._db = db
        self._schema = schema
        self._table = table
cardinal_pythonlib.json.serialize.simple_eq(one: Any, two: Any, attrs: List[str]) → bool[source]

Test if two objects are equal, based on a comparison of the specified attributes attrs.

cardinal_pythonlib.json.serialize.strip_leading_underscores_from_keys(d: Dict[KT, VT]) → Dict[KT, VT][source]

Clones a dictionary, removing leading underscores from key names. Raises ValueError if this causes an attribute conflict.

cardinal_pythonlib.json.serialize.verify_initdict(initdict: Dict[str, Any]) → None[source]

Ensures that its parameter is a proper InitDict, or raises ValueError.

cardinal_pythonlib.json.serialize.wrap_args_kwargs_to_initdict(init_args_kwargs_fn: Callable[[Any], Tuple[List[Any], Dict[str, Any]]], typename: str, check_result: bool = True) → Callable[[Any], Dict[str, Any]][source]

Wraps a function producing a KwargsDict, making it into a function producing an InitDict.

cardinal_pythonlib.json.serialize.wrap_kwargs_to_initdict(init_kwargs_fn: Callable[[Any], Dict[str, Any]], typename: str, check_result: bool = True) → Callable[[Any], Dict[str, Any]][source]

Wraps a function producing a KwargsDict, making it into a function producing an InitDict.