cardinal_pythonlib.json.serialize
Original code copyright (C) 2009-2022 Rudolf Cardinal (rudolf@pobox.com).
This file is part of cardinal_pythonlib.
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Functions to make it easy to serialize Python objects to/from JSON.
See notes_on_pickle_json.txt
.
The standard Python representation used internally is a dictionary like this:
{
__type__: 'MyClass',
args: [some, positional, args],
kwargs: {
'some': 1,
'named': 'hello',
'args': [2, 3, 4],
}
}
We will call this an InitDict
.
Sometimes positional arguments aren’t necessary and it’s convenient to work also with the simpler dictionary:
{
'some': 1,
'named': 'hello',
'args': [2, 3, 4],
}
… which we’ll call a KwargsDict
.
- class cardinal_pythonlib.json.serialize.JsonClassEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)
Provides a JSON encoder whose
default
method encodes a Python object to JSON with reference to ourTYPE_MAP
.Constructor for JSONEncoder, with sensible defaults.
If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.
If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.
If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.
If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.
If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.
If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.
If specified, separators should be an (item_separator, key_separator) tuple. The default is (’, ‘, ‘: ‘) if indent is
None
and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a
TypeError
.- default(obj: Any) Any
Implement this method in a subclass such that it returns a serializable object for
o
, or calls the base implementation (to raise aTypeError
).For example, to support arbitrary iterators, you could implement default like this:
def default(self, o): try: iterable = iter(o) except TypeError: pass else: return list(iterable) # Let the base class default method raise the TypeError return JSONEncoder.default(self, o)
- class cardinal_pythonlib.json.serialize.JsonDescriptor(typename: str, obj_to_dict_fn: Callable[[Any], Dict], dict_to_obj_fn: Callable[[Dict, Type[object]], Any], cls: Type[object], default_factory: Callable[[], Any] | None = None)
Describe how a Python class should be serialized to/from JSON.
- cardinal_pythonlib.json.serialize.args_kwargs_to_initdict(args: List[Any], kwargs: Dict[str, Any]) Dict[str, Any]
Converts a set of
args
andkwargs
to anInitDict
.
- cardinal_pythonlib.json.serialize.dict_to_enum_fn(d: Dict[str, Any], enum_class: Type[Enum]) Enum
Converts an
dict
to aEnum
.
- cardinal_pythonlib.json.serialize.dict_to_pendulum(d: Dict[str, Any], pendulum_class: Type[object]) DateTime
Converts a
dict
object back to aPendulum
.
- cardinal_pythonlib.json.serialize.dict_to_pendulumdate(d: Dict[str, Any], pendulumdate_class: Type[object]) Date
Converts a
dict
object back to apendulum.Date
.
- cardinal_pythonlib.json.serialize.dump_map(file: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) None
Prints the JSON “registered types” map to the specified file.
- cardinal_pythonlib.json.serialize.enum_to_dict_fn(e: Enum) Dict[str, Any]
Converts an
Enum
to adict
.
- cardinal_pythonlib.json.serialize.initdict_to_instance(d: Dict[str, Any], cls: Type[object]) Any
Converse of simple_to_dict(). Given that JSON dictionary, we will end up re-instantiating the class with
d = {'a': 1, 'b': 2, 'c': 3} new_x = SimpleClass(**d)
We’ll also support arbitrary creation, by using both
*args
and**kwargs
.
- cardinal_pythonlib.json.serialize.instance_to_initdict_simple(obj: Any) Dict[str, Any]
For use when object attributes (found in
obj.__dict__
) should be mapped directly to the serialized JSON dictionary. Typically used for classes like:class SimpleClass(object): def __init__(self, a, b, c): self.a = a self.b = b self.c = c Here, after x = SimpleClass(a=1, b=2, c=3) we will find that x.__dict__ == {'a': 1, 'b': 2, 'c': 3}
and that dictionary is a reasonable thing to serialize to JSON as keyword arguments.
We’ll also support arbitrary creation, by using both
*args
and**kwargs
. We may not use this format much, but it has the advantage of being an arbitrarily correct format for Python class construction.
- cardinal_pythonlib.json.serialize.instance_to_initdict_stripping_underscores(obj: Any) Dict[str, Any]
This is appropriate when a class uses a
'_'
prefix for all its__init__
parameters, like this:class UnderscoreClass(object): def __init__(self, a, b, c): self._a = a self._b = b self._c = c
Here, after
y = UnderscoreClass(a=1, b=2, c=3)
we will find that
y.__dict__ == {'_a': 1, '_b': 2, '_c': 3}
but we would like to serialize the parameters we can pass back to
__init__
, by removing the leading underscores, like this:{'a': 1, 'b': 2, 'c': 3}
- cardinal_pythonlib.json.serialize.json_class_decoder_hook(d: Dict) Any
Provides a JSON decoder that converts dictionaries to Python objects if suitable methods are found in our
TYPE_MAP
.
- cardinal_pythonlib.json.serialize.json_decode(s: str) Any
Decodes an object from JSON using our custom decoder.
- cardinal_pythonlib.json.serialize.json_encode(obj: Any, **kwargs) str
Encodes an object to JSON using our custom encoder.
The
**kwargs
can be used to pass things like'indent'
, for formatting.
- cardinal_pythonlib.json.serialize.kwargs_to_initdict(kwargs: Dict[str, Any]) Dict[str, Any]
Converts a set of
kwargs
to anInitDict
.
- cardinal_pythonlib.json.serialize.make_instance_to_initdict(attributes: List[str]) Callable[[Any], Dict]
Returns a function that takes an object (instance) and produces an
InitDict
enabling its re-creation.
- cardinal_pythonlib.json.serialize.obj_with_no_args_to_init_dict(obj: Any) Dict[str, Any]
Creates an empty
InitDict
, for use with an object that takes no arguments at creation.
- cardinal_pythonlib.json.serialize.pendulum_to_dict(p: DateTime) Dict[str, Any]
Converts a
Pendulum
ordatetime
object to adict
.
- cardinal_pythonlib.json.serialize.pendulumdate_to_dict(p: Date) Dict[str, Any]
Converts a
pendulum.Date
object to adict
.
- cardinal_pythonlib.json.serialize.register_class_for_json(cls: ~typing.Type[object], method: str = 'simple', obj_to_dict_fn: ~typing.Callable[[~typing.Any], ~typing.Dict] | None = None, dict_to_obj_fn: ~typing.Callable[[~typing.Dict, ~typing.Type[object]], ~typing.Any] = <function initdict_to_instance>, default_factory: ~typing.Callable[[], ~typing.Any] | None = None) None
Registers the class cls for JSON serialization.
If both
obj_to_dict_fn
and dict_to_obj_fn are registered, the framework uses these to convert instances of the class to/from Python dictionaries, which are in turn serialized to JSON.Otherwise:
if method == 'simple': # ... uses simple_to_dict and simple_from_dict (q.v.) if method == 'strip_underscore': # ... uses strip_underscore_to_dict and simple_from_dict (q.v.)
- cardinal_pythonlib.json.serialize.register_enum_for_json(*args, **kwargs) Any
Class decorator to register
Enum
-derived classes with our JSON system. See comments/help for@register_for_json
, above.
- cardinal_pythonlib.json.serialize.register_for_json(*args, **kwargs) Any
Class decorator to register classes with our JSON system.
If method is
'provides_init_args_kwargs'
, the class provides a functiondef init_args_kwargs(self) -> Tuple[List[Any], Dict[str, Any]]
that returns an
(args, kwargs)
tuple, suitable for passing to its__init__()
function as__init__(*args, **kwargs)
.If method is
'provides_init_kwargs'
, the class provides a functiondef init_kwargs(self) -> Dict
that returns a dictionary
kwargs
suitable for passing to its__init__()
function as__init__(**kwargs)
.Otherwise, the method argument is as for
register_class_for_json()
.
Usage looks like:
@register_for_json(method=METHOD_STRIP_UNDERSCORE) class TableId(object): def __init__(self, db: str = '', schema: str = '', table: str = '') -> None: self._db = db self._schema = schema self._table = table
- cardinal_pythonlib.json.serialize.simple_eq(one: Any, two: Any, attrs: List[str]) bool
Test if two objects are equal, based on a comparison of the specified attributes
attrs
.
- cardinal_pythonlib.json.serialize.strip_leading_underscores_from_keys(d: Dict) Dict
Clones a dictionary, removing leading underscores from key names. Raises
ValueError
if this causes an attribute conflict.
- cardinal_pythonlib.json.serialize.verify_initdict(initdict: Dict[str, Any]) None
Ensures that its parameter is a proper
InitDict
, or raisesValueError
.
- cardinal_pythonlib.json.serialize.wrap_args_kwargs_to_initdict(init_args_kwargs_fn: Callable[[Any], Tuple[List[Any], Dict[str, Any]]], typename: str, check_result: bool = True) Callable[[Any], Dict[str, Any]]
Wraps a function producing a
KwargsDict
, making it into a function producing anInitDict
.
- cardinal_pythonlib.json.serialize.wrap_kwargs_to_initdict(init_kwargs_fn: Callable[[Any], Dict[str, Any]], typename: str, check_result: bool = True) Callable[[Any], Dict[str, Any]]
Wraps a function producing a
KwargsDict
, making it into a function producing anInitDict
.