Marshmallow v2/v3 compatibility

Marshmallow

Marshmallow is a package for serialization and deserialization of complex data structures from and to Python types. You can discover its usage within Invenio in the documentation.

Compatibility of v2 and v3

If you are using or you are about to use Marshmallow in your repository implementation, you should know that the package is about to be upgraded to version 3.x. There have been significant changes in the code which will influence already existing code and you should be aware of them.

Note

The full guide is available in the Marshmallow documentation.

We are upgrading the Invenio modules to be compatible with Marshmallow v3.x. We will provide a transition period in which modules will support both v2.3.x and v3.x implementations of Marshmallow. Deprecated methods will be marked with warnings to provide you with time to adjust your code to the latest changes.

There are two options you could follow if you are using schemas from Marshmallow 2:

1. Stay with Marshmallow 2 for now by pinning the Marshmallow version in your instance.

2. Upgrade to Marshmallow 3. Follow the guide below and the Marshmallow documentation

Guide

After upgrading Marshmallow 3 in your dependencies you might find some of your code failing. Mostly it will be connected with schema validation and the webargs package. Webargs in version 5.4.0 is compatible with Marshmallow 3.x.

The biggest change is the return type of Schema().dump(), Schema().load(), Schema().dumps() and Schema().loads(). They do not return the (data, errors) named tuple and the ValidationError is raised instead. In this case if you were using .errors attribute, it will no longer work - you need to implement your own handlers for the type marshmallow.ValidationError. Alternatively by doing Schema.validate() you can still get a list of the errors, an example is located in the Marshmallow documentation.

Other significant changes and solutions:

• ValidationError is the Exception raised when marshmallow encounters an

  error during the validation, since the ValidationError is raised listing all the errors you can catch the Exception and then process them as a dictionary to mimic the previous behaviour.

• Unknown field. indicates that your schema is trying to validate data containing a field which was not defined in the schema. You can either add this field to the schema or use the unknown option of the Schema options to restore the previous, non-strict, behaviour.

• Not a valid type. indicates that the field has data of a different type than defined in the schema.

• Missing field. indicates that a field defined as required is missing from the data.

• JSON object has no attribute data Marshmallow schema provides methods .dump(..), .dumps(..) which no longer return namedtuple of (data, errors). They return json object only (formerly data). All the errors are raised, not returned.

• If your test data is not read properly when using webargs you are probably using old parameters when initialising the arguments. You should change the load_from argument to data_key.

# Marhsmallow 2.x
'part_number': fields.Int(
    load_from='partNumber',
    location='query',
    required=True,
),

# Marhsmallow 3.x
'part_number': fields.Int(
    data_key='partNumber',
    location='query',
    required=True,
),

The full list of Marshmallow v3 changes can be found in the Marshmallow upgrade guide.