It has been several months since I here talked about my Qt-based library project for serializing data from an object view to JSON / XML and vice versa.



And no matter how proud I am of the built architecture, I must admit - the implementation turned out, frankly, controversial.



All this resulted in a large-scale revision, the results of which will be discussed in this article. For details - under the cut!

QSerializer died

QSerializer had drawbacks, the solution of which often became an even bigger drawback, here are a few of them:

• Very expensive (serialization, keeping property keepers on the heap, controlling the lifetime of the keepers, etc.)
• Working only with QObject-based classes
• Nested "complex" objects and their collections must also be QObject-based
• Inability to supplement collections during deserialization
• Only theoretically infinite nesting
• The inability to work with significant types of "complex" objects, due to the prohibition of copying from QObject
• The need for mandatory registration of types in the Qt meta-object system
• Common "library" problems like linking and portability issues between platforms

Among other things, I wanted to be able to serialize any object "here and now", when this had to use a huge binding of methods in the QSerializer namespace.

Long live QSerializer!

QSerializer was not complete. It was necessary to come up with a solution in which the user would not depend on the QObject, it would be possible to work with value types and at a lower cost.



In a comment to the previous article , the usermicrolanoticed that you can think about using Q_GADGET .



Advantages Q_GADGET :

• No restrictions on copying
• Has a static instance of QMetaObject to access properties

Relying on Q_GADGET , I had to reconsider the approach to how to create JSON and XML based on the declared class fields. The problem of "high cost" was manifested primarily due to:

• Large storage class size (at least 40 bytes)
• Allocating a heap for new guardian entities for each property and controlling their TTL

To reduce the cost, I formulated the following requirement:

The presence in each serializable object of wire methods for serialization / deserialization of all properties of the class and the presence of methods for reading and writing values ​​for each property using the format assigned for this property

Macros

Getting around C ++ 's strong typing, which complicates automatic serialization, is not easy, and previous experience has shown this. Macros, on the other hand, can be a great help for solving such a problem (almost the entire Qt meta-object system is built on macros), because using macros, you can do code generation of methods and properties.



Yes, macros are often evil in their purest form - they are almost impossible to debug. I could compare writing a macro to generate code to putting a crystal shoe on your boss's heel, but difficult doesn't mean impossible!



Lyrical digression about macros

— , , «» (). .

QSerializer currently provides 2 ways to declare a class as serializable: inherit from the QSerializer class or use the QS_CLASS code generation macro .



First of all, you need to define the Q_GADGET macro in the body of the class, this gives access to the staticMetaObject, it will store the properties generated by the macros.



Inheriting from QSerializer will allow you to cast multiple serializable objects to one type and serialize them in bulk.



The QSerializer class contains 4 explorer methods that allow you to parse the properties of an object and one virtual method to get an instance of a QMetaObject:

QJsonValue toJson() const
void fromJson(const QJsonValue &)
QDomNode toXml() const
void fromXml(const QDomNode &)
virtual const QMetaObject * metaObject() const

Q_GADGET does not have all the meta-object binding that Q_OBJECT provides .



Inside the QSerializer, the staticMetaObject instance will represent the QSerializer class, but not derive from it in any way, so when creating the QSerializer-based class, you must override the metaObject method. You can add the QS_SERIALIZER macro to the class body and it will override the metaObject method for you.



Also, using staticMetaObject instead of storing a QMetaObject instance in each object saves 40 bytes from the class size, well, in general, beauty!



If you don't want to inherit for some reason, you can define the QS_CLASS macro in the body of the serialized class, it will generate all the required methods instead of inheriting from QSerializer.

Declaration of fields

Separately, there are 4 kinds of serializable data in JSON and XML, without which serialization to these formats will not be complete. The table shows the types of data and the corresponding macros as a way of describing:



We will assume that the code that generates these macros is called a description, and the macros that generate it are called descriptive.



There is only one principle for generating a description - for a specific field, generate JSON and XML property and define methods for writing / reading values.



Let's analyze the generation of a JSON description using the example of a primitive data type field:

/* Create JSON property and methods for primitive type field*/
#define QS_JSON_FIELD(type, name)                                                           
    Q_PROPERTY(QJsonValue name READ get_json_##name WRITE set_json_##name)                  
    private:                                                                                
        QJsonValue get_json_##name() const {                                                
            QJsonValue val = QJsonValue::fromVariant(QVariant(name));                       
            return val;                                                                     
        }                                                                                   
        void set_json_##name(const QJsonValue & varname){                                   
            name = varname.toVariant().value<type>();                                       
        }   
...
int digit;
QS_JSON_FIELD(int, digit)  

For the int digit field, a property digit with the QJsonValue type will be generated and private write and read methods - get_json_digit and set_json_digit will be defined, they will then become conductors for serializing / deserializing the digit field using JSON.





And here is the generation of a JSON description for a complex structure:

/* Generate JSON-property and methods for some custom class */
/* Custom type must be provide methods fromJson and toJson */
#define QS_JSON_OBJECT(type, name)
    Q_PROPERTY(QJsonValue name READ get_json_##name WRITE set_json_##name)
    private:
    QJsonValue get_json_##name() const {
        QJsonObject val = name.toJson();
        return QJsonValue(val);
    }
    void set_json_##name(const QJsonValue & varname) {
        if(!varname.isObject())
        return;
        name.fromJson(varname);
    } 
...
SomeClass object;
QS_JSON_OBJECT(SomeClass, object)

Complex objects are a set of nested properties that will work as one "big" property for an external class, because such objects will also have wire methods. All you need to do for this is to call the appropriate guide method in the read and write methods of complex structures.

Class creation

Thus, we have a fairly simple infrastructure for creating a serializable class.



So, for example, you can make a class serializable by inheriting from QSerializer:

class SerializableClass : public QSerializer {
Q_GADGET
QS_SERIALIZER
QS_FIELD(int, digit)
QS_COLLECTION(QList, QString, strings)
};

Or like this, using the QS_CLASS macro :

class SerializableClass {
Q_GADGET
QS_CLASS
QS_FIELD(int, digit)
QS_COLLECTION(QList, QString, strings)
};

Limitations

Single Fields



User-defined or primitive types must provide a default constructor.



Collections



The collection class must be templated and provide methods clear, at, size, and append. You can use your own collections, subject to the conditions. Qt collections that satisfy these conditions: QVector, QStack, QList, QQueue.



Qt versions



Minimum version Qt 5.5.0

Minimum tested version Qt 5.9.0

Maximum tested version Qt 5.15.0

NOTE: you can participate in testing and test QSerializer on earlier versions of Qt

Outcome

When reworking QSerializer, I absolutely did not set myself the task of reducing it significantly. However, its size dropped from 9 files to 1, which also reduced its complexity. Now QSerializer is no longer a library in our usual form, now it is just a header file that you just need to include in the project and get all the functionality for comfortable serialization / deserialization. Development began back in March, a tricky architecture was invented and the project was overgrown with dependencies, crutches, rewritten from 0 several times. And all in order to ultimately turn into a small file.



Asking myself: "Was it worth the effort spent on it?", I answer: "Yes, it was." I have already tried it on my combat projects and the result pleased me.



Links

GitHub: link

Latest release: v1.1

Previous article: QSerializer: solution for simple JSON / XML serialization



Future list

• Substantial reduction in cost (can be done even cheaper)
• Compactness
• Working with significant types
• Basic description of serializable data
• Support for any templated collection that provides clear, at, size, and append methods. Even their own
• Fully mutable collections on deserialization
• Support for all popular primitive types
• Support for any custom type described using QSerializer
• No need to register custom types