Skip to content

(core) implement SessionWriter and SessionReader and many other things

Description

This is the foundation of the new serialization mechanism.

New "crypto" package in libs/core/crypto

  • secure_string: a std::basic_string implementation with a custom secure allocator which erase used memory on de-allocation. This class is mostly used to store sensitive data like password.
  • SHA256: computes SHA256 cryptographic hashes
  • Base64: encode/decode to/from base64 strings
  • AES256: encrypt/decrypt to/from strings using AES with 256 bits key.
  • Unit tests in libs/core/core/test/api/crypto/CryptoTest.xxx

New ArchiveReader and ArchiveWriter classes in libs/io/zip

  • Fixed some nasty bugs on ressource deallocation with WriteZipArchive / ReadZipArchive: Handle from files opened inside archive were not closed
  • Implements some thread safety measures: Due to minizip implementation, to avoid strange corruption problems, an archive could only be opened either in read or in write mode. In the same way, only one file in a archive should be opened at once. With old classes, no restrictions was applied, Yolo mode by default.
  • Allows to specify, as the zip format allows us to do, one compression method, one compression level, and one encryption key by file. This let us, for example, adapt the compression level, when the stored file is already strongly compressed, like for a mp4 file.
  • Use RAII mechanism to close file handle inside archive, global zip handle, etc. It means that opening the same archive, the same file inside an archive in the same scope, or even simply when the ostream/istream is alive, will dead lock (it's a feature, not a bug...)
  • Unit tests in libs/io/zip/test/tu/ArchiveTest.xxx

Refactor UUID

Not planned, but since core::tools::Object::getUUID() was not const, it requires some attention. Basically, only the generator has been kept in core::tools::UUID and all code went in core::tools::Object.

  • Removed double mutex, strange double indirection, etc.. while retaining most of the feature (especially the lazy getter/generator, which a really doubt it is useful, but I did not Benchmarked it, so I decided to keep it)
  • core::tools::Object::getUUID() is now const at the operation doesn't modify external and even internal class state.
  • Unit tests in libs/core/core/test/api/tools/UUIDTest.xxx

Implementation of SessionWriter, SessionReader, ... All in libs/io/session

New PasswordKeeper password management system

Designed to hold passwords, in the case you don't want the user to retype every 3 seconds its password.

  • Replace the less secure and more handcrafted solution in libs/ui/base/preferences/helper.xxx
  • Store the password in an AES256 encrypted secure_string. The key is computed in a deterministic way (no surprise), but should not be so easy to guess. Once the password is no more needed, or when the application quit, it will be erased from memory.
  • Allows to store a "global" password along several "instance" passwords.
  • libs/ui/base/preferences/helper.xxx has been a bit refactored to use the new PasswordKeeper
  • No need to say, a password should never be serialized on disk or in a database. Use only password hash for that.
  • Unit tests in libs/io/session/test/tu/PasswordKeeperTest.xxx

New SessionWriter

It's an implementation of a base::reader::IObjectWriter. It's purpose is to be used in a service to "write" an object to a session archive. It is the public API interface for writing a session archive. Basic usage would be something like:

auto sessionWriter = io::session::SessionWriter::New();
sessionWriter->setObject(myActivitySeries);
sessionWriter->setFile("session.zip");
sessionWriter->setPassword("123")
sessionWriter->write();

New SessionReader

It's an implementation of a base::reader::IObjectReader. It's purpose is to be used in a service to "read" an object from a session archive. It is the public API interface for reading a session archive. Basic usage would be something like:

auto sessionReader = io::session::SessionReader::New();
sessionReader->setFile("session.zip");
sessionReader->setPassword("123")
sessionWriter->read();

auto myActivitySeries = sessionReader->getObject();

You may have noticed the slight change in classical IObjectReader API usage. It is no more needed to guess, before reading, the type of object to deserialize. Instead of "setting" and empty object, we simply "get" it back, once the "read" done

New SessionSerializer

This class contains the the main serialization algorithm. It also contains a specialized serializer (one for each kind of data object) registry, so it can delegate specific serialization tasks. The serialization is done on two medium: one boost ptree and, for big binary file, directly into the archive. The final ptree will also be stored in the archive as the index.json.

The algorithm is really straightforward, not to say "basic":

  1. Extract the UUID of the input object and the class name.
  2. With the UUID, look into an object cache. If the object is already serialized, put a reference on the current ptree node, and stop.
  3. With the classname, find a suitable serializer and call it. The specific serializer will update the current ptree node and maybe store binary file to the archive.
  4. The serializer from step (3) will eventually return also a list of "children" object (this is the case for composite objects). Recall recursively the step (1) on them.
  5. If the object contains fields, recall recursively the step (1) on them.

this MR text will be my base for README.md which I will write once everything are reviewed. Don't panic if you don't see it now, a new MR will be issued later.

New SessionDeserializer

This class is the counterpart of SessionSerializer. Instead of a specialized serializer registry, we have a specialized deserializer registry.

The algorithm is a bit more complex, but still straightforward:

  1. Extract the UUID and the class name of the current object stored in the ptree index.
  2. With the UUID, look into an object cache. If the object is already deserialized return it, and stop.
  3. With the UUID, look into the global UUID object registry. Use it to avoid unneeded object instanciation. It also allow us to safely deserialize children which have direct reference on parent objects.
  4. With the classname, find a suitable deserializer.
  5. First, deserialize the child objects by recalling recursively the step (1) on them.
  6. Deserialize current object, passing the deserialized child objects from step (5)
  7. If the object contains fields, recall recursively the step (1) on them.

Specific object serializer/deserializer

For now, only a very small subset is implemented. This subset should however cover most cases encountered while writing a serializer for a new kind of data object:

  • ActivitySeries[De]serializer:
    • Demonstrate how to serialize object with a child Composite reference and how to use another serializer to mange inheritance (with Series).
  • Composite[De]serializer
  • Equipment[De]serializer
  • Generic[De]serializer:
    • Demonstrate how to serialize "generic" object (Integer, Float, Boolean,String)
  • Mesh[De]serializer
    • Demonstrate how to serialize big binary data to archive
  • Patient[De]serializer
    • Demonstrate how to encrypt sensitive data, regardless of the encryption status of the archive
  • Series[De]serializer
  • String[De]serializer
    • This serialize String to and from Base64. Since boost ptree have serious flaws with strings with special characters in it, encoding to base64 is a suitable workaround, but still a bit overkill..
  • Study[De]serializer

Unit tests

all in libs/io/session/test/tu/SessionTest.hpp

Closes #696 (closed)

How to test it?

Launch unit tests / modify unit tests..

Edited by Didier WECKMANN

Merge request reports