Skip to content
  • Didier WECKMANN's avatar
    feat(core): basic session implementation · 20bdecfc
    Didier WECKMANN authored
    This is the foundation of the new serialization mechanism.
    
    ### New "crypto" package in `libs/core/crypto`
    
    - `secure_string`: a std::basic_string implementation with a custom `secure` allocator which erase used memory on de-allocation. This class is mostly used to store sensitive data like password.
    - `SHA256`: computes SHA256 cryptographic hashes
    - `Base64`: encode/decode to/from base64 strings
    - `AES256`: encrypt/decrypt to/from strings using AES with 256 bits key.
    - Unit tests in `libs/core/core/test/api/crypto/CryptoTest.xxx`
    
    ### New `ArchiveReader` and `ArchiveWriter` classes in `libs/io/zip`
    
    - Fixed some nasty bugs on ressource deallocation with WriteZipArchive / ReadZipArchive: Handle from files opened inside archive were not closed
    - Implements some thread safety measures: Due to minizip implementation, to avoid strange corruption problems, an archive could only be opened either in `read` or in `write` mode. In the same way, only one file in a archive should be opened at once. With old classes, no restrictions was applied, Yolo mode by default.
    - Allows to specify, as the zip format allows us to do, one compression method, one compression level, and one encryption key by file. This let us, for example, adapt the compression level, when the stored file is already strongly compressed, like for a mp4 file.
    - Use RAII mechanism to close file handle inside archive, global zip handle, etc. It means that opening the same archive, the same file inside an archive in the same scope, or even simply when the ostream/istream is alive, will dead lock (it's a feature, not a bug...)
    - Unit tests in `libs/io/zip/test/tu/ArchiveTest.xxx`
    
    
    ### Refactor UUID
    
    Not planned, but since `core::tools::Object::getUUID()` was not `const`, it requires some attention. Basically, only the generator has been kept in `core::tools::UUID` and all code went in `core::tools::Object`.
    - Removed double mutex, strange double indirection, etc.. while retaining most of the feature (especially the `lazy` getter/generator, which a really doubt it is useful, but I did not Benchmarked it, so I decided to keep it)
    - `core::tools::Object::getUUID()` is now const at the operation doesn't modify external and even internal class state.
    - Unit tests in `libs/core/core/test/api/tools/UUIDTest.xxx`
    
    
    ### Implementation of SessionWriter, SessionReader, ... All in `libs/io/session`
    
    #### New `PasswordKeeper` password management system
    Designed to hold passwords, in the case you don't want the user to retype every 3 seconds its password.
    
    - Replace the less secure and more `handcrafted` solution in `libs/ui/base/preferences/helper.xxx`
    - Store the password in an AES256 encrypted `secure_string`. The key is computed in a deterministic way (no surprise), but should not be so easy to guess. Once the password is no more needed, or when the application quit, it will be erased from memory.
    - Allows to store a "global" password along several "instance" passwords.
    - `libs/ui/base/preferences/helper.xxx` has been a bit refactored to use the new `PasswordKeeper`
    - No need to say, a password should never be serialized on disk or in a database. Use only password hash for that.
    - Unit tests in `libs/io/session/test/tu/PasswordKeeperTest.xxx`
    
    #### New `SessionWriter`
    
    It's an implementation of a base::reader::IObjectWriter. It's purpose is to be used in a service to "write" an object to a session archive. It is the public API interface for writing a session archive. Basic usage would be something like:
    
    ```c++
    auto sessionWriter = io::session::SessionWriter::New();
    sessionWriter->setObject(myActivitySeries);
    sessionWriter->setFile("session.zip");
    sessionWriter->setPassword("123")
    sessionWriter->write();
    ```
    
    #### New `SessionReader`
    
    It's an implementation of a base::reader::IObjectReader. It's purpose is to be used in a service to "read" an object from a session archive. It is the public API interface for reading a session archive. Basic usage would be something like:
    
    ```c++
    auto sessionReader = io::session::SessionReader::New();
    sessionReader->setFile("session.zip");
    sessionReader->setPassword("123")
    sessionWriter->read();
    
    auto myActivitySeries = sessionReader->getObject();
    ```
    > You may have noticed the slight change in classical `IObjectReader` API usage. It is no more needed to guess, before reading, the type of object to deserialize. Instead of "setting" and empty object, we simply "get" it back, once the "read" done
    
    
    #### New `SessionSerializer`
    This class contains the the main serialization algorithm. It also contains a specialized serializer (one for each kind of data object) registry, so it can delegate specific serialization tasks. The serialization is done on two medium: one boost ptree and, for big binary file, directly into the archive. The final ptree will also be stored in the archive as the index.json.
    
    The algorithm is really straightforward, not to say "basic":
    
    1. Extract the UUID of the input object and the class name.
    2. With the UUID, look into an object cache. If the object is already serialized, put a reference on the current ptree node, and stop.
    3. With the classname, find a suitable serializer and call it. The specific serializer will update the current ptree node and maybe store binary file to the archive.
    4. The serializer from step (3) will eventually return also a list of "children" object (this is the case for composite objects). Recall recursively the step (1) on them.
    5. If the object contains `fields`, recall recursively the step (1) on them.
    
    > this MR text will be my base for README.md which I will write once everything are reviewed. Don't panic if you don't see it now, a new MR will be issued later.
    
    #### New `SessionDeserializer`
    This class is the counterpart of `SessionSerializer`. Instead of a specialized serializer registry, we have a specialized deserializer registry.
    
    The algorithm is a bit more complex, but still straightforward:
    
    1. Extract the UUID and the class name of the current object stored in the ptree index.
    2. With the UUID, look into an object cache. If the object is already deserialized return it, and stop.
    3. With the UUID, look into the global UUID object registry. Use it to avoid unneeded object instanciation. It also allow us to safely deserialize children which have direct reference on parent objects.
    4. With the classname, find a suitable deserializer.
    5. First, deserialize the child objects by recalling recursively the step (1) on them.
    6. Deserialize current object, passing the deserialized child objects from step (5)
    7. If the object contains `fields`, recall recursively the step (1) on them.
    
    #### Specific object serializer/deserializer
    
    For now, only a very small subset is implemented. This subset should however cover most cases encountered while writing a serializer for a new kind of data object:
    - ActivitySeries[De]serializer:
      - Demonstrate how to serialize object with a child `Composite` reference and how to use another serializer to mange inheritance (with `Series`).
    - Composite[De]serializer
    - Equipment[De]serializer
    - Generic[De]serializer:
      - Demonstrate how to serialize "generic" object (Integer, Float, Boolean,String)
    - Mesh[De]serializer
      - Demonstrate how to serialize big binary data to archive
    - Patient[De]serializer
      - Demonstrate how to encrypt sensitive data, regardless of the encryption status of the archive
    - Series[De]serializer
    - String[De]serializer
      - This serialize String to and from Base64. Since boost ptree have serious flaws with strings with special characters in it, encoding to base64 is a suitable workaround, but still a bit overkill.. 
    - Study[De]serializer
    
    #### Unit tests
    all in `libs/io/session/test/tu/SessionTest.hpp`
    20bdecfc