(core) implement SessionWriter and SessionReader and many other things
Description
This is the foundation of the new serialization mechanism.
libs/core/crypto
New "crypto" package in -
secure_string
: a std::basic_string implementation with a customsecure
allocator which erase used memory on de-allocation. This class is mostly used to store sensitive data like password. -
SHA256
: computes SHA256 cryptographic hashes -
Base64
: encode/decode to/from base64 strings -
AES256
: encrypt/decrypt to/from strings using AES with 256 bits key. - Unit tests in
libs/core/core/test/api/crypto/CryptoTest.xxx
ArchiveReader
and ArchiveWriter
classes in libs/io/zip
New - Fixed some nasty bugs on ressource deallocation with WriteZipArchive / ReadZipArchive: Handle from files opened inside archive were not closed
- Implements some thread safety measures: Due to minizip implementation, to avoid strange corruption problems, an archive could only be opened either in
read
or inwrite
mode. In the same way, only one file in a archive should be opened at once. With old classes, no restrictions was applied, Yolo mode by default. - Allows to specify, as the zip format allows us to do, one compression method, one compression level, and one encryption key by file. This let us, for example, adapt the compression level, when the stored file is already strongly compressed, like for a mp4 file.
- Use RAII mechanism to close file handle inside archive, global zip handle, etc. It means that opening the same archive, the same file inside an archive in the same scope, or even simply when the ostream/istream is alive, will dead lock (it's a feature, not a bug...)
- Unit tests in
libs/io/zip/test/tu/ArchiveTest.xxx
Refactor UUID
Not planned, but since core::tools::Object::getUUID()
was not const
, it requires some attention. Basically, only the generator has been kept in core::tools::UUID
and all code went in core::tools::Object
.
- Removed double mutex, strange double indirection, etc.. while retaining most of the feature (especially the
lazy
getter/generator, which a really doubt it is useful, but I did not Benchmarked it, so I decided to keep it) -
core::tools::Object::getUUID()
is now const at the operation doesn't modify external and even internal class state. - Unit tests in
libs/core/core/test/api/tools/UUIDTest.xxx
libs/io/session
Implementation of SessionWriter, SessionReader, ... All in
PasswordKeeper
password management system
New Designed to hold passwords, in the case you don't want the user to retype every 3 seconds its password.
- Replace the less secure and more
handcrafted
solution inlibs/ui/base/preferences/helper.xxx
- Store the password in an AES256 encrypted
secure_string
. The key is computed in a deterministic way (no surprise), but should not be so easy to guess. Once the password is no more needed, or when the application quit, it will be erased from memory. - Allows to store a "global" password along several "instance" passwords.
-
libs/ui/base/preferences/helper.xxx
has been a bit refactored to use the newPasswordKeeper
- No need to say, a password should never be serialized on disk or in a database. Use only password hash for that.
- Unit tests in
libs/io/session/test/tu/PasswordKeeperTest.xxx
SessionWriter
New It's an implementation of a base::reader::IObjectWriter. It's purpose is to be used in a service to "write" an object to a session archive. It is the public API interface for writing a session archive. Basic usage would be something like:
auto sessionWriter = io::session::SessionWriter::New();
sessionWriter->setObject(myActivitySeries);
sessionWriter->setFile("session.zip");
sessionWriter->setPassword("123")
sessionWriter->write();
SessionReader
New It's an implementation of a base::reader::IObjectReader. It's purpose is to be used in a service to "read" an object from a session archive. It is the public API interface for reading a session archive. Basic usage would be something like:
auto sessionReader = io::session::SessionReader::New();
sessionReader->setFile("session.zip");
sessionReader->setPassword("123")
sessionWriter->read();
auto myActivitySeries = sessionReader->getObject();
You may have noticed the slight change in classical
IObjectReader
API usage. It is no more needed to guess, before reading, the type of object to deserialize. Instead of "setting" and empty object, we simply "get" it back, once the "read" done
SessionSerializer
New This class contains the the main serialization algorithm. It also contains a specialized serializer (one for each kind of data object) registry, so it can delegate specific serialization tasks. The serialization is done on two medium: one boost ptree and, for big binary file, directly into the archive. The final ptree will also be stored in the archive as the index.json.
The algorithm is really straightforward, not to say "basic":
- Extract the UUID of the input object and the class name.
- With the UUID, look into an object cache. If the object is already serialized, put a reference on the current ptree node, and stop.
- With the classname, find a suitable serializer and call it. The specific serializer will update the current ptree node and maybe store binary file to the archive.
- The serializer from step (3) will eventually return also a list of "children" object (this is the case for composite objects). Recall recursively the step (1) on them.
- If the object contains
fields
, recall recursively the step (1) on them.
this MR text will be my base for README.md which I will write once everything are reviewed. Don't panic if you don't see it now, a new MR will be issued later.
SessionDeserializer
New This class is the counterpart of SessionSerializer
. Instead of a specialized serializer registry, we have a specialized deserializer registry.
The algorithm is a bit more complex, but still straightforward:
- Extract the UUID and the class name of the current object stored in the ptree index.
- With the UUID, look into an object cache. If the object is already deserialized return it, and stop.
- With the UUID, look into the global UUID object registry. Use it to avoid unneeded object instanciation. It also allow us to safely deserialize children which have direct reference on parent objects.
- With the classname, find a suitable deserializer.
- First, deserialize the child objects by recalling recursively the step (1) on them.
- Deserialize current object, passing the deserialized child objects from step (5)
- If the object contains
fields
, recall recursively the step (1) on them.
Specific object serializer/deserializer
For now, only a very small subset is implemented. This subset should however cover most cases encountered while writing a serializer for a new kind of data object:
- ActivitySeries[De]serializer:
- Demonstrate how to serialize object with a child
Composite
reference and how to use another serializer to mange inheritance (withSeries
).
- Demonstrate how to serialize object with a child
- Composite[De]serializer
- Equipment[De]serializer
- Generic[De]serializer:
- Demonstrate how to serialize "generic" object (Integer, Float, Boolean,String)
- Mesh[De]serializer
- Demonstrate how to serialize big binary data to archive
- Patient[De]serializer
- Demonstrate how to encrypt sensitive data, regardless of the encryption status of the archive
- Series[De]serializer
- String[De]serializer
- This serialize String to and from Base64. Since boost ptree have serious flaws with strings with special characters in it, encoding to base64 is a suitable workaround, but still a bit overkill..
- Study[De]serializer
Unit tests
all in libs/io/session/test/tu/SessionTest.hpp
Closes #696 (closed)
How to test it?
Launch unit tests / modify unit tests..