Commit 60a298a9 authored by Didier WECKMANN's avatar Didier WECKMANN
Browse files

feat(io): Drop in replacement for SReader and SWriter to serialize an application session

### Main feature

Two services `sight::module::io::session::SReader` and `sight::module::io::session::SWriter` were implemented. They read/write a root data object from/to a session file on a filesystem.

The session file is indeed a standard "ZIP" archive, while the compression algorithm for files inside the session archive is ZSTD. A standard archive reader could open a session file, if it is able to handle ZIP archive with ZSTD compression.

The archive can be password protected using AES256 algorithm and the compression level is set individually, depending of the type of data to serialize.

The service configuration includes specifying the file extension and the password policy for encryption.

Configuration example:

```xml
<service type="sight::module::io::session::SReader">
    <inout key="data" uid="..." />
    <dialog extension=".sample" description="Sample Sight session file" policy=always/>
    <password policy="once" encryption="salted"/>
</service>
```

The dialog policy specifies when the open dialog is shown:

* **never**: never show the open dialog
* **once**: show only once, store the location as long as the service is started
* **always**: always show the location dialog
* **default**: default behavior, which is "always"

The password policy defines if we should protect the session file using a password and when to ask for it:

* **never**: a password will never be asked and the session file will never be encrypted.
* **once**: a password will be asked only once and stored in the memory for subsequent uses. The session file will be encrypted.
* **always**: a password will always be asked. The session file will be encrypted.
* **default**: uses the builtin default behavior which is "never".

The encryption policy defines if we uses the password as is or with salt. It can also be used to encrypt without password:

* **password**: Use the given password for encryption.
* **salted**: Use the given password with salt for encryption.
* **forced**: Force encryption with a pseudo random hidden password (if no password are provided).
* **default**: uses the builtin default behavior which is "password".

### General improvement:

* `ExActivities` has been modified to use the new session services instead of atoms
* new `TemporaryFile` class in `core::tools::System` that use ROII to delete the associated file as soon as the `TemporaryFile` class is destroyed.
* `core::tools::System::RobustRename()` now have an optional parameter to force renaming, even if the target already exists (it will be first deleted)
* `ui::base::Cursor` improvement: `BusyCursor`, `WaitCursor`, `CrossCursor` classes that use ROII to restore back "default" cursor, even if an exception occurs
* `ui::xxx::dialog::InputDialog` improvement: add a "bullet" mode for password field.
* `ui::xxx::dialog::MessageDialog` improvement: add a "retry" button.


Merge branch '752-io-drop-in-replacement-for-sreader-and-swriter-to-serialize-an-application-session' into 'dev'
Closes #752
See merge request sight/sight!606
parents 8bec599b bef282a8
......@@ -104,6 +104,7 @@
<type mode="reader" />
<selection mode="exclude" />
<addSelection service="sight::module::io::atoms::SReader" />
<addSelection service="sight::module::io::session::SReader" />
<addSelection service="sight::module::io::matrix::SAttachmentSeriesReader" />
</config>
</extension>
......@@ -204,4 +205,23 @@
</config>
</extension>
<!-- Service config used for session reader and writer -->
<extension implements="sight::service::extension::Config">
<id>DefaultSessionReaderConfig</id>
<service>sight::module::io::session::SReader</service>
<desc>Session reader</desc>
<config>
<dialog extension=".sight" description="Sight session file"/>
</config>
</extension>
<extension implements="sight::service::extension::Config">
<id>DefaultSessionWriterConfig</id>
<service>sight::module::io::session::SWriter</service>
<desc>Session writer</desc>
<config>
<dialog extension=".sight" description="Sight session file"/>
</config>
</extension>
</plugin>
sight_add_target( config_ui_activity TYPE MODULE )
add_dependencies(config_ui_activity
add_dependencies(config_ui_activity
data
module_activity
module_data
module_ui_base
module_ui_qt
module_io_session
module_io_atoms
module_io_activity
module_ui_icons
......
......@@ -67,9 +67,9 @@
<param name="ICON_PATH" />
<param name="seriesDB" />
<param name="SEQUENCER_CONFIG" />
<param name="WIZARD_CONFIG" default="defaultWizardConfig"/>
<param name="ACTIVITY_READER_CONFIG" default="ActivityReaderConfig" />
<param name="ACTIVITY_WRITER_CONFIG" default="ActivityWriterConfig" />
<param name="WIZARD_CONFIG" default="defaultWizardConfig" />
<param name="SESSION_READER_CONFIG" default="DefaultSessionReaderConfig" />
<param name="SESSION_WRITER_CONFIG" default="DefaultSessionWriterConfig" />
<param name="APPLICATION_ICON_PATH" default="module_ui_icons/sight_logo.svg" />
</parameters>
<desc>Configuration to launch activities sequentially</desc>
......@@ -77,7 +77,7 @@
<!-- ************************************ Begin Objects declaration **************************************** -->
<object uid="${seriesDB}" type="sight::data::SeriesDB" src="ref"/>
<object uid="${seriesDB}" type="sight::data::SeriesDB" src="ref" />
<!-- ************************************* Begin layouts declaration *************************************** -->
......@@ -121,41 +121,40 @@
<!-- ************************************* Begin editor declaration *************************************** -->
<service uid="openButton" type="sight::module::ui::qt::com::SSignalButton" >
<service uid="openButton" type="sight::module::ui::qt::com::SSignalButton">
<config>
<icon>sight::module::ui::icons/open.svg</icon>
<iconWidth>40</iconWidth>
<iconHeight>40</iconHeight>
<icon>sight::module::ui::icons/open.svg</icon>
<iconWidth>40</iconWidth>
<iconHeight>40</iconHeight>
</config>
</service>
<service uid="saveButton" type="sight::module::ui::qt::com::SSignalButton" >
<service uid="saveButton" type="sight::module::ui::qt::com::SSignalButton">
<config>
<icon>sight::module::ui::icons/save.svg</icon>
<iconWidth>40</iconWidth>
<iconHeight>40</iconHeight>
<icon>sight::module::ui::icons/save.svg</icon>
<iconWidth>40</iconWidth>
<iconHeight>40</iconHeight>
</config>
</service>
<service uid="appLogoView" type="sight::module::ui::qt::image::SImage" >
<service uid="appLogoView" type="sight::module::ui::qt::image::SImage">
<path>${APPLICATION_ICON_PATH}</path>
<height>70</height>
</service>
<service uid="ircadIhuLogoView" type="sight::module::ui::qt::image::SImage" >
<path>flaticons/IrcadIHU.svg</path>
<service uid="ircadIhuLogoView" type="sight::module::ui::qt::image::SImage">
<path>sight::module::ui::flaticons/IrcadIHU.svg</path>
<height>70</height>
</service>
<!-- ************************************* Begin reader/writer declaration *************************************** -->
<!-- ************************************* Begin reader/writer declaration ********************************* -->
<service uid="SDBReader" type="sight::module::ui::base::io::SSelector" config="${ACTIVITY_READER_CONFIG}">
<service uid="SDBReader" type="sight::module::io::session::SReader" config="${SESSION_READER_CONFIG}">
<inout key="data" uid="${seriesDB}" />
</service>
<!-- Service to save the launched activities -->
<service uid="SDBWriter" type="sight::module::ui::base::io::SSelector" config="${ACTIVITY_WRITER_CONFIG}">
<inout key="data" uid="${seriesDB}" />
<service uid="SDBWriter" type="sight::module::io::session::SWriter" config="${SESSION_WRITER_CONFIG}">
<in key="data" uid="${seriesDB}" />
</service>
<!-- *************************************** Begin view declaration **************************************** -->
......@@ -178,7 +177,7 @@
</service>
<!-- Launch the activity sequentially -->
<service uid="activitySequencer" type="sight::module::ui::qt::activity::SSequencer" config="${SEQUENCER_CONFIG}" >
<service uid="activitySequencer" type="sight::module::ui::qt::activity::SSequencer" config="${SEQUENCER_CONFIG}">
<inout key="seriesDB" uid="${seriesDB}" autoConnect="true" />
</service>
......@@ -264,4 +263,4 @@
<update uid="activitySequencer" />
</config>
</extension>
</extension>
\ No newline at end of file
<plugin id="sight::config::ui::activity">
<requirement id="sight::module::io::session" />
<requirement id="sight::module::io::atoms" />
<requirement id="sight::module::activity" />
<requirement id="sight::module::io::activity" />
......
......@@ -17,6 +17,7 @@ add_dependencies(ExActivities
activity_viz_negato
config_ui_activity
module_viz_sample
module_viz_scene3d
)
moduleParam(module_ui_qt
......
......@@ -36,6 +36,8 @@
<parameter replace="WIZARD_CONFIG" by="${WIZARD_CONFIG}" />
<parameter replace="ICON_PATH" by="${ICON_PATH}" />
<parameter replace="WID_PARENT" by="activityView" />
<parameter replace="SESSION_READER_CONFIG" by="SessionReaderConfig" />
<parameter replace="SESSION_WRITER_CONFIG" by="SessionWriterConfig" />
</service>
<!-- ******************************* Start services ***************************************** -->
......
<!-- This configuration allows to read an image and display it in a simple scene -->
<extension implements="sight::service::extension::AppConfig" >
<id>VolumeRendering</id>
<parameters>
<param name="WID_PARENT" />
<param name="image" />
</parameters>
<config>
<!-- ******************************* Objects declaration ****************************** -->
<object uid="${image}" type="sight::data::Image" src="ref" />
<object uid="TF" type="sight::data::TransferFunction" src="deferred" />
<object uid="TFPool" type="sight::data::Composite" />
<!-- ******************************* UI declaration *********************************** -->
<service uid="mainView" type="sight::module::ui::base::SView" >
<gui>
<layout type="sight::ui::base::LineLayoutManager">
<orientation value="vertical" />
<view proportion="0" />
<view proportion="0" />
<view proportion="0" />
</layout>
</gui>
<registry>
<parent wid="${WID_PARENT}" />
<view sid="imageScene" start="true" />
<view sid="tfm" start="true" />
<toolBar sid="toolBarView" start="true" />
</registry>
</service>
<service uid="toolBarView" type="sight::module::ui::base::SToolBar" >
<gui>
<layout>
<menuItem name="Import image" icon="sight::module::ui::flaticons/BlueLoad.svg" />
</layout>
</gui>
<registry>
<menuItem sid="importAct" start="true" />
</registry>
</service>
<!-- ******************************* Actions ****************************************** -->
<service uid="importAct" type="sight::module::ui::base::io::SSelector" >
<inout key="data" uid="${image}" />
<type mode="reader" />
<selection mode="include" />
<addSelection service="::sight::module::io::vtk::SImageReader" />
</service>
<!-- ************************************* Services ************************************ -->
<service uid="imageScene" type="sight::viz::scene3d::SRender">
<scene overlay="true">
<background color="#36393E" />
<layer id="default" order="1" />
<adaptor uid="cameraInteractorAdaptor" />
<adaptor uid="volumeRender" />
</scene>
</service>
<service uid="cameraInteractorAdaptor" type="sight::module::viz::scene3d::adaptor::STrackballCamera" >
<config layer="default" priority="0" />
</service>
<service uid="volumeRender" type="sight::module::viz::scene3d::adaptor::SVolumeRender">
<inout key="image" uid="${image}" autoConnect="true" />
<inout key="tf" uid="TF" />
<config layer="default" widgets="false" dynamic="true" preintegration="false" />
</service>
<service uid="tfm" type="sight::module::ui::qt::image::STransferFunction" >
<inout key="tfPool" uid="TFPool" autoConnect="true"/>
<out key="tf" uid="TF" />
</service>
<!-- ******************************* Start services ***************************************** -->
<start uid="mainView" />
<start uid="cameraInteractorAdaptor" />
<start uid="volumeRender" />
<!-- ******************************* Update services ***************************************** -->
</config>
</extension>
......@@ -7,6 +7,7 @@
<xi:include href="configurations/ExActivitiesBase.xml" xmlns:xi="http://www.w3.org/2003/XInclude" />
<xi:include href="configurations/ImageReading.xml" xmlns:xi="http://www.w3.org/2003/XInclude" />
<xi:include href="configurations/VolumeRendering.xml" xmlns:xi="http://www.w3.org/2003/XInclude" />
<extension implements="::sight::activity::extension::Activity" >
<id>ExImage_Import_Activity</id>
......@@ -100,6 +101,26 @@
</config>
</extension>
<extension implements="sight::service::extension::Config">
<id>SessionReaderConfig</id>
<service>sight::module::io::session::SReader</service>
<desc>Session reader</desc>
<config>
<dialog extension=".sample" description="Sample Sight session file" policy="always"/>
<password policy="once" encryption="salted"/>
</config>
</extension>
<extension implements="sight::service::extension::Config">
<id>SessionWriterConfig</id>
<service>sight::module::io::session::SWriter</service>
<desc>Session writer</desc>
<config>
<dialog extension=".sample" description="Sample Sight session file" policy="once"/>
<password policy="once" encryption="salted"/>
</config>
</extension>
<extension implements="::sight::service::extension::AppConfigParameters" >
<id>ExActivities_AppCfgParam</id>
<parameters>
......
......@@ -55,7 +55,7 @@ public:
protected:
/// Parse the configuration
/// Parses the configuration
ACTIVITY_API virtual void parseConfiguration(
const ConfigurationType& config,
const InOutMapType& inouts = InOutMapType()
......
......@@ -51,12 +51,10 @@ inline static T xxcrypt(
// Initialize key (265 bits) and iv(128 bits)
unsigned char key[HASH_SIZE];
unsigned char iv[EVP_MAX_IV_LENGTH];
unsigned char iv[EVP_MAX_IV_LENGTH] = {0};
// Compute password hash and randomize initialization vector
hash(password, key);
const auto pseudo_random = SIGHT_PSEUDO_RANDOM_HASH("");
std::memcpy(iv, pseudo_random.data(), sizeof(iv));
// Initialize the output with size of the original message expanded to block size
T output;
......
......@@ -23,14 +23,12 @@
#include "core/config.hpp"
#include "core/crypto/secure_string.hpp"
#include "core/tools/System.hpp"
// Convenience macro to generate a pseudo random hash in a pseudo predictable way.
#define SIGHT_PSEUDO_RANDOM_HASH(salt) \
::sight::core::crypto::hash( \
::sight::core::crypto::secure_string( \
std::to_string(::sight::core::tools::System::getPID()) \
+ __FILE__ \
sight::core::crypto::hash( \
sight::core::crypto::secure_string( \
__FILE__ \
+ std::to_string(__LINE__) \
+ salt \
) \
......
......@@ -102,6 +102,22 @@ void SystemTest::robustRenameTest()
std::filesystem::filesystem_error
);
// 3. Should NOT throw an exception, even if the original file already exists.
// (re)create the fake file.
{
std::fstream fs;
fs.open(originFile, std::ios::out);
fs.close();
}
CPPUNIT_ASSERT_NO_THROW(core::tools::System::robustRename(destinationFile, originFile, true));
CPPUNIT_ASSERT_NO_THROW(core::tools::System::robustRename(originFile, destinationFile, true));
CPPUNIT_ASSERT_MESSAGE("Destination file should exist.", std::filesystem::exists(destinationFile));
CPPUNIT_ASSERT_MESSAGE("Origin file shouldn't exist", !std::filesystem::exists(originFile));
// 4. Should do nothing if both path are indeed the same file
CPPUNIT_ASSERT_NO_THROW(core::tools::System::robustRename(destinationFile, destinationFile));
// Clean up.
std::filesystem::remove(destinationFile);
}
......
......@@ -330,35 +330,52 @@ void System::cleanAllTempFolders(const std::filesystem::path& dir) noexcept
//------------------------------------------------------------------------------
void System::robustRename(const std::filesystem::path& _p1, const std::filesystem::path& _p2)
void System::robustRename(
const std::filesystem::path& _p1,
const std::filesystem::path& _p2,
bool _force
)
{
std::error_code renameError;
// First try a basic rename.
std::filesystem::rename(_p1, _p2, renameError);
if(renameError) // Error
// If both paths are indeed the same, do nothing
if(_p1.lexically_normal().compare(_p2.lexically_normal()) == 0)
{
// Handle the Invalid cross-device link case: _p1 & _p2 are not on the same disk/volume.
if(renameError == std::make_error_code(std::errc::cross_device_link))
return;
}
std::error_code error;
// Try a basic rename.
std::filesystem::rename(_p1, _p2, error);
if(error)
{
// Try to remove target if force is enabled
if(_force)
{
// Use a copy-remove scenario instead of the rename.
std::error_code copyError;
std::filesystem::copy(_p1, _p2, copyError);
if(!copyError) // Success
{
//Remove old file.
std::filesystem::remove(_p1); // throw an exception if it fails.
}
else // Error
std::filesystem::remove(_p2);
}
if(error == std::make_error_code(std::errc::cross_device_link))
{
std::filesystem::copy(_p1, _p2, error);
if(error == std::make_error_code(std::errc::operation_not_permitted))
{
throw std::filesystem::filesystem_error(copyError.message(), copyError);
// This happens on copying on different filesystem types, i.e. EXT4 -> NTFS
// In this case we use an alternative but less performant copy function
std::filesystem::remove(_p2);
std::ifstream src(_p1.string(), std::ios::binary);
std::ofstream dst(_p2.string(), std::ios::binary);
dst << src.rdbuf();
}
// Early return, copy-remove is done.
return;
std::filesystem::remove(_p1);
}
else
{
// This will throw the expected exception
std::filesystem::rename(_p1, _p2);
}
// Throw all others errors.
throw std::filesystem::filesystem_error(renameError.message(), renameError);
}
}
......
......@@ -39,6 +39,38 @@ class CORE_CLASS_API System
{
public:
/// Convenience class that deletes the associated temporary file on destruction
class TemporaryFile
{
public:
TemporaryFile(const TemporaryFile&) = delete;
TemporaryFile(TemporaryFile&&) = delete;
TemporaryFile& operator=(const TemporaryFile&) = delete;
TemporaryFile& operator=(TemporaryFile&&) = delete;
inline TemporaryFile() :
m_filePath(getTemporaryFolder() / genTempFileName())
{
}
inline ~TemporaryFile()
{
std::filesystem::remove(m_filePath);
}
//------------------------------------------------------------------------------
inline std::filesystem::path getTemporaryFilePath() const
{
return m_filePath;
}
private:
const std::filesystem::path m_filePath;
};
/**
* @brief Returns the system's temporary folder.
* Returns the value returned by std::filesystem::temp_directory_path, or
......@@ -92,9 +124,14 @@ public:
* @brief renames file or folder, use std::filesystem::rename first, use a copy-remove scenario if rename fails.
* @param _from source path of the file to rename.
* @param _to destination path of the renamed file.
* @param _force remove the destination in all cases.
* @throws std::filesystem_error if it fails.
*/
CORE_API static void robustRename(const std::filesystem::path& _from, const std::filesystem::path& _to);
CORE_API static void robustRename(
const std::filesystem::path& _from,
const std::filesystem::path& _to,
bool _force = false
);
/**
* @brief Sets the temporary folder prefix.
......
......@@ -53,6 +53,16 @@ public:
SIGHT_DECLARE_SERVICE(IReader, sight::service::IService);
/// Enum to define a password policy
enum class DialogPolicy : uint8_t
{
NEVER = 0, /// Never use show the dialog
ONCE = 1, /// Show only once, store the location as long as the service is started
ALWAYS = 2, /// Always show the location dialog
DEFAULT = ALWAYS, /// Default behavior if nothing is set
INVALID = 255 /// Used for error management
};
/**
* @name Slots API
* @{
......@@ -158,6 +168,51 @@ public:
//@}
/// Convenience function to convert from dialogPolicy enum value to string
constexpr static std::string_view dialogPolicyToString(DialogPolicy policy) noexcept
{
switch(policy)
{
case DialogPolicy::NEVER:
return "never";
case DialogPolicy::ONCE:
return "once";
case DialogPolicy::ALWAYS:
return "always";
default:
return "default";
}
}
/// Convenience function to convert from string to PasswordPolicy enum value
constexpr static DialogPolicy stringToDialogPolicy(std::string_view policy) noexcept
{
if(constexpr auto NEVER = dialogPolicyToString(DialogPolicy::NEVER); policy == NEVER)
{
return DialogPolicy::NEVER;
}
else if(constexpr auto ONCE = dialogPolicyToString(DialogPolicy::ONCE); policy == ONCE)
{
return DialogPolicy::ONCE;
}
else if(constexpr auto ALWAYS = dialogPolicyToString(DialogPolicy::ALWAYS); policy == ALWAYS)
{
return DialogPolicy::ALWAYS;
}
else if(policy.empty() || policy == "default")
{
return DialogPolicy::DEFAULT;
}
else
{
// Error case
return DialogPolicy::INVALID;
}
}
protected:
IO_BASE_API IReader() noexcept;
......
......@@ -54,6 +54,16 @@ public:
SIGHT_DECLARE_SERVICE(IWriter, sight::service::IService);
/// Enum to define a password policy
enum class DialogPolicy : uint8_t
{
NEVER = 0, /// Never use show the dialog
ONCE = 1, /// Show only once, store the location as long as the service is started
ALWAYS = 2, /// Always show the location dialog
DEFAULT = ALWAYS, /// Default behavior if nothing is set
INVALID = 255 /// Used for error management
};
/**
* @name Slots API
* @{
......@@ -152,6 +162,51 @@ public:
/// Returns if reading has been cancelled by user
IO_BASE_API bool hasFailed() const;
/// Convenience function to convert from dialogPolicy enum value to string
constexpr static std::string_view dialogPolicyToString(DialogPolicy policy) noexcept
{
switch(policy)
{
case DialogPolicy::NEVER:
return "never";
case DialogPolicy::ONCE:
return "once";
case DialogPolicy::ALWAYS:
return "always";
default:
return "default";
}
}
/// Convenience function to convert from string to PasswordPolicy enum value
constexpr static DialogPolicy stringToDialogPolicy(std::string_view policy) noexcept