enh(build): minimalistic tool to build sight-dependent projects
Description
All sight
-like projects share a similar structure.
Currently, we do not have a unified way to compile these projects. Some use custom cmake-presets
, some use VSCode
stuff, and some like me build manually with ccmake && ninja
.
On the other hand, the research division isn't as versed in the ways of compiling.
It's usually a bit hard for them to compile a sight
-like project.
Proposal
I've been doing this on my free time, so I thought I might as well share this simple tool. Should you want to use CMake presets instead, you may perhaps take inspiration from this (like the build tree organisation).
I propose to use a simple Python script and a YML
file that could go in a ./sight-build/submodules.yml
(and really effectively just calls cmake -S ... -B ... -D...
followed by cmake --build ... --target all install
with what you gave as argument).
Linux
.
sight-compile
The YML
would look like that:
submodules:
sight:
deps: ""
cmake-options: ""
sight2:
deps:
- sight
cmake-options: ""
Here is an example of submodules.yml
that would go in a /.sight-build
directory.
As-is, the script call looks like:
./sight-compile --source /home/you/git/sight-foo --build /home/you/builds/sight-foo --type all --cmake-options "-DCCACHE=1" --clear all
With -h
printing:
usage: sight-compile [-h] -src SOURCE -b BUILD -t {release,debug,all} [-g GENERATOR] [-cmakeop CMAKE_OPTIONS]
[-cl {install,none,all}] [--configure {none,main,all,deps}]
#-------------------------------------------------------------------------
sight-compile is a minimalistic Python script to compile sight-like repositories.
Its goal is to provide a simple interface to set up build directories for the project and its dependencies, generally added as a submodule.
Using this script generates the following tree:
build-directory-you-gave
├── release
│ ├── build
│ └── install
├── debug
│ ├── build
│ └── install
└── deps
├── release
│ ├── a_dependency
│ │ ├── build
│ │ └── install
│ └── another_dependency
│ ├── build
│ └── install
└── debug
├── a_dependency
│ ├── build
│ └── install
└── another_dependency
├── build
└── install
Appropriate CMake configure/generate/build commands are then issued in each directory.
To register submodules, add a .sight-build/submodules.yml in the root directory of your project.
The format for this file must be:
submodules:
sight:
deps: ""
cmake-options: ""
not-sight:
deps:
- sight
cmake-options: ""
# ...
#-------------------------------------------------------------------------
options:
-h, --help show this help message and exit
-src SOURCE, --source SOURCE
Path to the root directory of the project, e.g. /home/you/git/the-project.
-b BUILD, --build BUILD
Path to the build directory, e.g. /home/you/builds.
-t {release,debug,all}, --type {release,debug,all}
Build types.
-g GENERATOR, --generator GENERATOR
Generator to use for CMake. Default is Ninja.
-cmakeop CMAKE_OPTIONS, --cmake_options CMAKE_OPTIONS
CMake options for the main project.
-cl {install,none,all}, --clear {install,none,all}
Clear the target directory of the builds of the project. 'none': don't touch the directory.
'install': only remove installs. 'all': remove everything. When selecting 'all', you will be
prompted to validate this action.
--configure {none,main,all,deps}
Do the CMake configure step for the project and/or its dependencies. Default is 'all'.
#-------------------------------------------------------------------------
EXAMPLE OF USAGE:
./sight-compile --source /home/you/git/sight-foo --build /home/you/builds/sight-foo --type all --cmake-options "-DCCACHE=1" --clear all
Functional specifications
Proposal
As per Since non-devs are mostly familiar with Python, I propose to provide this minimalistic tool in this language. As you obviously know, it's also cross-platform, which means less code to maintain compared to a bash / PowerShell approach.
From the user point of view:
- A simple call should suffice to build, in particular, non-dev wanting to build from source should be able to get things done with
./sight-compile --source /home/you/git/sight-foo --build /home/you/builds/sight-foo --type release
. Ideally, it should be intuitive to use even for people not at ease with CMake (again, the AI division). - This tool should work for all
sight
-like projects, includingsight
. - Should allow building (almost) anywhere, assuming the user has the required permissions.
- Should be possible to set CMake options both for the project and for the submodules it depends on with minimal overhead. In particular, it should be possible to NOT build tests and examples unless explicitly specifying overwise (this time, this is certainly more useful for devs).
- Windows and Linux users should be able to use it.
We sometimes need to erase old builds (when updating from a very, very old branch, for example). It should be possible to do it with this script.
- Should allow clearing the old build, either completely or only the
install
s. - Big red warning when requiring to clear all the build directory.
Technical specifications
Proposal
As per I'd rather avoid reinventing the wheel too much. This sort of tool is even some kind of redundant with cmake-presets
.
These still require a certain form of comprehension of CMake, and do not work very well with submodules yet.
In the end, what we really want to do is really just calling cmake -B ... -S ... -D...
followed by cmake --build ... --target all install
in the right directories the correct number of times. This tool should do this, and only this. I stress again, inventing a new abstraction on CMake is definitely not a good idea.
For submodules, the build directory can be deduced (actually, selected from) what the user selected. The only additional input potentially required is something to indicate what folder is a submodule, the dependencies between them and the CMake options (3 parameters per submodule).
- Specify submodules to build with a
YML
file, located in a hidden directory.sight-build
submodules:
sight:
deps: ""
cmake-options: "-DSIGHT_BUILD_TESTS=OFF -DSIGHT_BUILD_EXAMPLES=OFF"
not-night:
deps:
- sight
cmake-options: "-DSIGHT_SOME_OPTION=OFF"
The main project may also need some -D
to pass to CMake. These can simply be passed as string as ./sight-build --cmake-options "-DSIGHT_ENABLE_FOO=1 -DSIGHT_DISABLE_BAR=1"
etc.
- NOTE: Upon clearing directories, systematically check if the path still exists. The user may have started deleting stuff manually.
Test plan
Proposal
As per - Test various repositories, including
sight
. - Ask people in the DL department if this is intuitive enough for them to use.