Adding support for other sandboxes (namely recodex-guardian).

CMakeLists.txt

@@ -1,6 +1,6 @@
 cmake_minimum_required(VERSION 3.11.0)
 project(recodex-worker)
-set(RECODEX_VERSION 1.9.0)
+set(RECODEX_VERSION 2.0.0)
 enable_testing()
 
 set(EXEC_NAME ${PROJECT_NAME})
@@ -135,6 +135,9 @@ set(SOURCE_FILES
 	${FILEMAN_DIR}/prefixed_file_manager.h
 
 	${SANDBOX_DIR}/sandbox_base.h
+	${SANDBOX_DIR}/sandbox_base.cpp
+	${SANDBOX_DIR}/guardian_sandbox.h
+	${SANDBOX_DIR}/guardian_sandbox.cpp
 	${SANDBOX_DIR}/isolate_sandbox.h
 	${SANDBOX_DIR}/isolate_sandbox.cpp
 
@@ -156,8 +159,8 @@ set(SOURCE_FILES
 	${TASKS_DIR}/internal/mkdir_task.cpp
 	${TASKS_DIR}/internal/rm_task.h
 	${TASKS_DIR}/internal/rm_task.cpp
-	${TASKS_DIR}/internal/archivate_task.h
-	${TASKS_DIR}/internal/archivate_task.cpp
+	${TASKS_DIR}/internal/archive_task.h
+	${TASKS_DIR}/internal/archive_task.cpp
 	${TASKS_DIR}/internal/extract_task.h
 	${TASKS_DIR}/internal/extract_task.cpp
 	${TASKS_DIR}/internal/fetch_task.h

README.md

@@ -1,7 +1,6 @@
 # Worker
 
 [![Linux Build Status](https://github.com/ReCodEx/worker/workflows/Linux%20Build/badge.svg)](https://github.com/ReCodEx/worker/actions)
-[![Windows Build Status](https://github.com/ReCodEx/worker/workflows/Windows%20Build/badge.svg)](https://github.com/ReCodEx/worker/actions)
 [![codecov](https://codecov.io/gh/ReCodEx/worker/branch/master/graph/badge.svg?token=AYHQA9R8PJ)](https://codecov.io/gh/ReCodEx/worker)
 [![License](http://img.shields.io/:license-mit-blue.svg)](http://badges.mit-license.org)
 [![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg)](http://recodex.github.io/worker/)
@@ -45,11 +44,8 @@ use. The package names are for CentOS if not specified otherwise.
   (`libzmq3-dev` on Debian)
 - YAML-CPP library, `yaml-cpp` and `yaml-cpp-devel` (`libyaml-cpp0.5v5` and
   `libyaml-cpp-dev` on Debian)
-- libcurl library `libcurl-devel` (`libcurl4-gnutls-dev` on Debian)
-- libarchive library as optional dependency. Installing will speed up build
-  process, otherwise libarchive is built from source during installation.
-  Package name is `libarchive` and `libarchive-devel` (`libarchive-dev` on
-  Debian)
+- libcurl library `libcurl-devel` (`libcurl4-dev` on Debian)
+- libarchive library as optional dependency. Installing will speed up build process, otherwise libarchive is built from source during installation. Package name is `libarchive` and `libarchive-devel` (`libarchive-dev` on Debian)
 
 **Isolate** (only for Linux installations)
 
@@ -125,21 +121,13 @@ manageable in long term horizon.
 
 #### Install worker on Windows
 
-There are basically two main dependencies needed, **Windows 7** or higher and
-**Visual Studio 2019+**. There is a simple installation batch script provided 
-which should do all the work on Windows machine. The script uses MsBuild from 
-VS2019 and 64-bit compilation, if you wish to use different compile option,
-please revisit mentioned script.
+We found no practical applications to run worker on Windows since we rely solely on linux-based (cgroups, namespaces, etc.) sandboxing. Therefore, it was decided to drop Windows support in 2026. The cmake and the code is still maintained in a way, that windows build is possible, but we no longer test it.
 
-The script is placed in *install* directory alongside supportive scripts for
-UNIX systems and is named *win-build.cmd*. Provided script will do almost
-all the work connected with building and dependency resolving (using
-**NuGet** package manager and `msbuild` building system). Script should be
-run under 64-bit version of _Developer Command Prompt for VS2019_ and from
-*install* directory.
+There is a batch script that should help you started with the build process and some old notes are below:
 
-Building and installing of worker is then quite simple, script has command line
-parameters which can be used to specify what will be done:
+The script is placed in *install* directory alongside supportive scripts for UNIX systems and is named *win-build.cmd*. Provided script will do almost all the work connected with building and dependency resolving (using **NuGet** package manager and `msbuild` building system). Script should be run under 64-bit version of _Developer Command Prompt for VS2019_ and from *install* directory.
+
+Building and installing of worker is then quite simple, script has command line parameters which can be used to specify what will be done:
 
 - *-build* -- It is the default options if none specified. Builds worker and its
   tests, all is saved in *build* folder and subfolders.
@@ -157,11 +145,8 @@ install> win-build.cmd -test
 install> win-build.cmd -package
 ```
 
-All build binaries and cmake temporary files can be found in *build* folder,
-classically there will be subfolder *Release* which will contain compiled
-application with all needed dlls. Once if clickable installation binary is
-created, it can be found in *build* folder under name 
-*recodex-worker-VERSION-x64.exe*.
+All build binaries and cmake temporary files can be found in *build* folder, classically there will be subfolder *Release* which will contain compiled application with all needed dlls. Once if clickable installation binary is created, it can be found in *build* folder under name *recodex-worker-VERSION-x64.exe*.
+
 
 #### Usage
 

examples/config.yml

@@ -35,6 +35,10 @@ logger:
     level: "debug"  # level of logging - one of "debug", "warn", "emerg"
     max-size: 1048576  # 1 MB; max size of file before log rotation
     rotations: 3  # number of rotations kept
+sandbox: "recodex-guardian"  # if not specifies, recodex-guardian is used by default; "isolate" is also supported
+sandbox-cpuset:  # optional CG cpuset configuration for sandbox (relevant only for recodex-guardian)
+    cpus: "0-3"  # list/range of CPUs to be used by the sandbox (empty/missing = all)
+    numa-nodes: "0"  # list/range of NUMA nodes to be used by the sandbox (empty/missing = all)
 limits:
     time: 30            # seconds
     wall-time: 30       # seconds
@@ -58,12 +62,12 @@ limits:
         # PYTHONHASHSEED: 0
         # PYTHONIOENCODING: utf-8
         # VIRTUAL_ENV: /path/to/python/venv
-    bound-directories: # All aditional dirs mapped into sandbox (SDKs, libs, venvs, ...), essential dirs (bin, dev, lib, usr, ...) are mapped automatically
+    bound-directories: # All additional dirs mapped into sandbox (SDKs, libs, venvs, ...), essential dirs (bin, dev, lib, usr, ...) are mapped automatically
         - dst: "/tmp"
           mode: "tmp"  # if mode is omitted, dir is mounted as read-only (which is typical for lib dirs)
         #- src: "/usr/share/something"  # actually existing directory
         #  dst: "/something"            # where the directory will be mounted in sandbox fs
-        #  mode: "rw"  # multiple modes can be separated by comma, see http://www.ucw.cz/moe/isolate.1.html (directory rules - options)
+        #  mode: "rw"  # multiple modes can be separated by comma
 max-output-length: 4096  # in bytes
 max-carboncopy-length: 1048576  # in bytes
 cleanup-submission: false  # if true, then folders with data concerning submissions will be cleared after evaluation, should be used carefully, can produce huge amount of used disk space

examples/job-config.yml

@@ -38,7 +38,7 @@ tasks:
             - "test"
             - "main.c"
       sandbox:  # if defined task is external and will be run in sandbox
-        name: "isolate"  # mandatory information
+        name: ""  # empty string means default sandbox (set in config, recodex-guardian is the fallback)
         limits:  # if not defined, then worker default configuration of limits is loaded
              # anything of the specified limits can be omitted and will be loaded from worker defaults
             - hw-group-id: "group1"  # determines specific limits for specific machines
@@ -68,7 +68,7 @@ tasks:
           - [100, 200]  # an interval (inclusive)
           # codes 0, 1, 100, 101, ... 199, and 200 will all be accepted
       sandbox:
-        name: "isolate"
+        name: ""
         limits:
             - hw-group-id: "group1"  # determines specific limits for specific machines
               time: 1  # seconds

recodex-worker.spec

@@ -1,7 +1,7 @@
 %define name recodex-worker
 %define short_name worker
-%define version 1.9.1
-%define unmangled_version 885c6bb4b3fa8e21400636f1bee1aefed19956e9
+%define version 2.0.0
+%define unmangled_version e66ddc766c796cda748c6af47ea2c6eef2dcda46
 %define release 1
 
 %define spdlog_name spdlog
@@ -18,7 +18,7 @@ Prefix: %{_prefix}
 Vendor: Petr Stefan <UNKNOWN>
 Url: https://github.com/ReCodEx/worker
 BuildRequires: systemd gcc-c++ cmake zeromq-devel cppzmq-devel yaml-cpp-devel libcurl-devel libarchive-devel boost-devel
-Requires: systemd isolate
+Requires: systemd
 
 #Source0: %{name}-%{unmangled_version}.tar.gz
 Source0: https://github.com/ReCodEx/%{short_name}/archive/%{unmangled_version}.tar.gz#/%{short_name}-%{unmangled_version}.tar.gz

src/config/sandbox_config.h

@@ -16,59 +16,77 @@ class sandbox_config
 	 * Name of sandbox which will be used.
 	 */
 	std::string name = "";
+
 	/**
 	 * Redirect standard input from given file.
 	 * @note Path must be accessible from inside of sandbox.
 	 */
 	std::string std_input = "";
+
 	/**
 	 * Redirect standard output to given file.
 	 * @note Path must be accessible from inside of sandbox.
 	 */
 	std::string std_output = "";
+
 	/**
 	 * Redirect standard error output to given file.
 	 * @note Path must be accessible from inside of sandbox.
 	 */
 	std::string std_error = "";
+
 	/**
 	 * If true then stderr is redirected to stdout.
 	 */
 	bool stderr_to_stdout = false;
+
 	/**
 	 * If true then stdout and stderr will be written in the results.
 	 */
 	bool output = false;
+
 	/**
 	 * File to which stdout will be copied after execution.
 	 * Global worker limit for carboncopies is applied.
 	 * @note Path is outside the sandbox.
 	 */
 	std::string carboncopy_stdout = "";
+
 	/**
 	 * File to which stderr will be copied after execution.
 	 * Global worker limit for carboncopies is applied.
 	 * @note Path is outside the sandbox.
 	 */
 	std::string carboncopy_stderr = "";
+
 	/**
 	 * Change working directory to subdirectory inside the sandbox.
 	 * @note Path must be accessible from inside of sandbox.
 	 */
 	std::string chdir = "";
+
 	/**
 	 * Working directory relative to the directory with the source files.
 	 */
 	std::string working_directory = "";
+
 	/**
 	 * Associative array of loaded limits with textual index identifying its hw group.
 	 */
 	std::map<std::string, std::shared_ptr<sandbox_limits>> loaded_limits;
 
+	/**
+	 * List of CPU cores to be used by the sandbox (empty = all).
+	 * See worker_config::sandbox_cpus_ for more details.
+	 */
+	std::string cpus = "";
 
 	/**
-	 * Constructor with defaults.
+	 * List of NUMA nodes to be used by the sandbox (empty = all).
+	 * See worker_config::sandbox_numa_nodes_ for more details.
 	 */
+	std::string numa_nodes = "";
+
 	sandbox_config() = default;
 };