Design and Implementation of a Custom Real-Time Operating System (RTOS) for Enhanced Safety and Performance in Robotic Systems

SAFEBOT Program Overview:

Collaborative robots, or cobots, promise to revolutionize the manufacturing floor, offering enhanced flexibility and enabling close proximity to human-robot interactions. These cobots blend human adaptability with robotic efficiency, supporting evolving trends in mass customization and agile supply chains while alleviating repetitive tasks for workers. However, traditional industrial robots, due to their speed, outperform cobots in productivity. This disparity is largely rooted in safety concerns when robots operate near humans.

Guidelines like ISO/TS 15066 detail the safety protocols for cobots, including distinct operation modes. For example, the Speed and Separation Monitoring mode ensures a set distance between humans and robots, but this limits the robot's speed, affecting productivity. Similarly, Power and Force Limiting allows human-robot contact, but constraints on speed and payload to prevent injury affect its operational efficiency.

At SAFEBOT, our mission is to bridge this gap. We're pioneering an on-robot sensing and augmentation system that facilitates real-time adaptation, using technology like RaDAR, StereoVision and Time of Flight. For human-robot collaboration, our technology aims to enhance cobots' environmental awareness. This ensures efficient task completion without compromising on production speed or safety, addressing the current challenges faced by industry cobots.

The need for RTOS in collaborative robotics:

In the rapidly evolving domain of robotic applications, especially when dealing with distributed multimodal proximity safety systems, there is an intrinsic need for systems that offer the robust guarantees of a Real-Time Operating System (RTOS), but which are versatile enough to be deployed across a broad spectrum of processor architectures, integrating heterogeneous processing elements like CPUs, GPUs, and FPGAs. This flexibility is crucial given the high computational demands of contemporary robotic sensory tasks, which involve interfacing with a diverse array of sensor devices (from cameras to LIDARs).

Moreover, these deployments must also be dynamic and adaptable according to the context (i.e. stationary, mobile deployment), tasks and the environment (i.e. when light is scarce, or bright sunlight or foggy, sensor openings are partially dirty) perceived by the cobot. Depending on the dynamic demand, all the signals produced by the sensory subsystems of the cobot are sent to other sensory subsystems or to the central cognitive processor for further processing.

The challenge of harmonizing safety, efficiency and adaptability in cobots mirrors the mixed-criticality paradigm in real-time systems. A potential approach would be to tap into this paradigm, allowing the student to draw insights and, after that, design an RTOS rooted in this synergy. There exists a significant body of work on this subject. For instance, studies on mixed-criticality systems by Esper et al. [1] or the foundational theory on "unrelated platforms" model for real-time systems as proposed by Baruah [2] and later expanded by Bertout et al. [3, 4] provide essential groundwork. Additionally, research into real-time shared memory management, as seen in works by Rivas et al. [5] and Schuh et al. [6], offers pivotal insights into optimizing multicore real-time systems for safety.

Yet, the practical applications of these theories face hurdles. COTS vendors, such as Xilinx, and proprietary solutions like CUDA, often provide APIs and programming models that are complex and non-transparent, thereby muddling development. These systems are often "optimized for average", lacking the precise real-time guarantees necessary for critical applications. As a result, we see a trend wherein the hardware's inherent capabilities are underutilized, often prompting developers to offload computation to larger computers, which compromises the lean form factor required for stationary and mobile robotic platforms. The current literature, while vast, lags in addressing these real-world nuances, with a bias often towards approaches that offload computations to the cloud. These approaches aren't feasible for collaborative robots: the induced latencies when processing large amounts of perception data become too high, too unpredictable and too energy-consuming for the safety-critical aspect of the robot.

PhD Objectives:

Given these challenges, our proposal aims to construct an RTOS framework anchored in Linux-RT, explicitly designed to simplify the development of intelligent safety proximity perception systems for collaborative robots. This framework would prioritize tapping into the full potential of heterogeneous platforms. Crucially, it will feature well-structured APIs that not only support the computational requirements of robotic workloads but also activate hardware mechanisms and uphold strict timing prerequisites, ensuring optimal performance in tandem with safety. The framework will offer modular components that can be combined to suit different kinds of sensors and robot morphologies.

To maximize the potential of heterogeneous platforms, the student will develop an RTOS framework that:

1. Facilitates the development processes by offering well-defined APIs optimized for distributed multi-sensor proximity safety systems for human-robot collaboration.
2. Enhances performance by harnessing the power of heterogeneous hardware mechanisms to boost cobots' efficiency in real-world scenarios.
3. Reinforces safety by ensuring strict timing requirements are consistently met, fortifying the trustworthiness of cobot-human collaborations.

The student will be expected to produce high-calibre research articles targeting premier robotics journals and real-time systems venues. This ensures that the research findings significantly contribute to and resonate within the global robotics and system communities, potentially open-sourcing (some parts of) the framework.

References

[1] Alexandre Esper, Geoffrey Nelissen, Vincent Nélis, and Eduardo Tovar. "An Industrial View on the Common Academic Understanding of Mixed-Criticality Systems." Real-Time Systems, 54(3):745–795, Jul 2018.

[2] S. Baruah. "Feasibility analysis of preemptive real-time systems upon heterogeneous multiprocessor platforms." 2004, 25th IEEE International Real-Time Systems Symposium.

[3] Antoine Bertout, Joël Goossens, Emmanuel Grolleau, and Xavier Poczekajlo. "Workload assignment for global real-time scheduling on unrelated multicore platforms." Proceedings of the 28th International Conference on Real-Time Networks and Systems (RTNS 2020).

[4] A. Bertout, J. Goossens, E. Grolleau and X. Poczekajlo. "Template schedule construction for global real-time scheduling on unrelated multiprocessor platforms." 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[5] Rivas, Juan M.; Goossens, Joël; Poczekajlo, Xavier; Paolillo, Antonio. "Implementation of Memory Centric Scheduling for COTS Multicore Real-Time Systems." ECRTS 2019.

[6] M. Schuh, C. Maiza, J. Goossens, P. Raymond and B. D. de Dinechin. "A study of predictable execution models implementation for industrial data-flow applications on a multicore platform with shared banked memory." 2020 IEEE Real-Time Systems Symposium (RTSS), Houston, TX, USA, 2020, pp. 283-295.