Eclipse Zenoh: Supporting ICOS’s Data Distribution from the Cloud-to-Thing continuum

The ICOS Software platform aims to cover the range of challenges that arise when addressing the Cloud-Edge-IoT continuum paradigm, proposing an approach that embeds a well-defined set of functionalities, culminating in the definition of an IoT2cloud operating system. The ICOS agent is the basic ICOS software component that needs to be installed on each infrastructure node capable of executing external software and/or providing data and metadata. ICOS nodes are any resource physical or virtual that is running an ICOS agent. ICOS nodes are by nature heterogeneous with several degrees of freedom:

• Computing power: data center server vs end-user device vs embedded sensors
• Energy requirements: devices may be battery-powered
• Mobility requirements: on-board units embedded in cars/trains, tractors
• Traffic patterns: sensors producing large amount of data towards the network (push up) vs servers serving data/video (push down)
• Activity pattern always on vs long sleep mode

ICOS platform must be able to gather the events and map them to controls of specific standards and/or to specific policies and rules that are stored on server-grade hardware and networks, as well as events triggered by microcontroller and constrained networks. Traditional messaging protocols, such as MQTT, AMQP, DDS, and CoAP share a common limitation - the opportunity for single point of failure due to client-server star topology of message flows ¹.

In the context of the WP4 (M06-M36) Development of the Intelligence Layer, Security Layer and Data Management Modules, ZettaScale leads the task on APIs and Interfaces, and will enable:

1. better collaboration between modules and apps, enabling task automation and data workflows to avoid data silos,
2. flexibility allowing the connection of different sources of data and supporting different formats/encodings, and
3. support for different strategies of reliability and congestion control required to balance between ICOS applications needs and systems constraints, thus allowing fine tuning and trade-offs between reliability, scalability, and resource consumption.

ICOS will leverage on the Eclipse Zenoh² project, which is a Pub/Sub/Query protocol to provide a set of unified abstractions to deal with data-in-motion, data-at-rest and computations at internet scale. Eclipse Zenoh is agnostic to the underlying technology, implementing a networking layer capable of running above a Data Link, Network, or Transport layers of the OSI stack. Eclipse Zenoh also runs on a heterogenous devices that includes a multitude of embedded devices all the way to the cloud.

Reliability, scalability and congestion control trade-offs

Providing many to many reliable communications over a wide area network requires either an infinite amount of memory or giving up progress. Thus, trade-offs need to be made regarding reliability for the system to work, scale smoothly and have sufficiently strong guarantees. System’s heterogeneity, with respect to network capabilities and node resources, has important consequences and requires proper strategies to avoid a slow node to impact the entire system.

In a reliable system, there is a constant trade-off between reliability, progress and memory consumption. Indeed a slow or non-responsive data consumer will force matching data producers to either:

• Store more and more messages thus consuming more and more memory.
• Drop messages thus losing reliability.
• Block new writes thus losing progress.

ICOS platform will rely on Eclipse Zenoh to provide different mechanisms of reliability and congestion control. It will enable users to clearly define the right level of control required to balance between applications needs and systems constraints, thus allowing users to operate under the right trade-offs between reliability, scalability, resource consumption and progress.

Dissemination and Open-source activities

Besides the planned technical contribution, ZettaScale has participated in the The Open Source Task Force³ representing ICOS, where six projects currently involved in the cluster of RIAs projects presented their status. The meeting was designed to bring together and begin work together. Each project brought their architectures and technical expertise with the Technical Managers of the cluster.

ZettaScale also participated in the World FIRA 2023⁴ where our team presented a series of demos on how to make agricultural robots and vehicles communicate easier, smarter and more efficient. This expertise will be beneficial to ICOS in the context of UC1: Agriculture Operational Robotic Platform (AORP) which is lead by L-PIMR and PSNC partners.