rfc9556.original   rfc9556.txt 
Network Working Group J. Hong Internet Research Task Force (IRTF) J. Hong
Internet-Draft ETRI Request for Comments: 9556 ETRI
Intended status: Informational Y.-G. Hong Category: Informational Y-G. Hong
Expires: 18 March 2024 Daejeon University ISSN: 2070-1721 Daejeon University
X. de Foy X. de Foy
InterDigital Communications, LLC InterDigital Communications, LLC
M. Kovatsch M. Kovatsch
Huawei Technologies Duesseldorf GmbH Huawei Technologies Duesseldorf GmbH
E. Schooler E. Schooler
Intel University of Oxford
D. Kutscher D. Kutscher
Hong Kong University of Science and Technology (Guangzhou) HKUST(GZ)
15 September 2023 March 2024
IoT Edge Challenges and Functions Internet of Things (IoT) Edge Challenges and Functions
draft-irtf-t2trg-iot-edge-10
Abstract Abstract
Many Internet of Things (IoT) applications have requirements that Many Internet of Things (IoT) applications have requirements that
cannot be satisfied by traditional cloud-based systems (i.e., cloud cannot be satisfied by centralized cloud-based systems (i.e., cloud
computing). These include time sensitivity, data volume, computing). These include time sensitivity, data volume,
connectivity cost, operation in the face of intermittent services, connectivity cost, operation in the face of intermittent services,
privacy, and security. As a result, IoT is driving the Internet privacy, and security. As a result, IoT is driving the Internet
toward edge computing. This document outlines the requirements of toward edge computing. This document outlines the requirements of
the emerging IoT Edge and its challenges. It presents a general the emerging IoT edge and its challenges. It presents a general
model and major components of the IoT Edge to provide a common basis model and major components of the IoT edge to provide a common basis
for future discussions in the T2TRG and other IRTF and IETF groups. for future discussions in the Thing-to-Thing Research Group (T2TRG)
This document is a product of the IRTF Thing-to-Thing Research Group and other IRTF and IETF groups. This document is a product of the
(T2TRG). IRTF T2TRG.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This document is not an Internet Standards Track specification; it is
provisions of BCP 78 and BCP 79. published for informational purposes.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Research Task Force
and may be updated, replaced, or obsoleted by other documents at any (IRTF). The IRTF publishes the results of Internet-related research
time. It is inappropriate to use Internet-Drafts as reference and development activities. These results might not be suitable for
material or to cite them other than as "work in progress." deployment. This RFC represents the consensus of the Thing-to-Thing
Research Group of the Internet Research Task Force (IRTF). Documents
approved for publication by the IRSG are not candidates for any level
of Internet Standard; see Section 2 of RFC 7841.
This Internet-Draft will expire on 18 March 2024. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc9556.
Copyright Notice Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents
license-info) in effect on the date of publication of this document. (https://trustee.ietf.org/license-info) in effect on the date of
Please review these documents carefully, as they describe your rights publication of this document. Please review these documents
and restrictions with respect to this document. carefully, as they describe your rights and restrictions with respect
to this document.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Background
2.1. Internet of Things (IoT) . . . . . . . . . . . . . . . . 3 2.1. Internet of Things (IoT)
2.2. Cloud Computing . . . . . . . . . . . . . . . . . . . . . 4 2.2. Cloud Computing
2.3. Edge Computing . . . . . . . . . . . . . . . . . . . . . 4 2.3. Edge Computing
2.4. Examples of IoT Edge Computing Use Cases . . . . . . . . 6 2.4. Examples of IoT Edge Computing Use Cases
3. IoT Challenges Leading Towards Edge Computing . . . . . . . . 10 3. IoT Challenges Leading toward Edge Computing
3.1. Time Sensitivity . . . . . . . . . . . . . . . . . . . . 10 3.1. Time Sensitivity
3.2. Connectivity Cost . . . . . . . . . . . . . . . . . . . . 10 3.2. Connectivity Cost
3.3. Resilience to Intermittent Services . . . . . . . . . . . 11 3.3. Resilience to Intermittent Services
3.4. Privacy and Security . . . . . . . . . . . . . . . . . . 11 3.4. Privacy and Security
4. IoT Edge Computing Functions . . . . . . . . . . . . . . . . 11 4. IoT Edge Computing Functions
4.1. Overview of IoT Edge Computing Today . . . . . . . . . . 12 4.1. Overview of IoT Edge Computing
4.2. General Model . . . . . . . . . . . . . . . . . . . . . . 14 4.2. General Model
4.3. OAM Components . . . . . . . . . . . . . . . . . . . . . 17 4.3. OAM Components
4.3.1. Resource Discovery and Authentication . . . . . . . . 17 4.3.1. Resource Discovery and Authentication
4.3.2. Edge Organization and Federation . . . . . . . . . . 18 4.3.2. Edge Organization and Federation
4.3.3. Multi-Tenancy and Isolation . . . . . . . . . . . . . 19 4.3.3. Multi-Tenancy and Isolation
4.4. Functional Components . . . . . . . . . . . . . . . . . . 19 4.4. Functional Components
4.4.1. In-Network Computation . . . . . . . . . . . . . . . 19 4.4.1. In-Network Computation
4.4.2. Edge Storage and Caching . . . . . . . . . . . . . . 21 4.4.2. Edge Storage and Caching
4.4.3. Communication . . . . . . . . . . . . . . . . . . . . 21 4.4.3. Communication
4.5. Application Components . . . . . . . . . . . . . . . . . 22 4.5. Application Components
4.5.1. IoT Device Management . . . . . . . . . . . . . . . . 23 4.5.1. IoT Device Management
4.5.2. Data Management and Analytics . . . . . . . . . . . . 23 4.5.2. Data Management and Analytics
4.6. Simulation and Emulation Environments . . . . . . . . . . 24 4.6. Simulation and Emulation Environments
5. Security Considerations . . . . . . . . . . . . . . . . . . . 25 5. Security Considerations
6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 25 6. Conclusion
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 7. IANA Considerations
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 8. Informative References
9. Informative References . . . . . . . . . . . . . . . . . . . 26 Acknowledgements
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 Authors' Addresses
1. Introduction 1. Introduction
Currently, many IoT services leverage cloud computing platforms, At the time of writing, many IoT services leverage cloud computing
because they provide virtually unlimited storage and processing platforms because they provide virtually unlimited storage and
power. The reliance of IoT on back-end cloud computing provides processing power. The reliance of IoT on back-end cloud computing
additional advantages such as scalability and efficiency. Today's provides additional advantages, such as scalability and efficiency.
IoT systems are fairly static with respect to integrating and At the time of writing, IoT systems are fairly static with respect to
supporting computation. It is not that there is no computation, but integrating and supporting computation. It is not that there is no
that systems are often limited to static configurations (edge computation, but that systems are often limited to static
gateways and cloud services). configurations (edge gateways and cloud services).
However, IoT devices generate large amounts of data at the edges of However, IoT devices generate large amounts of data at the edges of
the network. To meet IoT use case requirements, data is increasingly the network. To meet IoT use case requirements, data is increasingly
being stored, processed, analyzed, and acted upon close to the data being stored, processed, analyzed, and acted upon close to the data
sources. These requirements include time sensitivity, data volume, sources. These requirements include time sensitivity, data volume,
connectivity cost, and resiliency in the presence of intermittent connectivity cost, and resiliency in the presence of intermittent
connectivity, privacy, and security, which cannot be addressed by connectivity, privacy, and security, which cannot be addressed by
centralized cloud computing. A more flexible approach is necessary centralized cloud computing. A more flexible approach is necessary
to address these needs effectively. This involves distributing to address these needs effectively. This involves distributing
computing (and storage) and seamlessly integrating it into the edge- computing (and storage) and seamlessly integrating it into the edge-
cloud continuum. We refer to this integration of edge computing and cloud continuum. We refer to this integration of edge computing and
IoT as "IoT edge computing". This draft describes the related IoT as "IoT edge computing". This document describes the related
background, use cases, challenges, system models, and functional background, use cases, challenges, system models, and functional
components. components.
Owing to the dynamic nature of the IoT edge computing landscape, this Owing to the dynamic nature of the IoT edge computing landscape, this
document does not list existing projects in this field. Section 4.1 document does not list existing projects in this field. Section 4.1
presents a high-level overview of the field, based on a limited presents a high-level overview of the field based on a limited review
review of standards, research, open-source and proprietary products of standards, research, and open-source and proprietary products in
in [I-D.defoy-t2trg-iot-edge-computing-background]. [EDGE-COMPUTING-BACKGROUND].
This document represents the consensus of the Thing-to-Thing Research This document represents the consensus of the Thing-to-Thing Research
Group (T2TRG). It has been reviewed extensively by the Research Group (T2TRG). It has been reviewed extensively by the research
Group (RG) members who are actively involved in the research and group members who are actively involved in the research and
development of the technology covered by this document. It is not an development of the technology covered by this document. It is not an
IETF product and is not a standard. IETF product and is not a standard.
2. Background 2. Background
2.1. Internet of Things (IoT) 2.1. Internet of Things (IoT)
Since the term "Internet of Things" (IoT) was coined by Kevin Ashton Since the term "Internet of Things" was coined by Kevin Ashton in
in 1999 working on Radio-Frequency Identification (RFID) technology 1999 while working on Radio-Frequency Identification (RFID)
[Ashton], the concept of IoT has evolved. It now reflects a vision technology [Ashton], the concept of IoT has evolved. At the time of
of connecting the physical world to the virtual world of computers writing, it reflects a vision of connecting the physical world to the
using (often wireless) networks over which things can send and virtual world of computers using (often wireless) networks over which
receive information without human intervention. Recently, the term things can send and receive information without human intervention.
has become more literal by connecting things to the Internet and Recently, the term has become more literal by connecting things to
converging on Internet and Web technologies. the Internet and converging on Internet and web technologies.
A Thing is a physical item made available in the IoT, thereby A "Thing" is a physical item made available in the IoT, thereby
enabling digital interaction with the physical world for humans, enabling digital interaction with the physical world for humans,
services, and/or other Things ([I-D.irtf-t2trg-rest-iot]). In this services, and/or other Things [REST-IOT]. In this document, we will
document we will use the term "IoT device" to designate the embedded use the term "IoT device" to designate the embedded system attached
system attached to the Thing. to the Thing.
Resource-constrained Things such as sensors, home appliances and Resource-constrained Things, such as sensors, home appliances, and
wearable devices often have limited storage and processing power, wearable devices, often have limited storage and processing power,
which can provide challenges with respect to reliability, which can create challenges with respect to reliability, performance,
performance, energy consumption, security, and privacy [Lin]. Some, energy consumption, security, and privacy [Lin]. Some, less-
less resource-constrained Things, can generate a voluminous amount of resource-constrained Things, can generate a voluminous amount of
data. This range of factors led IoT designs that integrate Things data. This range of factors led to IoT designs that integrate Things
into larger distributed systems, for example edge or cloud computing into larger distributed systems, for example, edge or cloud computing
systems. systems.
2.2. Cloud Computing 2.2. Cloud Computing
Cloud computing has been defined in [NIST]: "cloud computing is a Cloud computing has been defined in [NIST]:
model for enabling ubiquitous, convenient, on-demand network access
to a shared pool of configurable computing resources (e.g., networks,
servers, storage, applications, and services) that can be rapidly
provisioned and released with minimal management effort or service
provider interaction". The low cost and massive availability of
storage and processing power enabled the realization of another
computing model, in which virtualized resources can be leased in an
on-demand fashion and be provided as general utilities. Platform-as-
a-Service and cloud computing platforms widely adopted this paradigm
for delivering services over the Internet, gaining both economical
and technical benefits [Botta].
Today, an unprecedented volume and variety of data is generated by | cloud computing is a model for enabling ubiquitous, convenient,
Things, and applications deployed at the network edge consume this | on-demand network access to a shared pool of configurable
data. In this context, cloud-based service models are not suitable | computing resources (e.g., networks, servers, storage,
for some classes of applications which require very short response | applications, and services) that can be rapidly provisioned and
times, access to local personal data, or generate vast amounts of | released with minimal management effort or service provider
data. These applications may instead leverage edge computing. | interaction.
The low cost and massive availability of storage and processing power
enabled the realization of another computing model in which
virtualized resources can be leased in an on-demand fashion and
provided as general utilities. Platform-as-a-Service (PaaS) and
cloud computing platforms widely adopted this paradigm for delivering
services over the Internet, gaining both economical and technical
benefits [Botta].
At the time of writing, an unprecedented volume and variety of data
is generated by Things, and applications deployed at the network edge
consume this data. In this context, cloud-based service models are
not suitable for some classes of applications that require very short
response times, require access to local personal data, or generate
vast amounts of data. These applications may instead leverage edge
computing.
2.3. Edge Computing 2.3. Edge Computing
Edge computing, also referred to as fog computing in some settings, Edge computing, also referred to as "fog computing" in some settings,
is a new paradigm in which substantial computing and storage is a new paradigm in which substantial computing and storage
resources are placed at the edge of the Internet, close to mobile resources are placed at the edge of the Internet, close to mobile
devices, sensors, actuators, or machines. Edge computing happens devices, sensors, actuators, or machines. Edge computing happens
near data sources [Mahadev], as well as close to where decisions are near data sources [Mahadev] as well as close to where decisions are
made or where interactions with the physical world take place made or where interactions with the physical world take place
("close" here can refer to a distance which is topological, physical, ("close" here can refer to a distance that is topological, physical,
latency-based, etc.). It processes both downstream data (originating latency-based, etc.). It processes both downstream data (originating
from cloud services) and upstream data (originating from end devices from cloud services) and upstream data (originating from end devices
or network elements). The term "fog computing" usually represents or network elements). The term "fog computing" usually represents
the notion of multi-tiered edge computing, that is, several layers of the notion of multi-tiered edge computing, that is, several layers of
compute infrastructure between end devices and cloud services. compute infrastructure between end devices and cloud services.
An edge device is any computing or networking resource residing An edge device is any computing or networking resource residing
between end-device data sources and cloud-based data centers. In between end-device data sources and cloud-based data centers. In
edge computing, end devices consume and produce data. At the network edge computing, end devices consume and produce data. At the network
edge, devices not only request services and information from the edge, devices not only request services and information from the
Cloud but also handle computing tasks including processing, storage, cloud but also handle computing tasks including processing, storing,
caching, and load balancing on data sent to and from the Cloud [Shi]. caching, and load balancing on data sent to and from the cloud [Shi].
This does not preclude end devices from hosting computation This does not preclude end devices from hosting computation
themselves, when possible, independently or as part of a distributed themselves, when possible, independently or as part of a distributed
edge computing platform. edge computing platform.
Several standards developing organization (SDO) and industry forums Several Standards Developing Organizations (SDOs) and industry forums
have provided definitions of edge and fog computing: have provided definitions of edge and fog computing:
* ISO defines edge computing as a "form of distributed computing in * ISO defines edge computing as a "form of distributed computing in
which significant processing and data storage takes place on nodes which significant processing and data storage takes place on nodes
which are at the edge of the network" [ISO_TR]. which are at the edge of the network" [ISO_TR].
* ETSI defines multi-access edge computing as a "system which * ETSI defines multi-access edge computing as a "system which
provides an IT service environment and cloud-computing provides an IT service environment and cloud-computing
capabilities at the edge of an access network which contains one capabilities at the edge of an access network which contains one
or more type of access technology, and in close proximity to its or more type of access technology, and in close proximity to its
users" [ETSI_MEC_01]. users" [ETSI_MEC_01].
* The Industry IoT Consortium (IIC, now incorporating what was * The Industry IoT Consortium (IIC) (now incorporating what was
formerly OpenFog) defines fog computing as "a horizontal, system- formerly OpenFog) defines fog computing as "a horizontal, system-
level architecture that distributes computing, storage, control level architecture that distributes computing, storage, control
and networking functions closer to the users along a cloud-to- and networking functions closer to the users along a cloud-to-
thing continuum" [OpenFog]. thing continuum" [OpenFog].
Based on these definitions, we can summarize a general philosophy of Based on these definitions, we can summarize a general philosophy of
edge computing as distributing the required functions close to users edge computing as distributing the required functions close to users
and data, while the difference to classic local systems is the usage and data, while the difference to classic local systems is the usage
of management and orchestration features adopted from cloud of management and orchestration features adopted from cloud
computing. computing.
Actors from various industries approach edge computing using Actors from various industries approach edge computing using
different terms and reference models although, in practice, these different terms and reference models, although, in practice, these
approaches are not incompatible and may integrate with each other: approaches are not incompatible and may integrate with each other:
* The telecommunication industry tends to use a model where edge * The telecommunication industry tends to use a model where edge
computing services are deployed over Network Function computing services are deployed over a Network Function
Virtualization (NFV) infrastructure, at aggregation points or in Virtualization (NFV) infrastructure, at aggregation points, or in
proximity to the user equipment (e.g., gNodeBs) [ETSI_MEC_03]. proximity to the user equipment (e.g., gNodeBs) [ETSI_MEC_03].
* Enterprise and campus solutions often interpret edge computing as * Enterprise and campus solutions often interpret edge computing as
an "edge cloud", that is, a smaller data center directly connected an "edge cloud", that is, a smaller data center directly connected
to the local network (often referred to as "on-premise"). to the local network (often referred to as "on-premise").
* The automation industry defines the edge as the connection point * The automation industry defines the edge as the connection point
between IT and OT (Operational Technology). Hence, edge computing between IT and Operational Technology (OT). Hence, edge computing
sometimes refers to applying IT solutions to OT problems, such as sometimes refers to applying IT solutions to OT problems, such as
analytics, more flexible user interfaces, or simply having more analytics, more-flexible user interfaces, or simply having more
computing power than an automation controller. computing power than an automation controller.
2.4. Examples of IoT Edge Computing Use Cases 2.4. Examples of IoT Edge Computing Use Cases
IoT edge computing can be used in home, industry, grid, healthcare, IoT edge computing can be used in home, industry, grid, healthcare,
city, transportation, agriculture, and/or educational scenarios. city, transportation, agriculture, and/or educational scenarios.
Here, we discuss only a few examples of such use cases, to identify Here, we discuss only a few examples of such use cases to identify
differentiating requirements, providing references to other use differentiating requirements, providing references to other use
cases. cases.
*Smart Factory* *Smart Factory*
As part of the Fourth Industrial Revolution, smart factories run
real-time processes based on IT technologies, such as artificial
intelligence and big data. Even a very small environmental change
in a smart factory can lead to a situation in which production
efficiency decreases or product quality problems occur.
Therefore, simple but time-sensitive processing can be performed
at the edge, for example, controlling the temperature and humidity
in the factory or operating machines based on the real-time
collection of the operational status of each machine. However,
data requiring highly precise analysis, such as machine life-cycle
management or accident risk prediction, can be transferred to a
central data center for processing.
As part of the 4th industrial revolution, smart factories run real- The use of edge computing in a smart factory can reduce the cost
time processes based on IT technologies, such as artificial of network and storage resources by reducing the communication
intelligence and big data. Even a very small environmental change in load to the central data center or server. It is also possible to
a smart factory can lead to a situation in which production improve process efficiency and facility asset productivity through
efficiency decreases or product quality problems occur. Therefore, real-time prediction of failures and to reduce the cost of failure
simple but time-sensitive processing can be performed at the edge, through preliminary measures. In the existing manufacturing
for example, controlling the temperature and humidity in the factory, field, production facilities are manually run according to a
or operating machines based on the real-time collection of the program entered in advance; however, edge computing in a smart
operational status of each machine. However, data requiring highly factory enables tailoring solutions by analyzing data at each
precise analysis, such as machine lifecycle management or accident production facility and machine level. Digital twins [Jones] of
risk prediction, can be transferred to a central data center for IoT devices have been jointly used with edge computing in
processing. industrial IoT scenarios [Chen].
The use of edge computing in a smart factory can reduce the cost of
network and storage resources by reducing the communication load to
the central data center or server. It is also possible to improve
process efficiency and facility asset productivity through real-time
prediction of failures and to reduce the cost of failure through
preliminary measures. In the existing manufacturing field,
production facilities are manually run according to a program entered
in advance; however, edge computing in a smart factory enables
tailoring solutions by analyzing data at each production facility and
machine level. Digital twins [Jones] of IoT devices have been
jointly used with edge computing in industrial IoT scenarios [Chen].
*Smart Grid* *Smart Grid*
In future smart-city scenarios, the smart grid will be critical in
In future smart city scenarios, the Smart Grid will be critical in ensuring highly available/efficient energy control in city-wide
ensuring highly available/efficient energy control in city-wide electricity management. Edge computing is expected to play a
electricity management. Edge computing is expected to play a significant role in these systems to improve the transmission
significant role in these systems to improve the transmission efficiency of electricity, to react to and restore power after a
efficiency of electricity, to react to, and restore power after a disturbance, to reduce operation costs, and to reuse energy
disturbance, to reduce operation costs, and to reuse energy effectively since these operations involve local decision-making.
effectively, since these operations involve local decision-making. In addition, edge computing can help monitor power generation and
In addition, edge computing can help monitor power generation and power demand and make local electrical energy storage decisions in
power demand, and make local electrical energy storage decisions in smart grid systems.
smart grid systems.
*Smart Agriculture* *Smart Agriculture*
Smart agriculture integrates information and communication
technologies with farming technology. Intelligent farms use IoT
technology to measure and analyze parameters, such as the
temperature, humidity, sunlight, carbon dioxide, and soil quality,
in crop cultivation facilities. Depending on the analysis
results, control devices are used to set the environmental
parameters to an appropriate state. Remote management is also
possible through mobile devices, such as smartphones.
Smart agriculture integrates information and communication In existing farms, simple systems, such as management according to
technologies with farming technology. Intelligent farms use IoT temperature and humidity, can be easily and inexpensively
technology to measure and analyze parameters, such as the implemented using IoT technology. Field sensors gather data on
temperature, humidity, sunlight, carbon dioxide, and soil quality, in field and crop condition. This data is then transmitted to cloud
crop cultivation facilities. Depending on the analysis results, servers that process data and recommend actions. The use of edge
control devices are used to set the environmental parameters to an computing can reduce the volume of back-and-forth data
appropriate state. Remote management is also possible through mobile transmissions significantly, resulting in cost and bandwidth
devices such as smartphones. savings. Locally generated data can be processed at the edge, and
local computing and analytics can drive local actions. With edge
In existing farms, simple systems such as management according to computing, it is easy for farmers to select large amounts of data
temperature and humidity can be easily and inexpensively implemented for processing, and data can be analyzed even in remote areas with
using IoT technology. Field sensors gather data on field and crop poor access conditions. Other applications include enabling
condition. This data is then transmitted to cloud servers that dashboarding, for example, to visualize the farm status, as well
process data and recommend actions. The use of edge computing can as enhancing Extended Reality (XR) applications that require edge
reduce the volume of back-and-forth data transmissions significantly, audio/video processing. As the number of people working on
resulting in cost and bandwidth savings. Locally generated data can farming has been decreasing over time, increasing automation
be processed at the edge, and local computing and analytics can drive enabled by edge computing can be a driving force for future smart
local actions. With edge computing, it is easy for farmers to select agriculture.
large amounts of data for processing, and data can be analyzed even
in remote areas with poor access conditions. Other applications
include enabling dashboarding, for example, to visualize the farm
status, as well as enhancing Extended Reality (XR) applications that
require edge audio/video processing. As the number of people working
on farming has been decreasing over time, increasing automation
enabled by edge computing can be a driving force for future smart
agriculture.
*Smart Construction* *Smart Construction*
Safety is critical at construction sites. Every year, many
construction workers lose their lives because of falls,
collisions, electric shocks, and other accidents. Therefore,
solutions have been developed to improve construction site safety,
including the real-time identification of workers, monitoring of
equipment location, and predictive accident prevention. To deploy
these solutions, many cameras and IoT sensors have been installed
on construction sites to measure noise, vibration, gas
concentration, etc. Typically, the data generated from these
measurements is collected in on-site gateways and sent to remote
cloud servers for storage and analysis. Thus, an inspector can
check the information stored on the cloud server to investigate an
incident. However, this approach can be expensive because of
transmission costs (for example, of video streams over a mobile
network connection) and because usage fees of private cloud
services.
Safety is critical at construction sites. Every year, many Using edge computing, data generated at the construction site can
construction workers lose their lives because of falls, collisions, be processed and analyzed on an edge server located within or near
electric shocks, and other accidents. Therefore, solutions have been the site. Only the result of this processing needs to be
developed to improve construction site safety, including the real- transferred to a cloud server, thus reducing transmission costs.
time identification of workers, monitoring of equipment location, and It is also possible to locally generate warnings to prevent
predictive accident prevention. To deploy these solutions, many accidents in real time.
cameras and IoT sensors have been installed on construction sites, to
measure noise, vibration, gas concentration, etc. Typically, the
data generated from these measurements is collected in on-site
gateways and sent to remote cloud servers for storage and analysis.
Thus, an inspector can check the information stored on the cloud
server to investigate an incident. However, this approach can be
expensive because of transmission costs, for example, of video
streams over a mobile network connection, and because usage fees of
private cloud services.
Using edge computing, data generated at the construction site can be
processed and analyzed on an edge server located within or near the
site. Only the result of this processing needs to be transferred to
a cloud server, thus reducing transmission costs. It is also
possible to locally generate warnings to prevent accidents in real-
time.
*Self-Driving Car* *Self-Driving Car*
Edge computing plays a crucial role in safety-focused self-driving
car systems. With a multitude of sensors, such as high-resolution
cameras, radars, Light Detection and Ranging (LiDAR), sonar
sensors, and GPS systems, autonomous vehicles generate vast
amounts of real-time data. Local processing utilizing edge
computing nodes allows for efficient collection and analysis of
this data to monitor vehicle distances and road conditions and
respond promptly to unexpected situations. Roadside computing
nodes can also be leveraged to offload tasks when necessary, for
example, when the local processing capacity of the car is
insufficient because of hardware constraints or a large data
volume.
Edge computing plays a crucial role in safety-focused self-driving For instance, when the car ahead slows, a self-driving car adjusts
car systems. With a multitude of sensors, such as high-resolution its speed to maintain a safe distance, or when a roadside signal
cameras, radar, LIDAR, sonar sensors, and GPS systems, autonomous changes, it adapts its behavior accordingly. In another example,
vehicles generate vast amounts of real-time data. Local processing cars equipped with self-parking features utilize local processing
utilizing edge computing nodes allows for efficient collection and to analyze sensor data, determine suitable parking spots, and
analysis of this data to monitor vehicle distances and road execute precise parking maneuvers without relying on external
conditions and respond promptly to unexpected situations. Roadside processing or connectivity. It is also possible to use in-cabin
computing nodes can also be leveraged to offload tasks when cameras coupled with local processing to monitor the driver's
necessary, for example, when the local processing capacity of the car attention level and detect signs of drowsiness or distraction.
is insufficient because of hardware constraints or a large data The system can issue warnings or implement preventive measures to
volume. ensure driver safety.
For instance, when the car ahead slows, a self-driving car adjusts
its speed to maintain a safe distance, or when a roadside signal
changes, it adapts its behavior accordingly. In another example,
cars equipped with self-parking features utilize local processing to
analyze sensor data, determine suitable parking spots, and execute
precise parking maneuvers without relying on external processing or
connectivity. It is also possible to use in-cabin cameras coupled
with local processing to monitor the driver's attention level and
detect signs of drowsiness or distraction. The system can issue
warnings or implement preventive measures to ensure driver safety.
Edge computing empowers self-driving cars by enabling real-time Edge computing empowers self-driving cars by enabling real-time
processing, reducing latency, enhancing data privacy, and optimizing processing, reducing latency, enhancing data privacy, and
bandwidth usage. By leveraging local processing capabilities, self- optimizing bandwidth usage. By leveraging local processing
driving cars can make rapid decisions, adapt to changing capabilities, self-driving cars can make rapid decisions, adapt to
environments, and ensure safer and more efficient autonomous driving changing environments, and ensure safer and more efficient
experiences. autonomous driving experiences.
*Digital Twin* *Digital Twin*
A digital twin can simulate different scenarios and predict
outcomes based on real-time data collected from the physical
environment. This simulation capability empowers proactive
maintenance, optimization of operations, and the prediction of
potential issues or failures. Decision makers can use digital
twins to test and validate different strategies, identify
inefficiencies, and optimize performance.
A digital twin can simulate different scenarios and predict outcomes With edge computing, real-time data is collected, processed, and
based on real-time data collected from the physical environment. analyzed directly at the edge, allowing for the accurate
This simulation capability empowers proactive maintenance, monitoring and simulation of physical assets. Moreover, edge
optimization of operations, and the prediction of potential issues or computing effectively minimizes latency, enabling rapid responses
failures. Decision makers can use digital twins to test and validate to dynamic conditions as computational resources are brought
different strategies, identify inefficiencies, and optimize closer to the physical object. Running digital twin processing at
performance. the edge enables organizations to obtain timely insights and make
informed decisions that maximize efficiency and performance.
With edge computing, real-time data is collected, processed, and
analyzed directly at the edge, allowing for the accurate monitoring
and simulation of physical assets. Moreover, edge computing
effectively minimizes latency, enabling rapid responses to dynamic
conditions as computational resources are brought closer to the
physical object. Running digital twin processing at the edge enables
organizations to obtain timely insights and make informed decisions
that maximize efficiency and performance.
*Other Use Cases* *Other Use Cases*
Artificial intelligence (AI) / machine learning (ML) systems at
the edge empower real-time analysis, faster decision-making,
reduced latency, improved operational efficiency, and personalized
experiences across various industries by bringing AI and ML
capabilities closer to edge devices.
AI/ML systems at the edge empower real-time analysis, faster In addition, oneM2M has studied several IoT edge computing use
decision-making, reduced latency, improved operational efficiency, cases, which are documented in [oneM2M-TR0001], [oneM2M-TR0018],
and personalized experiences across various industries, by bringing and [oneM2M-TR0026]. The edge-computing-related requirements
artificial intelligence and machine learning capabilities closer to raised through the analysis of these use cases are captured in
edge devices. [oneM2M-TS0002].
In addition, oneM2M has studied several IoT edge computing use cases,
which are documented in [oneM2M-TR0001], [oneM2M-TR0018] and
[oneM2M-TR0026]. The edge computing related requirements raised
through the analysis of these use cases are captured in
[oneM2M-TS0002].
3. IoT Challenges Leading Towards Edge Computing 3. IoT Challenges Leading toward Edge Computing
This section describes the challenges faced by IoT that are This section describes the challenges faced by the IoT that are
motivating the adoption of edge computing. These are distinct from motivating the adoption of edge computing. These are distinct from
the research challenges applicable to IoT edge computing, some of the research challenges applicable to IoT edge computing, some of
which are mentioned in Section 4. which are mentioned in Section 4.
IoT technology is used with increasingly demanding applications, for IoT technology is used with increasingly demanding applications in
example, in industrial, automotive and healthcare domains, leading to domains such as industrial, automotive, and healthcare, which leads
new challenges. For example, industrial machines such as laser to new challenges. For example, industrial machines, such as laser
cutters produce over 1 terabyte of data per hour, and similar amounts cutters, produce over 1 terabyte of data per hour, and similar
can be generated in autonomous cars [NVIDIA]. 90% of IoT data is amounts can be generated in autonomous cars [NVIDIA]. 90% of IoT
expected to be stored, processed, analyzed, and acted upon close to data is expected to be stored, processed, analyzed, and acted upon
the source [Kelly], as cloud computing models alone cannot address close to the source [Kelly], as cloud computing models alone cannot
these new challenges [Chiang]. address these new challenges [Chiang].
Below, we discuss IoT use case requirements that are moving cloud Below, we discuss IoT use case requirements that are moving cloud
capabilities to be more proximate, distributed, and disaggregated. capabilities to be more proximate, distributed, and disaggregated.
3.1. Time Sensitivity 3.1. Time Sensitivity
Many industrial control systems, such as manufacturing systems, smart Often, many industrial control systems, such as manufacturing
grids, and oil and gas systems often require stringent end-to-end systems, smart grids, and oil and gas systems, require stringent end-
latency between the sensor and control nodes. While some IoT to-end latency between the sensor and control nodes. While some IoT
applications may require latency below a few tens of milliseconds applications may require latency below a few tens of milliseconds
[Weiner], industrial robots and motion control systems have use cases [Weiner], industrial robots and motion control systems have use cases
for cycle times in the order of microseconds [_60802]. In some for cycle times in the order of microseconds [IEC_IEEE_60802]. In
cases, speed-of-light limitations may simply prevent a cloud-based some cases, speed-of-light limitations may simply prevent cloud-based
solutions; however, this is not the only challenge relative to time solutions; however, this is not the only challenge relative to time
sensitivity. Guarantees for bounded latency and jitter ([RFC8578] sensitivity. Guarantees for bounded latency and jitter ([RFC8578],
section 7) are also important for industrial IoT applications. This Section 7) are also important for industrial IoT applications. This
means that control packets must arrive with as little variation as means that control packets must arrive with as little variation as
possible and within a strict deadline. Given the best-effort possible and within a strict deadline. Given the best-effort
characteristics of the Internet, this challenge is virtually characteristics of the Internet, this challenge is virtually
impossible to address, without using end-to-end guarantees for impossible to address without using end-to-end guarantees for
individual message delivery and continuous data flows. individual message delivery and continuous data flows.
3.2. Connectivity Cost 3.2. Connectivity Cost
Some IoT deployments may not face bandwidth constraints when Some IoT deployments may not face bandwidth constraints when
uploading data to the Cloud. 5G and Wi-Fi 6 networks both uploading data to the cloud. Theoretically, both 5G and Wi-Fi 6
theoretically top out at 10 gigabits per second (i.e., 4.5 terabytes networks top out at 10 gigabits per second (i.e., 4.5 terabytes per
per hour), allowing to transfer large amounts of uplink data. hour), allowing the transfer of large amounts of uplink data.
However, the cost of maintaining continuous high-bandwidth However, the cost of maintaining continuous high-bandwidth
connectivity for such usage is unjustifiable and impractical for most connectivity for such usage is unjustifiable and impractical for most
IoT applications. In some settings, for example, in aeronautical IoT applications. In some settings, for example, in aeronautical
communication, higher communication costs reduce the amount of data communication, higher communication costs reduce the amount of data
that can be practically uploaded even further. Minimizing reliance that can be practically uploaded even further. Therefore, minimizing
on high-bandwidth connectivity is therefore a requirement, for reliance on high-bandwidth connectivity is a requirement; this can be
example, by processing data at the edge and deriving summarized or done, for example, by processing data at the edge and deriving
actionable insights that can be transmitted to the Cloud. summarized or actionable insights that can be transmitted to the
cloud.
3.3. Resilience to Intermittent Services 3.3. Resilience to Intermittent Services
Many IoT devices, such as sensors, actuators, and controllers, have Many IoT devices, such as sensors, actuators, and controllers, have
very limited hardware resources and cannot rely solely on their own very limited hardware resources and cannot rely solely on their own
resources to meet their computing and/or storage needs. They require resources to meet their computing and/or storage needs. They require
reliable, uninterrupted, or resilient services to augment their reliable, uninterrupted, or resilient services to augment their
capabilities to fulfill their application tasks. This is difficult capabilities to fulfill their application tasks. This is difficult
and partly impossible to achieve using cloud services for systems and partly impossible to achieve using cloud services for systems
such as vehicles, drones, or oil rigs that have intermittent network such as vehicles, drones, or oil rigs that have intermittent network
connectivity. Conversely, a cloud back-end might want to device data connectivity. Conversely, a cloud backend might want to device data
even if it is currently asleep. even if it is currently asleep.
3.4. Privacy and Security 3.4. Privacy and Security
When IoT services are deployed at home, personal information can be When IoT services are deployed at home, personal information can be
learned from detected usage data. For example, one can extract learned from detected usage data. For example, one can extract
information about employment, family status, age, and income by information about employment, family status, age, and income by
analyzing smart-meter data [ENERGY]. Policy makers have begun to analyzing smart meter data [ENERGY]. Policy makers have begun to
provide frameworks that limit the usage of personal data and impose provide frameworks that limit the usage of personal data and impose
strict requirements on data controllers and processors. Data stored strict requirements on data controllers and processors. Data stored
indefinitely in the Cloud also increases the risk of data leakage, indefinitely in the cloud also increases the risk of data leakage,
for instance, through attacks on rich targets. for instance, through attacks on rich targets.
It is often argues that industrial systems do not provide privacy It is often argued that industrial systems do not provide privacy
implications, as no personal data is gathered. However, data from implications, as no personal data is gathered. However, data from
such systems is often highly sensitive, as one might be able to infer such systems is often highly sensitive, as one might be able to infer
trade secrets such as the setup of production lines. Hence, owners trade secrets, such as the setup of production lines. Hence, owners
of these systems are generally reluctant to upload IoT data to the of these systems are generally reluctant to upload IoT data to the
Cloud. cloud.
Furthermore, passive observers can perform traffic analysis on Furthermore, passive observers can perform traffic analysis on
device-to-cloud paths. Therefore, hiding traffic patterns associated device-to-cloud paths. Therefore, hiding traffic patterns associated
with sensor networks can be another requirement for edge computing. with sensor networks can be another requirement for edge computing.
4. IoT Edge Computing Functions 4. IoT Edge Computing Functions
We first look at the current state of IoT edge computing We first look at the current state of IoT edge computing
(Section 4.1), and then define a general system model (Section 4.2). (Section 4.1) and then define a general system model (Section 4.2).
This provides a context for IoT edge-computing functions, which are This provides a context for IoT edge computing functions, which are
listed in Section 4.3, Section 4.4 and Section 4.5. listed in Sections 4.3, 4.4, and 4.5.
4.1. Overview of IoT Edge Computing Today 4.1. Overview of IoT Edge Computing
This section provides an overview of today's IoT edge computing field This section provides an overview of the current (at the time of
based on a limited review of standards, research, open-source and writing) IoT edge computing field based on a limited review of
proprietary products in standards, research, and open-source and proprietary products in
[I-D.defoy-t2trg-iot-edge-computing-background]. [EDGE-COMPUTING-BACKGROUND].
IoT gateways, both open-source (such as EdgeX Foundry or Home Edge) IoT gateways, both open-source (such as EdgeX Foundry or Home Edge)
and proprietary products, represent a common class of IoT edge- and proprietary products, represent a common class of IoT edge
computing products, where the gateway provides a local service on computing products, where the gateway provides a local service on
customer premises and is remotely managed through a cloud service. customer premises and is remotely managed through a cloud service.
IoT communication protocols are typically used between IoT devices IoT communication protocols are typically used between IoT devices
and the gateway, including CoAP [RFC7252], MQTT [mqtt5], and many and the gateway, including a Constrained Application Protocol (CoAP)
specialized IoT protocols (such as OPC UA and DDS in the Industrial [RFC7252], Message Queuing Telemetry Transport (MQTT) [MQTT5], and
IoT space), while the gateway communicates with the distant cloud many specialized IoT protocols (such as Open Platform Communications
typically using HTTPS. Virtualization platforms enable the Unified Architecture (OPC UA) and Data Distribution Service (DDS) in
deployment of virtual edge computing functions (using VMs and the industrial IoT space), while the gateway communicates with the
application containers), including IoT gateway software, on servers distant cloud typically using HTTPS. Virtualization platforms enable
in the mobile network infrastructure (at base stations and the deployment of virtual edge computing functions (using Virtual
concentration points), edge data centers (in central offices), and Machines (VMs) and application containers), including IoT gateway
regional data centers located near central offices. End devices are software, on servers in the mobile network infrastructure (at base
envisioned to become computing devices in forward-looking projects, stations and concentration points), edge data centers (in central
but are not commonly used today. offices), and regional data centers located near central offices.
End devices are envisioned to become computing devices in forward-
looking projects but are not commonly used at the time of writing.
In addition to open-source and proprietary solutions, a horizontal In addition to open-source and proprietary solutions, a horizontal
IoT service layer is standardized by the oneM2M standards body to IoT service layer is standardized by the oneM2M standards body to
reduce fragmentation, increase interoperability and promote reuse in reduce fragmentation, increase interoperability, and promote reuse in
the IoT ecosystem. Furthermore, ETSI MEC developed an IoT API the IoT ecosystem. Furthermore, ETSI Multi-access Edge Computing
[ETSI_MEC_33] that enables the deployment of heterogeneous IoT (MEC) developed an IoT API [ETSI_MEC_33] that enables the deployment
platforms and provides a means to configure the various components of of heterogeneous IoT platforms and provides a means to configure the
an IoT system. various components of an IoT system.
Physical or virtual IoT gateways can host application programs that Physical or virtual IoT gateways can host application programs that
are typically built using an SDK to access local services through a are typically built using an SDK to access local services through a
programmatic API. Edge cloud system operators host their customers' programmatic API. Edge cloud system operators host their customers'
application VMs or containers on servers located in or near access application VMs or containers on servers located in or near access
networks that can implement local edge services. For example, mobile networks that can implement local edge services. For example, mobile
networks can provide edge services for radio-network information, networks can provide edge services for radio network information,
location, and bandwidth management. location, and bandwidth management.
Resilience in the IoT can entail the ability to operate autonomously Resilience in the IoT can entail the ability to operate autonomously
in periods of disconnectedness to preserve the integrity and safety in periods of disconnectedness to preserve the integrity and safety
of the controlled system, possibly in a degraded mode. IoT devices of the controlled system, possibly in a degraded mode. IoT devices
and gateways are often expected to operate in always-on and and gateways are often expected to operate in always-on and
unattended modes, using fault detection and unassisted recovery unattended modes, using fault detection and unassisted recovery
functions. functions.
The life cycle management of services and applications on physical The life-cycle management of services and applications on physical
IoT gateways is generally cloud-based. Edge cloud management IoT gateways is generally cloud based. Edge cloud management
platforms and products (such as StarlingX, Akraino Edge Stack, or platforms and products (such as StarlingX, Akraino Edge Stack, or
proprietary products from major Cloud providers) adapt cloud proprietary products from major cloud providers) adapt cloud
management technologies (e.g., Kubernetes) to the edge cloud, that management technologies (e.g., Kubernetes) to the edge cloud, that
is, to smaller, distributed computing devices running outside a is, to smaller, distributed computing devices running outside a
controlled data center. The service and application life-cycle is controlled data center. Typically, the service and application life
typically using an NFV-like management and orchestration model. cycle is using an NFV-like management and orchestration model.
The platform typically enables advertising or consuming services The platform generally enables advertising or consuming services
hosted on the platform (e.g., the Mp1 interface in ETSI MEC supports hosted on the platform (e.g., the Mp1 interface in ETSI MEC supports
service discovery and communication), and enables communication with service discovery and communication), and enables communication with
local and remote endpoints (e.g., message routing function in IoT local and remote endpoints (e.g., message routing function in IoT
gateways). The platform is typically extensible to edge applications gateways). The platform is usually extensible to edge applications
because it can advertise a service that other edge applications can because it can advertise a service that other edge applications can
consume. The IoT communication services include protocol consume. The IoT communication services include protocol
translation, analytics, and transcoding. Communication between edge- translation, analytics, and transcoding. Communication between edge
computing devices is enabled in tiered or distributed deployments. computing devices is enabled in tiered or distributed deployments.
An edge cloud platform may enable pass-through without storage or An edge cloud platform may enable pass-through without storage or
local storage (e.g., on IoT gateways). Some edge cloud platforms use local storage (e.g., on IoT gateways). Some edge cloud platforms use
distributed storage such as that provided by a distributed storage distributed storage such as that provided by a distributed storage
platform (e.g., EdgeFS, Ceph), or, in more experimental settings, by platform (e.g., EdgeFS and Ceph) or, in more experimental settings,
an ICN network, for example, systems such as Chipmunk [chipmunk] and by an Information-Centric Networking (ICN) network, for example,
Kua [kua] have been proposed as distributed information-centric systems such as Chipmunk [Chipmunk] and Kua [Kua] have been proposed
objects stores. External storage, for example, on databases in as distributed information-centric objects stores. External storage,
distant or local IT cloud, is typically used for filtered data deemed for example, on databases in a distant or local IT cloud, is
worthy of long-term storage, although in some cases it may be for all typically used for filtered data deemed worthy of long-term storage;
data, for example when required for regulatory reasons. although, in some cases, it may be for all data, for example, when
required for regulatory reasons.
Stateful computing is supported on platforms that host native Stateful computing is the default on most systems, VMs, and
programs, VMs, or containers. Stateless computing is supported on containers. Stateless computing is supported on platforms providing
platforms providing a "serverless computing" service (also known as a "serverless computing" service (also known as function-as-
function-as-a-service, e.g., using stateless containers), or on a-service, e.g., using stateless containers) or on systems based on
systems based on named function networking. named function networking.
In many IoT use cases, a typical network usage pattern is a high In many IoT use cases, a typical network usage pattern is a high-
volume uplink with some form of traffic reduction enabled by volume uplink with some form of traffic reduction enabled by
processing over edge-computing devices. Alternatives to traffic processing over edge computing devices. Alternatives to traffic
reduction include deferred transmission (to off-peak hours or using reduction include deferred transmission (to off-peak hours or using
physical shipping). Downlink traffic includes application control physical shipping). Downlink traffic includes application control
and software updates. Downlink-heavy traffic patterns are not and software updates. Downlink-heavy traffic patterns are not
excluded but are more often associated with non-IoT usage (e.g., excluded but are more often associated with non-IoT usage (e.g.,
video CDNs). video Content Delivery Networks (CDNs)).
4.2. General Model 4.2. General Model
Edge computing is expected to play an important role in deploying new Edge computing is expected to play an important role in deploying new
IoT services integrated with Big Data and AI enabled by flexible in- IoT services integrated with big data and AI enabled by flexible in-
network computing platforms. Although there are many approaches to network computing platforms. Although there are many approaches to
edge computing, in this section, we attempt to lay out a general edge computing, this section lays out an attempt at a general model
model and the list associated logical functions. In practice, this and lists associated logical functions. In practice, this model can
model can be mapped to different architectures, such as: be mapped to different architectures, such as:
* A single IoT gateway, or a hierarchy of IoT gateways, typically * A single IoT gateway, or a hierarchy of IoT gateways, typically
connected to the cloud (e.g., to extend the traditional cloud- connected to the cloud (e.g., to extend the centralized cloud-
based management of IoT devices and data to the edge). The IoT based management of IoT devices and data to the edge). The IoT
gateway plays a common role in providing access to a heterogeneous gateway plays a common role in providing access to a heterogeneous
set of IoT devices/sensors, handling IoT data, and delivering IoT set of IoT devices/sensors, handling IoT data, and delivering IoT
data to its final destination in a cloud network. Whereas an IoT data to its final destination in a cloud network. An IoT gateway
gateway requires interactions with the cloud, it can also operate requires interactions with the cloud; however, it can also operate
independently in a disconnected mode. independently in a disconnected mode.
* A set of distributed computing nodes, for example, embedded in * A set of distributed computing nodes, for example, embedded in
switches, routers, edge cloud servers, or mobile devices. Some switches, routers, edge cloud servers, or mobile devices. Some
IoT devices have sufficient computing capabilities to participate IoT devices have sufficient computing capabilities to participate
in such distributed systems owing to advances in hardware in such distributed systems owing to advances in hardware
technology. In this model, edge-computing nodes can collaborate technology. In this model, edge computing nodes can collaborate
to share resources. to share resources.
* A hybrid system involving both IoT gateways and supporting * A hybrid system involving both IoT gateways and supporting
functions in distributed computing nodes. functions in distributed computing nodes.
In the general model described in Figure 1, the edge computing domain In the general model described in Figure 1, the edge computing domain
is interconnected with IoT devices (southbound connectivity), is interconnected with IoT devices (southbound connectivity),
possibly with a remote/cloud network (northbound connectivity), and possibly with a remote/cloud network (northbound connectivity), and
with a service operator's system. Edge-computing nodes provide with a service operator's system. Edge computing nodes provide
multiple logical functions or components that may not be present in a multiple logical functions or components that may not be present in a
given system. They may be implemented in a centralized or given system. They may be implemented in a centralized or
distributed fashion, at the network edge, or through interworking distributed fashion, at the network edge, or through interworking
between the edge network and remote cloud networks. between the edge network and remote cloud networks.
+---------------------+ +---------------------+
| Remote network | +---------------+ | Remote Network | +---------------+
|(e.g., cloud network)| | Service | |(e.g., cloud network)| | Service |
+-----------+---------+ | Operator | +-----------+---------+ | Operator |
| +------+--------+ | +------+--------+
| | | |
+--------------+-------------------+-----------+ +--------------+-------------------+-----------+
| Edge Computing Domain | | Edge Computing Domain |
| | | |
| One or more Computing Nodes | | One or more computing nodes |
| (IoT gateway, end devices, switches, | | (IoT gateway, end devices, switches, |
| routers, mini/micro-data centers, etc.) | | routers, mini/micro-data centers, etc.) |
| | | |
| OAM Components | | OAM Components |
| - Resource Discovery and Authentication | | - Resource Discovery and Authentication |
| - Edge Organization and Federation | | - Edge Organization and Federation |
| - Multi-Tenancy and Isolation | | - Multi-Tenancy and Isolation |
| - ... | | - ... |
| | | |
| Functional Components | | Functional Components |
skipping to change at page 15, line 39 skipping to change at line 682
| - ... | | - ... |
| | | |
| Application Components | | Application Components |
| - IoT Devices Management | | - IoT Devices Management |
| - Data Management and Analytics | | - Data Management and Analytics |
| - ... | | - ... |
| | | |
+------+--------------+-------- - - - -+- - - -+ +------+--------------+-------- - - - -+- - - -+
| | | | | | | | | |
| | +-----+--+ | | +-----+--+
+----+---+ +-----+--+ | |compute | | +----+---+ +-----+--+ | |Compute | |
| End | | End | ... |node/end| | End | | End | ... |Node/End|
|Device 1| |Device 2| ...| |device n| | |Device 1| |Device 2| ...| |Device n| |
+--------+ +--------+ +--------+ +--------+ +--------+ +--------+
+ - - - - - - - -+ + - - - - - - - -+
Figure 1: Model of IoT Edge Computing Figure 1: Model of IoT Edge Computing
In the distributed model described in Figure 2, the edge-computing In the distributed model described in Figure 2, the edge computing
domain is composed of IoT edge gateways and IoT devices which are domain is composed of IoT edge gateways and IoT devices that are also
also used as computing nodes. Edge computing domains are connected used as computing nodes. Edge computing domains are connected to a
to a remote/cloud network and their respective service operator's remote/cloud network and their respective service operator's system.
system. IoT devices/computing nodes provide logical functions, for IoT devices/computing nodes provide logical functions, for example,
example as part of distributed machine learning or distributed image as part of distributed machine learning or distributed image
processing applications. The processing capabilities in IoT devices processing applications. The processing capabilities in IoT devices
are limited; they require the support of other nodes, and in a are limited; they require the support of other nodes. In a
distributed machine learning application, the training process for AI distributed machine learning application, the training process for AI
services can be executed at IoT edge gateways or cloud networks and services can be executed at IoT edge gateways or cloud networks, and
the prediction (inference) service is executed in the IoT devices. the prediction (inference) service is executed in the IoT devices.
In a distributed image processing application, some image processing Similarly, in a distributed image processing application, some image
functions can be similarly executed at the edge or in the cloud, processing functions can be executed at the edge or in the cloud. To
while preprocessing, which helps limiting the amount of uploaded limit the amount of data to be uploaded to central cloud functions,
data, is performed by the IoT device. IoT edge devices may pre-process data.
+----------------------------------------------+ +----------------------------------------------+
| Edge Computing Domain | | Edge Computing Domain |
| | | |
| +--------+ +--------+ +--------+ | | +--------+ +--------+ +--------+ |
| |Compute | |Compute | |Compute | | | |Compute | |Compute | |Compute | |
| |node/End| |node/End| .... |node/End| | | |Node/End| |Node/End| .... |Node/End| |
| |device 1| |device 2| .... |device m| | | |Device 1| |Device 2| .... |Device m| |
| +----+---+ +----+---+ +----+---+ | | +----+---+ +----+---+ +----+---+ |
| | | | | | | | | |
| +---+-------------+-----------------+--+ | | +---+-------------+-----------------+--+ |
| | IoT Edge Gateway | | | | IoT Edge Gateway | |
| +-----------+-------------------+------+ | | +-----------+-------------------+------+ |
| | | | | | | |
+--------------+-------------------+-----------+ +--------------+-------------------+-----------+
| | | |
+-----------+---------+ +------+-------+ +-----------+---------+ +------+-------+
| Remote network | | Service | | Remote Network | | Service |
|(e.g., cloud network)| | Operator(s) | |(e.g., cloud network)| | Operator(s) |
+-----------+---------+ +------+-------+ +-----------+---------+ +------+-------+
| | | |
+--------------+-------------------+-----------+ +--------------+-------------------+-----------+
| | | | | | | |
| +-----------+-------------------+------+ | | +-----------+-------------------+------+ |
| | IoT Edge Gateway | | | | IoT Edge Gateway | |
| +---+-------------+-----------------+--+ | | +---+-------------+-----------------+--+ |
| | | | | | | | | |
| +----+---+ +----+---+ +----+---+ | | +----+---+ +----+---+ +----+---+ |
| |Compute | |Compute | |Compute | | | |Compute | |Compute | |Compute | |
| |node/End| |node/End| .... |node/End| | | |Node/End| |Node/End| .... |Node/End| |
| |device 1| |device 2| .... |device n| | | |Device 1| |Device 2| .... |Device n| |
| +--------+ +--------+ +--------+ | | +--------+ +--------+ +--------+ |
| | | |
| Edge Computing Domain | | Edge Computing Domain |
+----------------------------------------------+ +----------------------------------------------+
Figure 2: Example: Machine Learning over a Distributed IoT Edge Figure 2: Example of Machine Learning over a Distributed IoT Edge
Computing System Computing System
In the following, we enumerate major edge computing domain In the following, we enumerate major edge computing domain
components. They are here loosely organized into OAM (Operations, components. Here, they are loosely organized into Operations,
Administration, and Maintenance), functional, and application Administration, and Maintenance (OAM); functional; and application
components, with the understanding that the distinction between these components, with the understanding that the distinction between these
classes may not always be clear, depending on actual system classes may not always be clear, depending on actual system
architectures. Some representative research challenges are architectures. Some representative research challenges are
associated with those functions. We used input from co-authors, IRTF associated with those functions. We used input from coauthors,
attendees, and some comprehensive reviews of the field ([Yousefpour], participants of T2TRG meetings, and some comprehensive reviews of the
[Zhang2], [Khan]). field ([Yousefpour], [Zhang2], and [Khan]).
4.3. OAM Components 4.3. OAM Components
Edge computing OAM extends beyond the network-related OAM functions Edge computing OAM extends beyond the network-related OAM functions
listed in [RFC6291]. In addition to infrastructure (network, listed in [RFC6291]. In addition to infrastructure (network,
storage, and computing resources), edge computing systems can also storage, and computing resources), edge computing systems can also
include computing environments (for VMs, software containers, include computing environments (for VMs, software containers, and
functions), IoT devices, data, and code. functions), IoT devices, data, and code.
Operation-related functions include performance monitoring for Operation-related functions include performance monitoring for
service-level agreement measurements, fault management and Service Level Agreement (SLA) measurements, fault management, and
provisioning for links, nodes, compute and storage resources, provisioning for links, nodes, compute and storage resources,
platforms, and services. Administration covers network/compute/ platforms, and services. Administration covers network/compute/
storage resources, platforms and services discovery, configuration, storage resources, platform and service discovery, configuration, and
and planning. Discovery during normal operation (e.g., discovery of planning. Discovery during normal operation (e.g., discovery of
compute or storage nodes by endpoints) is typically not included in compute or storage nodes by endpoints) is typically not included in
OAM; however, in this document, we do not address it separately. OAM; however, in this document, we do not address it separately.
Management covers the monitoring and diagnostics of failures, as well Management covers the monitoring and diagnostics of failures, as well
as means to minimize their occurrence and take corrective actions. as means to minimize their occurrence and take corrective actions.
This may include software update management and high service This may include software update management and high service
availability through redundancy and multipath communication. availability through redundancy and multipath communication.
Centralized (e.g., SDN) and decentralized management systems can be Centralized (e.g., Software-Defined Networking (SDN)) and
used. Finally, we arbitrarily chose to address data management as an decentralized management systems can be used. Finally, we
application component, however, in some systems, data management may arbitrarily chose to address data management as an application
be considered similar to a network management function. component; however, in some systems, data management may be
considered similar to a network management function.
We further detail a few relevant OAM components. We further detail a few relevant OAM components.
4.3.1. Resource Discovery and Authentication 4.3.1. Resource Discovery and Authentication
Discovery and authentication may target platforms and , Discovery and authentication may target platforms and infrastructure
infrastructure resources, such as computing, networking, and storage, resources, such as computing, networking, and storage, as well as
as well as other resources such as IoT devices, sensors, data, code other resources, such as IoT devices, sensors, data, code units,
units, services, applications, and users interacting with the system. services, applications, and users interacting with the system. In a
Broker-based solutions can be used, for example, using an IoT gateway broker-based system, an IoT gateway can act as a broker to discover
as a broker to discover IoT resources. More decentralized solutions IoT resources. More decentralized solutions can also be used in
can also be used in replacement or complement, for example, CoAP replacement of or in complement to the broker-based solutions; for
enables multicast discovery of an IoT device, and CoAP service example, CoAP enables multicast discovery of an IoT device and CoAP
discovery enables obtaining a list of resources made available by service discovery enables one to obtain a list of resources made
this device [RFC7252]. For device authentication, current available by this device [RFC7252]. For device authentication,
centralized gateway-based systems rely on the installation of a current centralized gateway-based systems rely on the installation of
secret on IoT devices and computing devices (e.g., a device a secret on IoT devices and computing devices (e.g., a device
certificate stored in a hardware security module, or a combination of certificate stored in a hardware security module or a combination of
code and data stored in a trusted execution environment). code and data stored in a trusted execution environment).
Related challenges include: Related challenges include:
* Discovery, authentication, and trust establishment between IoT * Discovery, authentication, and trust establishment between IoT
devices, compute nodes, and platforms, with regard to concerns devices, compute nodes, and platforms, with regard to concerns
such as mobility, heterogeneous devices and networks, scale, such as mobility, heterogeneous devices and networks, scale,
multiple trust domains, constrained devices, anonymity, and multiple trust domains, constrained devices, anonymity, and
traceability. traceability.
* Intermittent connectivity to the Internet, removing the need to * Intermittent connectivity to the Internet, removing the need to
rely on a third-party authority [Echeverria]. rely on a third-party authority [Echeverria].
* Resiliency to failure [Harchol], denial of service attacks, easier * Resiliency to failure [Harchol], denial-of-service attacks, and
physical access for attackers. easier physical access for attackers.
4.3.2. Edge Organization and Federation 4.3.2. Edge Organization and Federation
In a distributed system context, once edge devices have discovered In a distributed system context, once edge devices have discovered
and authenticated each other, they can be organized, or self- and authenticated each other, they can be organized or self-organized
organized, into hierarchies or clusters. The organizational into hierarchies or clusters. The organizational structure may range
structure may range from centralized to peer-to-peer, or it may be from centralized to peer-to-peer, or it may be closely tied to other
closely tied to other systems. Such groups can also form federations systems. Such groups can also form federations with other edges or
with other edges or with remote clouds. with remote clouds.
Related challenges include: Related challenges include:
* Support for scaling, and enabling fault-tolerance or self-healing * Support for scaling and enabling fault tolerance or self-healing
[Jeong]. In addition to using a hierarchical organization to cope [Jeong]. In addition to using a hierarchical organization to cope
with scaling, another available and possibly complementary with scaling, another available and possibly complementary
mechanism is multicast ([RFC7390] [I-D.ietf-core-groupcomm-bis]). mechanism is multicast [RFC7390] [CORE-GROUPCOMM-BIS]. Other
Other approaches include relying on blockchains [Ali]. approaches include relying on blockchains [Ali].
* Integration of edge computing with virtualized Radio Access * Integration of edge computing with virtualized Radio Access
Networks (Fog RAN) [I-D.bernardos-sfc-fog-ran] and 5G access Networks (Fog RAN) [SFC-FOG-RAN] and 5G access networks.
networks.
* Sharing resources in multi-vendor/operator scenarios, to optimize * Sharing resources in multi-vendor/operator scenarios to optimize
criteria such as profit [Anglano], resource usage, latency, and criteria such as profit [Anglano], resource usage, latency, and
energy consumption. energy consumption.
* Capacity planning, placement of infrastructure nodes to minimize * Capacity planning, placement of infrastructure nodes to minimize
delay [Fan], cost, energy, etc. delay [Fan], cost, energy, etc.
* Incentives for participation, for example, in peer-to-peer * Incentives for participation, for example, in peer-to-peer
federation schemes. federation schemes.
* Design of federated AI over IoT edge computing systems [Brecko], * Design of federated AI over IoT edge computing systems [Brecko],
for example, for anomaly detection. for example, for anomaly detection.
4.3.3. Multi-Tenancy and Isolation 4.3.3. Multi-Tenancy and Isolation
Some IoT edge computing systems make use of virtualized (compute, Some IoT edge computing systems make use of virtualized (compute,
storage and networking) resources to address the need for secure storage, and networking) resources to address the need for secure
multi-tenancy at the edge. This leads to "edge clouds" that share multi-tenancy at the edge. This leads to "edge clouds" that share
properties with remotes clouds and can reuse some of their properties with remote clouds and can reuse some of their ecosystems.
ecosystems. Virtualization function management is largely covered by Virtualization function management is largely covered by ETSI NFV and
ETSI NFV and MEC standards and recommendations. Projects such as MEC standards and recommendations. Projects such as [LFEDGE-EVE]
[LFEDGE-EVE] further cover virtualization and its management in further cover virtualization and its management in distributed edge
distributed edge-computing settings. computing settings.
Related challenges include: Related challenges include:
* Adapting cloud management platforms to the edge, to account for * Adapting cloud management platforms to the edge to account for its
its distributed nature, e.g., using Conflict-free Replicated Data distributed nature, heterogeneity, need for customization, and
Types (CRDT) [Jeffery], heterogeneity and customization, e.g., limited resources (for example, using Conflict-free Replicated
using intent-based management mechanisms [Cao], and limited Data Types (CRDTs) [Jeffery] or intent-based management mechanisms
resources. [Cao]).
* Minimizing virtual function instantiation time and resource usage. * Minimizing virtual function instantiation time and resource usage.
4.4. Functional Components 4.4. Functional Components
4.4.1. In-Network Computation 4.4.1. In-Network Computation
A core function of IoT edge computing is to enable local computation A core function of IoT edge computing is to enable local computation
on a node at the network edge, typically for application-layer on a node at the network edge, typically for application-layer
processing, such as processing input data from sensors, making local processing, such as processing input data from sensors, making local
decisions, preprocessing data, offloading computation on behalf of a decisions, preprocessing data, and offloading computation on behalf
device, service, or user. Related functions include orchestrating of a device, service, or user. Related functions include
computation (in a centralized or distributed manner) and managing orchestrating computation (in a centralized or distributed manner)
application lifecycles. Support for in-network computation may vary and managing application life cycles. Support for in-network
in terms of capability, for example, computing nodes can host virtual computation may vary in terms of capability; for example, computing
machines, software containers, software actors, uni-kernels running nodes can host virtual machines, software containers, software
stateful or stateless code, or a rule engine providing an API to actors, unikernels running stateful or stateless code, or a rule
register actions in response to conditions such as IoT device ID, engine providing an API to register actions in response to conditions
sensor values to check, thresholds, etc. (such as an IoT device ID, sensor values to check, thresholds, etc.).
Edge offloading includes offloading to and from an IoT device, and to Edge offloading includes offloading to and from an IoT device and to
and from a network node. [Cloudlets] offer an example of offloading and from a network node. [Cloudlets] describes an example of
computation from an end device to a network node. In contrast, offloading computation from an end device to a network node. In
oneM2M is an example of a system that allows a cloud-based IoT contrast, oneM2M is an example of a system that allows a cloud-based
platform to transfer resources and tasks to a target edge node IoT platform to transfer resources and tasks to a target edge node
[oneM2M-TR0052]. Once transferred, the edge node can directly [oneM2M-TR0052]. Once transferred, the edge node can directly
support IoT devices that it serves with the service offloaded by the support IoT devices that it serves with the service offloaded by the
cloud (e.g., group management, location management, etc.). cloud (e.g., group management, location management, etc.).
QoS can be provided in some systems through the combination of QoS can be provided in some systems through the combination of
network QoS (e.g., traffic engineering or wireless resource network QoS (e.g., traffic engineering or wireless resource
scheduling) and compute/storage resource allocations. For example, scheduling) and compute/storage resource allocations. For example,
in some systems, a bandwidth manager service can be exposed to enable in some systems, a bandwidth manager service can be exposed to enable
allocation of the bandwidth to/from an edge-computing application allocation of the bandwidth to/from an edge computing application
instance. instance.
In-network computation can leverage the underlying services, provided In-network computation can leverage the underlying services provided
using data generated by IoT devices and access networks. Such using data generated by IoT devices and access networks. Such
services include IoT device location, radio network information, services include IoT device location, radio network information,
bandwidth management and congestion management (e.g., the congestion bandwidth management, and congestion management (e.g., the congestion
management feature of oneM2M [oneM2M-TR0052]). management feature of oneM2M [oneM2M-TR0052]).
Related challenges include: Related challenges include:
* (Computation placement) Selecting, in a centralized or * Computation placement: in a centralized or distributed/peer-to-
distributed/peer-to-peer manner, an appropriate compute device peer manner, selecting an appropriate compute device. The
based on available resources, location of data input and data selection is based on available resources, location of data input
sinks, compute node properties, etc., and with varying goals and data sinks, compute node properties, etc. with varying goals.
including end-to-end latency, privacy, high availability, energy These goals include end-to-end latency, privacy, high
conservation, or network efficiency, for example, using load- availability, energy conservation, or network efficiency (for
balancing techniques to avoid congestion. example, using load-balancing techniques to avoid congestion).
* Onboarding code on a platform or computing device, and invoking * Onboarding code on a platform or computing device and invoking
remote code execution, possibly as part of a distributed remote code execution, possibly as part of a distributed
programming model and with respect to similar concerns of latency, programming model and with respect to similar concerns of latency,
privacy, etc.: For example, offloading can be included in a privacy, etc. For example, offloading can be included in a
vehicular scenario [Grewe]. These operations should deal with vehicular scenario [Grewe]. These operations should deal with
heterogeneous compute nodes [Schafer], and may also support end heterogeneous compute nodes [Schafer] and may also support end
devices, including IoT devices, as compute nodes [Larrea]. devices, including IoT devices, as compute nodes [Larrea].
* Adapting Quality of Results (QoR) for applications where a perfect * Adapting Quality of Results (QoR) for applications where a perfect
result is not necessary [Li]. result is not necessary [Li].
* Assisted or automatic partitioning of code: for example, for * Assisted or automatic partitioning of code. For example, for
application programs [I-D.sarathchandra-coin-appcentres] or application programs [COIN-APPCENTRES] or network programs
network programs [I-D.hsingh-coinrg-reqs-p4comp]. [REQS-P4COMP].
* Supporting computation across trust domains: for example, * Supporting computation across trust domains. For example,
verifying computation results. verifying computation results.
* Support for computation mobility: relocating an instance from one * Supporting computation mobility: relocating an instance from one
compute node to another, while maintaining a given service level; compute node to another while maintaining a given service level;
session continuity when communicating with end devices that are session continuity when communicating with end devices that are
mobile, possibly at high speed (e.g., in vehicular scenarios); mobile, possibly at high speed (e.g., in vehicular scenarios);
defining lightweight execution environments for secure code defining lightweight execution environments for secure code
mobility, for example, using WebAssembly [Nieke]. mobility, for example, using WebAssembly [Nieke].
* Defining, managing, and verifying Service Level Agreements (SLA) * Defining, managing, and verifying SLAs for edge computing systems;
for edge-computing systems: pricing is a challenging task. pricing is a challenging task.
4.4.2. Edge Storage and Caching 4.4.2. Edge Storage and Caching
Local storage or caching enables local data processing (e.g., Local storage or caching enables local data processing (e.g.,
preprocessing or analysis) as well as delayed data transfer to the preprocessing or analysis) as well as delayed data transfer to the
cloud or delayed physical shipping. An edge node may offer local cloud or delayed physical shipping. An edge node may offer local
data storage (in which persistence is subject to retention policies), data storage (in which persistence is subject to retention policies),
caching, or both. Caching generally refers to temporary storage to caching, or both. Generally, "caching" refers to temporary storage
improve performance without persistence guarantees. An edge-caching to improve performance without persistence guarantees. An edge-
component manages data persistence, for example, it schedules the caching component manages data persistence; for example, it schedules
removal of data when it is no longer needed. Other related aspects the removal of data when it is no longer needed. Other related
include the authentication and encryption of data. Edge storage and aspects include the authentication and encryption of data. Edge
caching can take the form of a distributed storage systems. storage and caching can take the form of a distributed storage
system.
Related challenges include: Related challenges include:
* (Cache and data placement) Using cache positioning and data * Cache and data placement: using cache positioning and data
placement strategies to minimize data retrieval delay [Liu] and placement strategies to minimize data retrieval delay [Liu] and
energy consumption. Caches may be positioned in the access energy consumption. Caches may be positioned in the access-
network infrastructure or on end devices. network infrastructure or on end devices.
* Maintaining consistency, freshness, reliability, and privacy of * Maintaining consistency, freshness, reliability, and privacy of
stored/cached data in systems that are distributed, constrained, stored/cached data in systems that are distributed, constrained,
and dynamic (e.g., owing to end devices and computing nodes churn and dynamic (e.g., due to node mobility, energy-saving regimes,
or mobility), and which can have additional data governance and disruptions) and which can have additional data governance
constraints on data storage location. For example, [Mortazavi] constraints on data storage location. For example, [Mortazavi]
leverages a hierarchical storage organization. Freshness-related describes leveraging a hierarchical storage organization.
metrics include the age of information [Yates] that captures the Freshness-related metrics include the age of information [Yates]
timeliness of information received from a sender (e.g., an IoT that captures the timeliness of information received from a sender
device). (e.g., an IoT device).
4.4.3. Communication 4.4.3. Communication
An edge cloud may provide a northbound data plane or management plane An edge cloud may provide a northbound data plane or management plane
interface to a remote network, such as a cloud, home or enterprise interface to a remote network, such as a cloud, home, or enterprise
network. This interface does not exist in stand-alone (local-only) network. This interface does not exist in stand-alone (local-only)
scenarios. To support such an interface when it exists, an edge scenarios. To support such an interface when it exists, an edge
computing component needs to expose an API, deal with authentication computing component needs to expose an API, deal with authentication
and authorization, and support secure communication. and authorization, and support secure communication.
An edge cloud may provide an API or interface to local or mobile An edge cloud may provide an API or interface to local or mobile
users, for example, to provide access to services and applications, users, for example, to provide access to services and applications or
or to manage data published by local/mobile devices. to manage data published by local/mobile devices.
Edge-computing nodes communicate with IoT devices over a southbound Edge computing nodes communicate with IoT devices over a southbound
interface, typically for data acquisition and IoT device management. interface, typically for data acquisition and IoT device management.
Communication brokering is a typical function of IoT edge computing Communication brokering is a typical function of IoT edge computing
that facilitates communication with IoT devices, enabling clients to that facilitates communication with IoT devices, enables clients to
register as recipients for data from devices, as well as forwarding/ register as recipients for data from devices, forwards traffic to or
routing of traffic to or from IoT devices, enabling various data from IoT devices, enables various data discovery and redistribution
discovery and redistribution patterns, for example, north-south with patterns (for example, north-south with clouds and east-west with
clouds, east-west with other edge devices other edge devices [EDGE-DATA-DISCOVERY-OVERVIEW]). Another related
[I-D.mcbride-edge-data-discovery-overview]. Another related aspect aspect is dispatching alerts and notifications to interested
is dispatching alerts and notifications to interested consumers both consumers both inside and outside the edge computing domain.
inside and outside the edge-computing domain. Protocol translation, Protocol translation, analytics, and video transcoding can also be
analytics, and video transcoding can also be performed when performed when necessary. Communication brokering may be centralized
necessary. Communication brokering may be centralized in some in some systems, for example, using a hub-and-spoke message broker or
systems, for example, using a hub-and-spoke message broker, or
distributed with message buses, possibly in a layered bus approach. distributed with message buses, possibly in a layered bus approach.
Distributed systems can leverage direct communication between end Distributed systems can leverage direct communication between end
devices over device-to-device links. A broker can ensure devices over device-to-device links. A broker can ensure
communication reliability and traceability and, in some cases, communication reliability and traceability and, in some cases,
transaction management. transaction management.
Related challenges include: Related challenges include:
* Defining edge computing abstractions, such as PaaS [Yangui], * Defining edge computing abstractions, such as PaaS [Yangui],
suitable for users and cloud systems to interact with edge suitable for users and cloud systems to interact with edge
computing systems and dealing with interoperability issues such as computing systems and dealing with interoperability issues, such
data model heterogeneity. as data-model heterogeneity.
* Enabling secure and resilient communication between IoT devices * Enabling secure and resilient communication between IoT devices
and remote cloud, for example, through multipath support. and a remote cloud, for example, through multipath support.
4.5. Application Components 4.5. Application Components
IoT edge computing can host applications, such as those mentioned in IoT edge computing can host applications, such as those mentioned in
Section 2.4. While describing the components of individual Section 2.4. While describing the components of individual
applications is out of our scope, some of those applications share applications is out of our scope, some of those applications share
similar functions, such as IoT device management and data management, similar functions, such as IoT device management and data management,
as described below. as described below.
4.5.1. IoT Device Management 4.5.1. IoT Device Management
IoT device management includes managing information regarding IoT IoT device management includes managing information regarding IoT
devices, including their sensors, and how to communicate with them. devices, including their sensors and how to communicate with them.
Edge computing addresses the scalability challenges of a large number Edge computing addresses the scalability challenges of a large number
of IoT devices by separating the scalability domain into edge/local of IoT devices by separating the scalability domain into edge/local
networks and remote networks. For example, in the context of the networks and remote networks. For example, in the context of the
oneM2M standard, a device management functionality (called "software oneM2M standard, a device management functionality (called "software
campaign" in oneM2M) enables the installation, deletion, activation, campaign" in oneM2M) enables the installation, deletion, activation,
and deactivation of software functions/services on a potentially and deactivation of software functions/services on a potentially
large number of edge nodes [oneM2M-TR0052]. Using a dashboard or large number of edge nodes [oneM2M-TR0052]. Using a dashboard or
management software, a service provider issues these requests through management software, a service provider issues these requests through
an IoT cloud platform supporting the software campaign functionality. an IoT cloud platform supporting the software campaign functionality.
Challenges listed in Section 4.3.1 may be applicable to IoT devices The challenges listed in Section 4.3.1 may be applicable to IoT
management as well. device management as well.
4.5.2. Data Management and Analytics 4.5.2. Data Management and Analytics
Data storage and processing at the edge are major aspects of IoT edge Data storage and processing at the edge are major aspects of IoT edge
computing, directly addressing the high-level IoT challenges listed computing, directly addressing the high-level IoT challenges listed
in Section 3. Data analysis, for example, through AI/ML tasks in Section 3. Data analysis, for example, through AI/ML tasks
performed at the edge, may benefit from specialized hardware support performed at the edge, may benefit from specialized hardware support
on the computing nodes. on the computing nodes.
Related challenges include: Related challenges include:
* Addressing concerns regarding resource usage, security, and * Addressing concerns regarding resource usage, security, and
privacy when sharing, processing, discovering, or managing data: privacy when sharing, processing, discovering, or managing data:
for example presenting data in views composed of an aggregation of for example, presenting data in views composed of an aggregation
related data [Zhang]; protecting data communication between of related data [Zhang], protecting data communication between
authenticated peers [Basudan], classifying data (e.g., in terms of authenticated peers [Basudan], classifying data (e.g., in terms of
privacy, importance, validity), and compressing and encrypting privacy, importance, and validity), and compressing and encrypting
data, for example, using homomorphic encryption to directly data, for example, using homomorphic encryption to directly
process encrypted data [Stanciu]. process encrypted data [Stanciu].
* Other concerns regarding edge data discovery (e.g., streaming * Other concerns regarding edge data discovery (e.g., streaming
data, metadata, and events) include siloization and lack of data, metadata, and events) include siloization and lack of
standards in edge environments that can be dynamic (e.g., standards in edge environments that can be dynamic (e.g.,
vehicular networks) and heterogeneous vehicular networks) and heterogeneous
[I-D.mcbride-edge-data-discovery-overview]. [EDGE-DATA-DISCOVERY-OVERVIEW].
* Data-driven programming models [Renart], for example, event-based, * Data-driven programming models [Renart], for example, those that
including handling naming and data abstractions. are event based, including handling naming and data abstractions.
* Data integration in an environment that without data * Data integration in an environment without data standardization or
standardization, or where different sources use different where different sources use different ontologies
ontologies [Farnbauer-Schmidt]. [Farnbauer-Schmidt].
* Addressing concerns such as limited resources, privacy, dynamic, * Addressing concerns such as limited resources, privacy, and
and heterogeneous environments to deploy machine learning at the dynamic and heterogeneous environments to deploy machine learning
edge: for example, making machine learning more lightweight and at the edge: for example, making machine learning more lightweight
distributed (e.g., enabling distributed inference at the edge), and distributed (e.g., enabling distributed inference at the
supporting shorter training times and simplified models, and edge), supporting shorter training times and simplified models,
supporting models that can be compressed for efficient and supporting models that can be compressed for efficient
communication [Murshed]. communication [Murshed].
* Although edge computing can support IoT services independently of * Although edge computing can support IoT services independently of
cloud computing, it can also be connected to cloud computing. cloud computing, it can also be connected to cloud computing.
Thus, the relationship between IoT edge computing and cloud Thus, the relationship between IoT edge computing and cloud
computing, with regard to data management, is another potential computing, with regard to data management, is another potential
challenge [ISO_TR]. challenge [ISO_TR].
4.6. Simulation and Emulation Environments 4.6. Simulation and Emulation Environments
IoT Edge Computing introduces new challenges to the simulation and IoT edge computing introduces new challenges to the simulation and
emulation tools used by researchers and developers. A varied set of emulation tools used by researchers and developers. A varied set of
applications, networks, and computing technologies can coexist in a applications, networks, and computing technologies can coexist in a
distributed system, making modeling difficult. Scale, mobility, and distributed system, making modeling difficult. Scale, mobility, and
resource management are additional challenges [SimulatingFog]. resource management are additional challenges [SimulatingFog].
Tools include simulators, where simplified application logic runs on Tools include simulators, where simplified application logic runs on
top of a fog network model, and emulators, where actual applications top of a fog network model, and emulators, where actual applications
can be deployed, typically in software containers, over a cloud can be deployed, typically in software containers, over a cloud
infrastructure (e.g., Docker and Kubernetes) running over a network infrastructure (e.g., Docker and Kubernetes) running over a network
emulating network edge conditions such as variable delays, throughput emulating network edge conditions, such as variable delays,
and mobility events. To gain in scale, emulated and simulated throughput, and mobility events. To gain in scale, emulated and
systems can be used together in hybrid federation-based approaches simulated systems can be used together in hybrid federation-based
[PseudoDynamicTesting], whereas to gain in realism, physical devices approaches [PseudoDynamicTesting]; whereas to gain in realism,
can be interconnected with emulated systems. Examples of related physical devices can be interconnected with emulated systems.
work and platforms include the publicly accessible MEC sandbox work Examples of related work and platforms include the publicly
recently initiated in ETSI [ETSI_Sandbox], and open source simulators accessible MEC sandbox work recently initiated in ETSI [ETSI_Sandbox]
and emulators ([AdvantEDGE] emulator and tools cited in and open-source simulators and emulators ([AdvantEDGE] emulator and
[SimulatingFog]). EdgeNet [Senel] is a globally distributed edge tools cited in [SimulatingFog]). EdgeNet [Senel] is a globally
cloud for Internet researchers, using nodes contributed by distributed edge cloud for Internet researchers, which uses nodes
institutions, and based on Docker for containerization and Kubernetes contributed by institutions and which is based on Docker for
for deployment and node management. containerization and Kubernetes for deployment and node management.
Digital twins are virtual instances of a physical system (twin) that Digital twins are virtual instances of a physical system (twin) that
are continually updated with the latter's performance, maintenance, are continually updated with the latter's performance, maintenance,
and health status data throughout the life cycle of the physical and health status data throughout the life cycle of the physical
system. [Madni]. In contrast to a traditional emulation or system [Madni]. In contrast to an emulation or simulated
simulated environment, digital twins, once generated, are maintained environment, digital twins, once generated, are maintained in sync by
in sync by their physical twin, which can be, among many other their physical twin, which can be, among many other instances, an IoT
instances, an IoT device, edge device, an edge network. The benefits device, edge device, or an edge network. The benefits of digital
of digital twins go beyond those of emulation and include accelerated twins go beyond those of emulation and include accelerated business
business processes, enhanced productivity, and faster innovation with processes, enhanced productivity, and faster innovation with reduced
reduced costs [I-D.irtf-nmrg-network-digital-twin-arch]. costs [NETWORK-DIGITAL-TWIN-ARCH].
5. Security Considerations 5. Security Considerations
Privacy and security are drivers of the adoption of edge computing Privacy and security are drivers of the adoption of edge computing
for the IoT (Section 3.4). As discussed in Section 4.3.1, for the IoT (Section 3.4). As discussed in Section 4.3.1,
authentication and trust (among computing nodes, management nodes, authentication and trust (among computing nodes, management nodes,
and end devices) can be challenging as scale, mobility, and and end devices) can be challenging as scale, mobility, and
heterogeneity increase. The sometimes disconnected nature of edge heterogeneity increase. The sometimes disconnected nature of edge
resources can avoid reliance on third-party authorities. Distributed resources can avoid reliance on third-party authorities. Distributed
edge computing is exposed reliability and denial of service attacks. edge computing is exposed to reliability and denial-of-service
Personal or proprietary IoT data leakage is also a major threat, attacks. A personal or proprietary IoT data leakage is also a major
particularly because of the distributed nature of the systems threat, particularly because of the distributed nature of the systems
(Section 4.5.2). Furthermore, blockchain-based distributed IoT edge (Section 4.5.2). Furthermore, blockchain-based distributed IoT edge
computing must be designed for privacy, since public blockchain computing must be designed for privacy, since public blockchain
addressing does not guarantee absolute anonymity [Ali]. addressing does not guarantee absolute anonymity [Ali].
However, edge computing also offers solutions in the security space: However, edge computing also offers solutions in the security space:
maintaining privacy by computing sensitive data closer to data maintaining privacy by computing sensitive data closer to data
generators is a major use case for IoT edge computing. An edge cloud generators is a major use case for IoT edge computing. An edge cloud
can be used to perform actions based on sensitive data or to can be used to perform actions based on sensitive data or to
anonymize or aggregate data prior to transmission to a remote cloud anonymize or aggregate data prior to transmission to a remote cloud
server. Edge computing communication brokering functions can also be server. Edge computing communication brokering functions can also be
used to secure communication between edge and cloud networks. used to secure communication between edge and cloud networks.
6. Conclusion 6. Conclusion
IoT edge computing plays an essential role, complementary to the IoT edge computing plays an essential role, complementary to the
cloud, in enabling IoT systems in certain situations. In this cloud, in enabling IoT systems in certain situations. In this
document, we presented use cases and listing the core challenges document, we presented use cases and listed the core challenges faced
faced by IoT that drive the need for IoT edge computing. The first by the IoT that drive the need for IoT edge computing. Therefore,
part of this document may therefore help focus future research the first part of this document may help focus future research
efforts on the aspects of IoT edge computing where it is most useful. efforts on the aspects of IoT edge computing where it is most useful.
The second part of this document presents a general system model and The second part of this document presents a general system model and
structured overview of the associated research challenges and related structured overview of the associated research challenges and related
work. The structure, based on the system model, is not meant to be work. The structure, based on the system model, is not meant to be
restrictive and exists for the purpose of having a link between restrictive and exists for the purpose of having a link between
individual research areas and where they are applicable in an IoT individual research areas and where they are applicable in an IoT
edge computing system. edge computing system.
7. IANA Considerations 7. IANA Considerations
This document has no IANA actions. This document has no IANA actions.
8. Acknowledgements 8. Informative References
The authors would like to thank Joo-Sang Youn, Akbar Rahman, Michel
Roy, Robert Gazda, Rute Sofia, Thomas Fossati, Chonggang Wang, Marie-
José Montpetit, Carlos J. Bernardos, Milan Milenkovic, Dale Seed,
JaeSeung Song, Roberto Morabito, Carsten Bormann and Ari Keränen for
their valuable comments and suggestions on this document.
9. Informative References
[AdvantEDGE] [AdvantEDGE]
"Mobile Edge Emulation Platform", Source Code Repository, "AdvantEDGE, Mobile Edge Emulation Platform", commit
2020, <https://github.com/InterDigitalInc/AdvantEDGE>. 8f6edbe, May 2023,
<https://github.com/InterDigitalInc/AdvantEDGE>.
[Ali] Ali, M. S., Vecchio, M., and F. Antonelli, "Enabling a [Ali] Ali, M., Vecchio, M., and F. Antonelli, "Enabling a
Blockchain-Based IoT Edge", IEEE Internet of Things Blockchain-Based IoT Edge", IEEE Internet of Things
Magazine pp. 24-29, DOI 10.1109/IOTM.2019.1800024, 2018, Magazine, vol. 1, no.2, pp. 24-29,
DOI 10.1109/IOTM.2019.1800024, December 2018,
<https://doi.org/10.1109/IOTM.2019.1800024>. <https://doi.org/10.1109/IOTM.2019.1800024>.
[Anglano] Anglano, C., Canonico, M., Castagno, P., Guazzone, M., and [Anglano] Anglano, C., Canonico, M., Castagno, P., Guazzone, M., and
M. Sereno, "A game-theoretic approach to coalition M. Sereno, "A game-theoretic approach to coalition
formation in fog provider federations", 2018 Third formation in fog provider federations", 2018 Third
International Conference on Fog and Mobile Edge International Conference on Fog and Mobile Edge Computing
Computing (FMEC), DOI 10.1109/fmec.2018.8364054, April (FMEC), DOI 10.1109/fmec.2018.8364054, April 2018,
2018, <https://doi.org/10.1109/fmec.2018.8364054>. <https://doi.org/10.1109/fmec.2018.8364054>.
[Ashton] Ashton, K., "That Internet of Things thing", RFID J. vol. [Ashton] Ashton, K., "That 'Internet of Things' Thing", RFID
22, no. 7, pp. 97-114, 2009, Journal, vol. 22, no. 7, pp. 97-114, June 2009,
<http://www.itrco.jp/libraries/RFIDjournal- <http://www.itrco.jp/libraries/RFIDjournal-
That%20Internet%20of%20Things%20Thing.pdf>. That%20Internet%20of%20Things%20Thing.pdf>.
[Basudan] Basudan, S., Lin, X., and K. Sankaranarayanan, "A Privacy- [Basudan] Basudan, S., Lin, X., and K. Sankaranarayanan, "A Privacy-
Preserving Vehicular Crowdsensing-Based Road Surface Preserving Vehicular Crowdsensing-Based Road Surface
Condition Monitoring System Using Fog Computing", IEEE Condition Monitoring System Using Fog Computing", IEEE
Internet of Things Journal vol. 4, no. 3, pp. 772-782, Internet of Things Journal, vol. 4, no. 3, pp. 772-782,
DOI 10.1109/jiot.2017.2666783, June 2017, DOI 10.1109/jiot.2017.2666783, June 2017,
<https://doi.org/10.1109/jiot.2017.2666783>. <https://doi.org/10.1109/jiot.2017.2666783>.
[Botta] Botta, A., de Donato, W., Persico, V., and A. Pescape, [Botta] Botta, A., de Donato, W., Persico, V., and A. Pescapé,
"Integration of Cloud computing and Internet of Things: A "Integration of Cloud computing and Internet of Things: A
survey", Future Generation Computer Systems vol. 56, pp. survey", Future Generation Computer Systems, vol. 56, pp.
684-700, DOI 10.1016/j.future.2015.09.021, March 2016, 684-700, DOI 10.1016/j.future.2015.09.021, March 2016,
<https://doi.org/10.1016/j.future.2015.09.021>. <https://doi.org/10.1016/j.future.2015.09.021>.
[Brecko] Brecko, A., Kajati, E., Koziorek, J., and I. Zolotova, [Brecko] Brecko, A., Kajáti, E., Koziorek, J., and I. Zolotová,
"Federated Learning for Edge Computing: A Survey", Applied "Federated Learning for Edge Computing: A Survey", Applied
Sciences 12, no. 18 9124, DOI 10.3390/app12189124, 2022, Sciences 12(18):9124, DOI 10.3390/app12189124, September
<https://doi.org/10.3390/app12189124>. 2022, <https://doi.org/10.3390/app12189124>.
[Cao] Cao, L., Merican, A., Tootaghaj, D., Ahmed, F., Sharma, [Cao] Cao, L., Merican, A., Tootaghaj, D., Ahmed, F., Sharma,
P., and V. Saxena, "eCaaS: A Management Framework of Edge P., and V. Saxena, "eCaaS: A Management Framework of Edge
Container as a Service for Business Workload", Proceedings Container as a Service for Business Workload", Proceedings
of the 4th International Workshop on Edge Systems, of the 4th International Workshop on Edge Systems,
Analytics and Networking, DOI 10.1145/3434770.3459741, Analytics and Networking, DOI 10.1145/3434770.3459741,
April 2021, <https://doi.org/10.1145/3434770.3459741>. April 2021, <https://doi.org/10.1145/3434770.3459741>.
[Chen] Chen, B., Wan, J., Celesti, A., Li, D., Abbas, H., and Q. [Chen] Chen, B., Wan, J., Celesti, A., Li, D., Abbas, H., and Q.
Zhang, "Edge Computing in IoT-Based Manufacturing", IEEE Zhang, "Edge Computing in IoT-Based Manufacturing", IEEE
Communications Magazine vol. 56, no. 9, pp. 103-109, Communications Magazine, vol. 56, no. 9, pp. 103-109,
DOI 10.1109/mcom.2018.1701231, September 2018, DOI 10.1109/mcom.2018.1701231, September 2018,
<https://doi.org/10.1109/mcom.2018.1701231>. <https://doi.org/10.1109/mcom.2018.1701231>.
[Chiang] Chiang, M. and T. Zhang, "Fog and IoT: An Overview of [Chiang] Chiang, M. and T. Zhang, "Fog and IoT: An Overview of
Research Opportunities", IEEE Internet of Things Research Opportunities", IEEE Internet of Things Journal,
Journal vol. 3, no. 6, pp. 854-864, vol. 3, no. 6, pp. 854-864, DOI 10.1109/jiot.2016.2584538,
DOI 10.1109/jiot.2016.2584538, December 2016, December 2016,
<https://doi.org/10.1109/jiot.2016.2584538>. <https://doi.org/10.1109/jiot.2016.2584538>.
[chipmunk] Shin, Y., Park, S., Ko, N., and A. Jeong, "Chipmunk: [Chipmunk] Shin, Y., Park, S., Ko, N., and A. Jeong, "Chipmunk:
Distributed Object Storage for NDN", ACM, Proceedings of Distributed Object Storage for NDN", Proceedings of the
the 7th ACM Conference on Information-Centric Networking, 7th ACM Conference on Information-Centric Networking, ACM,
DOI 10.1145/3405656.3420231, September 2020, DOI 10.1145/3405656.3420231, September 2020,
<https://doi.org/10.1145/3405656.3420231>. <https://doi.org/10.1145/3405656.3420231>.
[Cloudlets] [Cloudlets]
Satyanarayanan, M., Bahl, P., Caceres, R., and N. Davies, Satyanarayanan, M., Bahl, P., Caceres, R., and N. Davies,
"The Case for VM-Based Cloudlets in Mobile Computing", "The Case for VM-Based Cloudlets in Mobile Computing",
IEEE Pervasive Computing vol. 8, no. 4, pp. 14-23, IEEE Pervasive Computing, vol. 8, no. 4, pp. 14-23,
DOI 10.1109/mprv.2009.82, October 2009, DOI 10.1109/mprv.2009.82, October 2009,
<https://doi.org/10.1109/mprv.2009.82>. <https://doi.org/10.1109/mprv.2009.82>.
[COIN-APPCENTRES]
Trossen, D., Sarathchandra, C., and M. Boniface, "In-
Network Computing for App-Centric Micro-Services", Work in
Progress, Internet-Draft, draft-sarathchandra-coin-
appcentres-04, 26 January 2021,
<https://datatracker.ietf.org/doc/html/draft-
sarathchandra-coin-appcentres-04>.
[CORE-GROUPCOMM-BIS]
Dijk, E., Wang, C., and M. Tiloca, "Group Communication
for the Constrained Application Protocol (CoAP)", Work in
Progress, Internet-Draft, draft-ietf-core-groupcomm-bis-
10, 23 October 2023,
<https://datatracker.ietf.org/doc/html/draft-ietf-core-
groupcomm-bis-10>.
[Echeverria] [Echeverria]
Echeverria, S., Klinedinst, D., Williams, K., and G. Echeverría, S., Klinedinst, D., Williams, K., and G.
Lewis, "Establishing Trusted Identities in Disconnected Lewis, "Establishing Trusted Identities in Disconnected
Edge Environments", 2016 IEEE/ACM Symposium on Edge Edge Environments", 2016 IEEE/ACM Symposium on Edge
Computing (SEC), DOI 10.1109/sec.2016.27, October 2016, Computing (SEC), DOI 10.1109/sec.2016.27, October 2016,
<https://doi.org/10.1109/sec.2016.27>. <https://doi.org/10.1109/sec.2016.27>.
[EDGE-COMPUTING-BACKGROUND]
de Foy, X., Hong, J., Hong, Y., Kovatsch, M., Schooler,
E., and D. Kutscher, "IoT Edge Computing: Initiatives,
Projects and Products", Work in Progress, Internet-Draft,
draft-defoy-t2trg-iot-edge-computing-background-00, 25 May
2020, <https://datatracker.ietf.org/doc/html/draft-defoy-
t2trg-iot-edge-computing-background-00>.
[EDGE-DATA-DISCOVERY-OVERVIEW]
McBride, M., Kutscher, D., Schooler, E., Bernardos, C. J.,
Lopez, D., and X. de Foy, "Edge Data Discovery for COIN",
Work in Progress, Internet-Draft, draft-mcbride-edge-data-
discovery-overview-05, 1 November 2020,
<https://datatracker.ietf.org/doc/html/draft-mcbride-edge-
data-discovery-overview-05>.
[ENERGY] Beckel, C., Sadamori, L., Staake, T., and S. Santini, [ENERGY] Beckel, C., Sadamori, L., Staake, T., and S. Santini,
"Revealing household characteristics from smart meter "Revealing household characteristics from smart meter
data", Energy vol. 78, pp. 397-410, data", Energy, vol. 78, pp. 397-410,
DOI 10.1016/j.energy.2014.10.025, December 2014, DOI 10.1016/j.energy.2014.10.025, December 2014,
<https://doi.org/10.1016/j.energy.2014.10.025>. <https://doi.org/10.1016/j.energy.2014.10.025>.
[ETSI_MEC_01] [ETSI_MEC_01]
ETSI, "Multi-access Edge Computing (MEC); Terminology", ETSI, "Multi-access Edge Computing (MEC); Terminology",
ETSI GS MEC 001, 2019, V2.1.1, ETSI GS MEC 001, January 2019,
<https://www.etsi.org/deliver/etsi_gs/ <https://www.etsi.org/deliver/etsi_gs/
MEC/001_099/001/02.01.01_60/gs_MEC001v020101p.pdf>. MEC/001_099/001/02.01.01_60/gs_MEC001v020101p.pdf>.
[ETSI_MEC_03] [ETSI_MEC_03]
ETSI, "Mobile Edge Computing (MEC); Framework and ETSI, "Multi-access Edge Computing (MEC); Framework and
Reference Architecture", ETSI GS MEC 003, 2019, Reference Architecture", V2.1.1, ETSI GS MEC 003, January
<https://www.etsi.org/deliver/etsi_gs/ 2019, <https://www.etsi.org/deliver/etsi_gs/
MEC/001_099/003/02.01.01_60/gs_MEC003v020101p.pdf>. MEC/001_099/003/02.01.01_60/gs_MEC003v020101p.pdf>.
[ETSI_MEC_33] [ETSI_MEC_33]
ETSI, "Multi-access Edge Computing (MEC); IoT API", ETSI ETSI, "Multi-access Edge Computing (MEC); IoT API",
GS MEC 033, 2022, <https://www.etsi.org/deliver/etsi_gs/ V3.1.1, ETSI GS MEC 033, December 2022,
<https://www.etsi.org/deliver/etsi_gs/
MEC/001_099/033/03.01.01_60/gs_MEC033v030101p.pdf>. MEC/001_099/033/03.01.01_60/gs_MEC033v030101p.pdf>.
[ETSI_Sandbox] [ETSI_Sandbox]
"Multi-access Edge Computing (MEC) MEC Sandbox Work Item", ETSI, "Multi-access Edge Computing (MEC) MEC Sandbox",
Portal, 2020, <https://portal.etsi.org/webapp/WorkProgram/ Portal, September 2023,
<https://portal.etsi.org/webapp/WorkProgram/
Report_WorkItem.asp?WKI_ID=57671>. Report_WorkItem.asp?WKI_ID=57671>.
[Fan] Fan, Q. and N. Ansari, "Cost Aware cloudlet Placement for [Fan] Fan, Q. and N. Ansari, "Cost Aware cloudlet Placement for
big data processing at the edge", 2017 IEEE International big data processing at the edge", 2017 IEEE International
Conference on Communications (ICC), Conference on Communications (ICC),
DOI 10.1109/icc.2017.7996722, May 2017, DOI 10.1109/icc.2017.7996722, May 2017,
<https://doi.org/10.1109/icc.2017.7996722>. <https://doi.org/10.1109/icc.2017.7996722>.
[Farnbauer-Schmidt] [Farnbauer-Schmidt]
Farnbauer-Schmidt, M., Lindner, J., Kaffenberger, C., and Farnbauer-Schmidt, M., Lindner, J., Kaffenberger, C., and
J. Albrecht, "Combining the Concepts of Semantic Data J. Albrecht, "Combining the Concepts of Semantic Data
Integration and Edge Computing", INFORMATIK 2019 50 Jahre Integration and Edge Computing", INFORMATIK 2019: 50 Jahre
Gesellschaft fur Informatik - Informatik fur Gesellschaft, Gesellschaft für Informatik – Informatik für Gesellschaf,
pp. 139-152, DOI 10.18420/inf2019_19, 2019, pp. 139-152, DOI 10.18420/inf2019_19, September 2019,
<https://doi.org/10.18420/inf2019_19>. <https://doi.org/10.18420/inf2019_19>.
[Grewe] Grewe, D., Wagner, M., Arumaithurai, M., Psaras, I., and [Grewe] Grewe, D., Wagner, M., Arumaithurai, M., Psaras, I., and
D. Kutscher, "Information-Centric Mobile Edge Computing D. Kutscher, "Information-Centric Mobile Edge Computing
for Connected Vehicle Environments: Challenges and for Connected Vehicle Environments: Challenges and
Research Directions", Proceedings of the Workshop on Research Directions", Proceedings of the Workshop on
Mobile Edge Communications pp. 7-12, Mobile Edge Communications, pp. 7-12,
DOI 10.1145/3098208.3098210, 2017, DOI 10.1145/3098208.3098210, August 2017,
<https://doi.org/10.1145/3098208.3098210>. <https://doi.org/10.1145/3098208.3098210>.
[Harchol] Harchol, Y., Mushtaq, A., McCauley, J., Panda, A., and S. [Harchol] Harchol, Y., Mushtaq, A., McCauley, J., Panda, A., and S.
Shenker, "CESSNA: Resilient Edge-Computing", Proceedings Shenker, "CESSNA: Resilient Edge-Computing", Proceedings
of the 2018 Workshop on Mobile Edge Communications, of the 2018 Workshop on Mobile Edge Communications,
DOI 10.1145/3229556.3229558, August 2018, DOI 10.1145/3229556.3229558, August 2018,
<https://doi.org/10.1145/3229556.3229558>. <https://doi.org/10.1145/3229556.3229558>.
[I-D.bernardos-sfc-fog-ran] [IEC_IEEE_60802]
Bernardos, C. J. and A. Mourad, "Service Function Chaining IEC/IEEE, "Use Cases IEC/IEEE 60802", V1.3, IEC/
Use Cases in Fog RAN", Work in Progress, Internet-Draft, IEEE 60802, September 2018,
draft-bernardos-sfc-fog-ran-10, 22 October 2021, <https://grouper.ieee.org/groups/802/1/files/public/
<https://datatracker.ietf.org/doc/html/draft-bernardos- docs2018/60802-industrial-use-cases-0918-v13.pdf>.
sfc-fog-ran-10>.
[I-D.defoy-t2trg-iot-edge-computing-background]
de Foy, X., Hong, J., Hong, Y., Kovatsch, M., Schooler,
E., and D. Kutscher, "IoT Edge Computing: Initiatives,
Projects and Products", Work in Progress, Internet-Draft,
draft-defoy-t2trg-iot-edge-computing-background-00, 25 May
2020, <https://datatracker.ietf.org/doc/html/draft-defoy-
t2trg-iot-edge-computing-background-00>.
[I-D.hsingh-coinrg-reqs-p4comp]
Singh, H. and M. Montpetit, "Requirements for P4 Program
Splitting for Heterogeneous Network Nodes", Work in
Progress, Internet-Draft, draft-hsingh-coinrg-reqs-p4comp-
03, 18 February 2021,
<https://datatracker.ietf.org/doc/html/draft-hsingh-
coinrg-reqs-p4comp-03>.
[I-D.ietf-core-groupcomm-bis]
Dijk, E., Wang, C., and M. Tiloca, "Group Communication
for the Constrained Application Protocol (CoAP)", Work in
Progress, Internet-Draft, draft-ietf-core-groupcomm-bis-
09, 10 July 2023, <https://datatracker.ietf.org/doc/html/
draft-ietf-core-groupcomm-bis-09>.
[I-D.irtf-nmrg-network-digital-twin-arch]
Zhou, C., Yang, H., Duan, X., Lopez, D., Pastor, A., Wu,
Q., Boucadair, M., and C. Jacquenet, "Digital Twin
Network: Concepts and Reference Architecture", Work in
Progress, Internet-Draft, draft-irtf-nmrg-network-digital-
twin-arch-03, 27 April 2023,
<https://datatracker.ietf.org/doc/html/draft-irtf-nmrg-
network-digital-twin-arch-03>.
[I-D.irtf-t2trg-rest-iot]
Keränen, A., Kovatsch, M., and K. Hartke, "Guidance on
RESTful Design for Internet of Things Systems", Work in
Progress, Internet-Draft, draft-irtf-t2trg-rest-iot-12, 25
July 2023, <https://datatracker.ietf.org/doc/html/draft-
irtf-t2trg-rest-iot-12>.
[I-D.mcbride-edge-data-discovery-overview]
McBride, M., Kutscher, D., Schooler, E., Bernardos, C. J.,
Lopez, D., and X. de Foy, "Edge Data Discovery for COIN",
Work in Progress, Internet-Draft, draft-mcbride-edge-data-
discovery-overview-05, 1 November 2020,
<https://datatracker.ietf.org/doc/html/draft-mcbride-edge-
data-discovery-overview-05>.
[I-D.sarathchandra-coin-appcentres]
Trossen, D., Sarathchandra, C., and M. Boniface, "In-
Network Computing for App-Centric Micro-Services", Work in
Progress, Internet-Draft, draft-sarathchandra-coin-
appcentres-04, 26 January 2021,
<https://datatracker.ietf.org/doc/html/draft-
sarathchandra-coin-appcentres-04>.
[ISO_TR] "Internet of things (IoT) - Edge computing", ISO/IEC TR [ISO_TR] "Internet of things (IoT) - Edge computing", ISO/IEC TR
30164, 2020, <https://www.iso.org/standard/53284.html>. 30164:2020, April 2020,
<https://www.iso.org/standard/53284.html>.
[Jeffery] Jeffery, A., Howard, H., and R. Mortier, "Rearchitecting [Jeffery] Jeffery, A., Howard, H., and R. Mortier, "Rearchitecting
Kubernetes for the Edge", Proceedings of the 4th Kubernetes for the Edge", Proceedings of the 4th
International Workshop on Edge Systems, Analytics International Workshop on Edge Systems, Analytics and
and Networking, DOI 10.1145/3434770.3459730, April 2021, Networking, DOI 10.1145/3434770.3459730, April 2021,
<https://doi.org/10.1145/3434770.3459730>. <https://doi.org/10.1145/3434770.3459730>.
[Jeong] Jeong, T., Chung, J., Hong, J., and S. Ha, "Towards a [Jeong] Jeong, T., Chung, J., Hong, J., and S. Ha, "Towards a
distributed computing framework for Fog", 2017 IEEE Fog distributed computing framework for Fog", 2017 IEEE Fog
World Congress (FWC), DOI 10.1109/fwc.2017.8368528, World Congress (FWC), DOI 10.1109/fwc.2017.8368528,
October 2017, <https://doi.org/10.1109/fwc.2017.8368528>. October 2017, <https://doi.org/10.1109/fwc.2017.8368528>.
[Jones] Jones, D., Snider, C., Nassehi, A., Yon, J., and B. Hicks, [Jones] Jones, D., Snider, C., Nassehi, A., Yon, J., and B. Hicks,
"Characterising the Digital Twin: A systematic literature "Characterising the Digital Twin: A systematic literature
review", CIRP Journal of Manufacturing Science and review", CIRP Journal of Manufacturing Science and
Technology vol. 29, pp. 36-52, Technology, vol. 29, pp. 36-52,
DOI 10.1016/j.cirpj.2020.02.002, May 2020, DOI 10.1016/j.cirpj.2020.02.002, May 2020,
<https://doi.org/10.1016/j.cirpj.2020.02.002>. <https://doi.org/10.1016/j.cirpj.2020.02.002>.
[Kelly] Kelly, R., "Internet of Things Data to Top 1.6 Zettabytes [Kelly] Kelly, R., "Internet of Things Data to Top 1.6 Zettabytes
by 2022", Retrieved on 2022-05-24, 2015, by 2020", April 2015,
<https://campustechnology.com/articles/2015/04/15/ <https://campustechnology.com/articles/2015/04/15/
internet-of-things-data-to-top-1-6-zettabytes-by- internet-of-things-data-to-top-1-6-zettabytes-by-
2020.aspx>. 2020.aspx>. Retrieved on 2022-05-24.
[Khan] Khan, L., Yaqoob, I., Tran, N., Kazmi, S., Dang, T., and [Khan] Khan, L., Yaqoob, I., Tran, N., Kazmi, S., Dang, T., and
C. Hong, "Edge-Computing-Enabled Smart Cities: A C. Hong, "Edge-Computing-Enabled Smart Cities: A
Comprehensive Survey", IEEE Internet of Things Comprehensive Survey", IEEE Internet of Things Journal,
Journal vol. 7, no. 10, pp. 10200-10232, vol. 7, no. 10, pp. 10200-10232,
DOI 10.1109/jiot.2020.2987070, October 2020, DOI 10.1109/jiot.2020.2987070, October 2020,
<https://doi.org/10.1109/jiot.2020.2987070>. <https://doi.org/10.1109/jiot.2020.2987070>.
[kua] Patil, V., Desai, H., and L. Zhang, "Kua: a distributed [Kua] Patil, V., Desai, H., and L. Zhang, "Kua: a distributed
object store over named data networking", ACM, Proceedings object store over named data networking", Proceedings of
of the 9th ACM Conference on Information- the 9th ACM Conference on Information-Centric Networking,
Centric Networking, DOI 10.1145/3517212.3558083, September DOI 10.1145/3517212.3558083, September 2022,
2022, <https://doi.org/10.1145/3517212.3558083>. <https://doi.org/10.1145/3517212.3558083>.
[Larrea] Larrea, J. and A. Barbalace, "The serverkernel operating [Larrea] Larrea, J. and A. Barbalace, "The serverkernel operating
system", Proceedings of the Third ACM International system", Proceedings of the Third ACM International
Workshop on Edge Systems, Analytics and Networking, Workshop on Edge Systems, Analytics and Networking,
DOI 10.1145/3378679.3394537, April 2020, DOI 10.1145/3378679.3394537, May 2020,
<https://doi.org/10.1145/3378679.3394537>. <https://doi.org/10.1145/3378679.3394537>.
[LFEDGE-EVE] [LFEDGE-EVE]
Linux Foundation, "Project Edge Virtualization Engine Linux Foundation, "Project Edge Virtualization Engine
(EVE)", Portal, retrieved on 2022-05-24, 2020, (EVE)", Portal, <https://www.lfedge.org/projects/eve>.
<https://www.lfedge.org/projects/eve>. Retrieved on 2022-05-24.
[Li] Li, Y., Chen, Y., Lan, T., and G. Venkataramani, "MobiQoR: [Li] Li, Y., Chen, Y., Lan, T., and G. Venkataramani, "MobiQoR:
Pushing the Envelope of Mobile Edge Computing Via Quality- Pushing the Envelope of Mobile Edge Computing Via Quality-
of-Result Optimization", 2017 IEEE 37th International of-Result Optimization", 2017 IEEE 37th International
Conference on Distributed Computing Systems (ICDCS), Conference on Distributed Computing Systems (ICDCS),
DOI 10.1109/icdcs.2017.54, June 2017, DOI 10.1109/icdcs.2017.54, June 2017,
<https://doi.org/10.1109/icdcs.2017.54>. <https://doi.org/10.1109/icdcs.2017.54>.
[Lin] Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., and W. [Lin] Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., and W.
Zhao, "A Survey on Internet of Things: Architecture, Zhao, "A Survey on Internet of Things: Architecture,
Enabling Technologies, Security and Privacy, and Enabling Technologies, Security and Privacy, and
Applications", IEEE Internet of Things Journal vol. 4, no. Applications", IEEE Internet of Things Journal, vol. 4,
5, pp. 1125-1142, DOI 10.1109/jiot.2017.2683200, October no. 5, pp. 1125-1142, DOI 10.1109/jiot.2017.2683200,
2017, <https://doi.org/10.1109/jiot.2017.2683200>. October 2017, <https://doi.org/10.1109/jiot.2017.2683200>.
[Liu] Liu, J., Bai, B., Zhang, J., and K. Letaief, "Cache [Liu] Liu, J., Bai, B., Zhang, J., and K. Letaief, "Cache
Placement in Fog-RANs: From Centralized to Distributed Placement in Fog-RANs: From Centralized to Distributed
Algorithms", IEEE Transactions on Wireless Algorithms", IEEE Transactions on Wireless Communications,
Communications vol. 16, no. 11, pp. 7039-7051, vol. 16, no. 11, pp. 7039-7051,
DOI 10.1109/twc.2017.2737015, November 2017, DOI 10.1109/twc.2017.2737015, November 2017,
<https://doi.org/10.1109/twc.2017.2737015>. <https://doi.org/10.1109/twc.2017.2737015>.
[Madni] Madni, A. M., Madni, C., and S. D. Lucero, "Leveraging [Madni] Madni, A., Madni, C., and S. Lucero, "Leveraging Digital
digital twin technology in model-based systems Twin Technology in Model-Based Systems Engineering",
engineering", Systems 7, no. 1 7, Systems 7(1):7, DOI 10.3390/systems7010007, January 2019,
DOI 10.3390/systems7010007, 2019,
<https://doi.org/10.3390/systems7010007>. <https://doi.org/10.3390/systems7010007>.
[Mahadev] Satyanarayanan, M., "The Emergence of Edge Computing", [Mahadev] Satyanarayanan, M., "The Emergence of Edge Computing",
Computer vol. 50, no. 1, pp. 30-39, DOI 10.1109/mc.2017.9, Computer, vol. 50, no. 1, pp. 30-39,
January 2017, <https://doi.org/10.1109/mc.2017.9>. DOI 10.1109/mc.2017.9, January 2017,
<https://doi.org/10.1109/mc.2017.9>.
[Mortazavi] [Mortazavi]
Hossein Mortazavi, S., Balasubramanian, B., de Lara, E., Mortazavi, S., Balasubramanian, B., de Lara, E., and S.
and S. P. Narayanan, "Toward Session Consistency for the Narayanan, "Toward Session Consistency for the Edge",
Edge", USENIX, Workshop on Hot Topics in Edge Computing USENIX Workshop on Hot Topics in Edge Computing (HotEdge
(HotEdge 18), 2018, 18), 2018,
<https://www.usenix.org/conference/hotedge18/presentation/ <https://www.usenix.org/conference/hotedge18/presentation/
mortazavi>. mortazavi>.
[mqtt5] OASIS Message Queuing Telemetry Transport (MQTT) TC, "MQTT [MQTT5] Banks, A., Ed., Briggs, E., Ed., Borgendale, K., Ed., and
Version 5.0", OASIS, March 2019, <https://docs.oasis- R. Gupta, Ed., "MQTT Version 5.0", OASIS Standard, March
open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html>. 2019, <https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-
v5.0.html>.
[Murshed] Murshed, M., Murphy, C., Hou, D., Khan, N., [Murshed] Murshed, M., Murphy, C., Hou, D., Khan, N.,
Ananthanarayanan, G., and F. Hussain, "Machine Learning at Ananthanarayanan, G., and F. Hussain, "Machine Learning at
the Network Edge: A Survey", ACM Computing Surveys vol. the Network Edge: A Survey", ACM Computing Surveys, vol.
54, no. 8, pp. 1-37, DOI 10.1145/3469029, November 2022, 54, no. 8, pp. 1-37, DOI 10.1145/3469029, October 2021,
<https://doi.org/10.1145/3469029>. <https://doi.org/10.1145/3469029>.
[NETWORK-DIGITAL-TWIN-ARCH]
Zhou, C., Yang, H., Duan, X., Lopez, D., Pastor, A., Wu,
Q., Boucadair, M., and C. Jacquenet, "Network Digital
Twin: Concepts and Reference Architecture", Work in
Progress, Internet-Draft, draft-irtf-nmrg-network-digital-
twin-arch-05, 4 March 2024,
<https://datatracker.ietf.org/doc/html/draft-irtf-nmrg-
network-digital-twin-arch-05>.
[Nieke] Nieke, M., Almstedt, L., and R. Kapitza, "Edgedancer: [Nieke] Nieke, M., Almstedt, L., and R. Kapitza, "Edgedancer:
Secure Mobile WebAssembly Services on the Edge", Secure Mobile WebAssembly Services on the Edge",
Proceedings of the 4th International Workshop on Edge Proceedings of the 4th International Workshop on Edge
Systems, Analytics and Networking, Systems, Analytics and Networking,
DOI 10.1145/3434770.3459731, April 2021, DOI 10.1145/3434770.3459731, April 2021,
<https://doi.org/10.1145/3434770.3459731>. <https://doi.org/10.1145/3434770.3459731>.
[NIST] Mell, P. and T. Grance, "The NIST definition of cloud [NIST] Mell, P. and T. Grance, "The NIST Definition of Cloud
computing", National Institute of Standards and Computing", NIST Special Publication 800-145,
Technology report, DOI 10.6028/nist.sp.800-145, 2011, DOI 10.6028/nist.sp.800-145, September 2011,
<https://doi.org/10.6028/nist.sp.800-145>. <https://doi.org/10.6028/nist.sp.800-145>.
[NVIDIA] Grzywaczewski, A., "Training AI for Self-Driving Vehicles: [NVIDIA] Grzywaczewski, A., "Training AI for Self-Driving Vehicles:
the Challenge of Scale", NVIDIA Developer Blog, retrieved the Challenge of Scale", NVIDIA Developer Blog, October
on 2022-05-24, 2017, <https://devblogs.nvidia.com/ 2017, <https://devblogs.nvidia.com/training-self-driving-
training-self-driving-vehicles-challenge-scale/>. vehicles-challenge-scale/>. Retrieved on 2022-05-24.
[oneM2M-TR0001] [oneM2M-TR0001]
Mladin, C., "TR 0001, Use Cases Collection", oneM2M, Mladin, C., "Use Cases Collection", oneM2M, v4.2.0,
October 2018, TR 0001, October 2018,
<https://member.onem2m.org/Application/documentapp/ <https://member.onem2m.org/Application/documentapp/
downloadLatestRevision/default.aspx?docID=28153>. downloadLatestRevision/default.aspx?docID=28153>.
[oneM2M-TR0018] [oneM2M-TR0018]
Lu, C. and M. Jiang, "TR 0018, Industrial Domain Lu, C. and M. Jiang, "Industrial Domain Enablement",
Enablement", oneM2M, February 2019, oneM2M, v2.5.2, TR 0018, February 2019,
<https://member.onem2m.org/Application/documentapp/ <https://member.onem2m.org/Application/documentapp/
downloadLatestRevision/default.aspx?docID=29334>. downloadLatestRevision/default.aspx?docID=29334>.
[oneM2M-TR0026] [oneM2M-TR0026]
Yamamoto, K., Mladin, C., and V. Kueh, "TR 0026, Vehicular Yamamoto, K., Mladin, C., and V. Kueh, "Vehicular Domain
Domain Enablement", oneM2M, January 2020, Enablement", oneM2M, TR 0026, January 2020,
<https://member.onem2m.org/Application/documentapp/ <https://member.onem2m.org/Application/documentapp/
downloadLatestRevision/default.aspx?docID=31410>. downloadLatestRevision/default.aspx?docID=31410>.
[oneM2M-TR0052] [oneM2M-TR0052]
Yamamoto, K. and C. Mladin, "TR 0052, Study on Edge and Yamamoto, K. and C. Mladin, "Study on Edge and Fog
Fog Computing in oneM2M systems", oneM2M, September 2020, Computing in oneM2M systems", oneM2M, TR 0052, September
<https://member.onem2m.org/Application/documentapp/ 2020, <https://member.onem2m.org/Application/documentapp/
downloadLatestRevision/default.aspx?docID=32633>. downloadLatestRevision/default.aspx?docID=32633>.
[oneM2M-TS0002] [oneM2M-TS0002]
He, S., "TS 0002, Requirements", oneM2M, February 2019, He, S., "TS 0002, Requirements", oneM2M, TS 0002, February
<https://member.onem2m.org/Application/documentapp/ 2019, <https://member.onem2m.org/Application/documentapp/
downloadLatestRevision/default.aspx?docID=29274>. downloadLatestRevision/default.aspx?docID=29274>.
[OpenFog] "OpenFog Reference Architecture for Fog Computing", [OpenFog] OpenFog Consortium, "OpenFog Reference Architecture for
OpenFog Consortium, 2017, <https://iiconsortium.org/pdf/ Fog Computing", February 2017,
<https://iiconsortium.org/pdf/
OpenFog_Reference_Architecture_2_09_17.pdf>. OpenFog_Reference_Architecture_2_09_17.pdf>.
[PseudoDynamicTesting] [PseudoDynamicTesting]
Ficco, M., Esposito, C., Xiang, Y., and F. Palmieri, Ficco, M., Esposito, C., Xiang, Y., and F. Palmieri,
"Pseudo-Dynamic Testing of Realistic Edge-Fog Cloud "Pseudo-Dynamic Testing of Realistic Edge-Fog Cloud
Ecosystems", IEEE Communications Magazine vol. 55, no. 11, Ecosystems", IEEE Communications Magazine, vol. 55, no.
pp. 98-104, DOI 10.1109/mcom.2017.1700328, November 2017, 11, pp. 98-104, DOI 10.1109/mcom.2017.1700328, November
<https://doi.org/10.1109/mcom.2017.1700328>. 2017, <https://doi.org/10.1109/mcom.2017.1700328>.
[Renart] Renart, E., Diaz-Montes, J., and M. Parashar, "Data-Driven [Renart] Renart, E., Diaz-Montes, J., and M. Parashar, "Data-Driven
Stream Processing at the Edge", 2017 IEEE 1st Stream Processing at the Edge", 2017 IEEE 1st
International Conference on Fog and Edge International Conference on Fog and Edge Computing
Computing (ICFEC), DOI 10.1109/icfec.2017.18, May 2017, (ICFEC), DOI 10.1109/icfec.2017.18, May 2017,
<https://doi.org/10.1109/icfec.2017.18>. <https://doi.org/10.1109/icfec.2017.18>.
[REQS-P4COMP]
Singh, H. and M. Montpetit, "Requirements for P4 Program
Splitting for Heterogeneous Network Nodes", Work in
Progress, Internet-Draft, draft-hsingh-coinrg-reqs-p4comp-
03, 18 February 2021,
<https://datatracker.ietf.org/doc/html/draft-hsingh-
coinrg-reqs-p4comp-03>.
[REST-IOT] Keränen, A., Kovatsch, M., and K. Hartke, "Guidance on
RESTful Design for Internet of Things Systems", Work in
Progress, Internet-Draft, draft-irtf-t2trg-rest-iot-13, 25
January 2024, <https://datatracker.ietf.org/doc/html/
draft-irtf-t2trg-rest-iot-13>.
[RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu,
D., and S. Mansfield, "Guidelines for the Use of the "OAM" D., and S. Mansfield, "Guidelines for the Use of the "OAM"
Acronym in the IETF", BCP 161, RFC 6291, Acronym in the IETF", BCP 161, RFC 6291,
DOI 10.17487/RFC6291, June 2011, DOI 10.17487/RFC6291, June 2011,
<https://www.rfc-editor.org/rfc/rfc6291>. <https://www.rfc-editor.org/info/rfc6291>.
[RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained
Application Protocol (CoAP)", RFC 7252, Application Protocol (CoAP)", RFC 7252,
DOI 10.17487/RFC7252, June 2014, DOI 10.17487/RFC7252, June 2014,
<https://www.rfc-editor.org/rfc/rfc7252>. <https://www.rfc-editor.org/info/rfc7252>.
[RFC7390] Rahman, A., Ed. and E. Dijk, Ed., "Group Communication for [RFC7390] Rahman, A., Ed. and E. Dijk, Ed., "Group Communication for
the Constrained Application Protocol (CoAP)", RFC 7390, the Constrained Application Protocol (CoAP)", RFC 7390,
DOI 10.17487/RFC7390, October 2014, DOI 10.17487/RFC7390, October 2014,
<https://www.rfc-editor.org/rfc/rfc7390>. <https://www.rfc-editor.org/info/rfc7390>.
[RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases",
RFC 8578, DOI 10.17487/RFC8578, May 2019, RFC 8578, DOI 10.17487/RFC8578, May 2019,
<https://www.rfc-editor.org/rfc/rfc8578>. <https://www.rfc-editor.org/info/rfc8578>.
[Schafer] Schafer, D., Edinger, J., VanSyckel, S., Paluska, J., and [Schafer] Schäfer, D., Edinger, J., VanSyckel, S., Paluska, J., and
C. Becker, "Tasklets: Overcoming Heterogeneity in C. Becker, "Tasklets: Overcoming Heterogeneity in
Distributed Computing Systems", 2016 IEEE 36th Distributed Computing Systems", 2016 IEEE 36th
International Conference on Distributed Computing Systems International Conference on Distributed Computing Systems
Workshops (ICDCSW), DOI 10.1109/icdcsw.2016.22, June 2016, Workshops (ICDCSW), DOI 10.1109/icdcsw.2016.22, June 2016,
<https://doi.org/10.1109/icdcsw.2016.22>. <https://doi.org/10.1109/icdcsw.2016.22>.
[Senel] Senel, B., Mouchet, M., Cappos, J., Fourmaux, O., [Senel] Şenel, B., Mouchet, M., Cappos, J., Fourmaux, O.,
Friedman, T., and R. McGeer, "EdgeNet: A Multi-Tenant and Friedman, T., and R. McGeer, "EdgeNet: A Multi-Tenant and
Multi-Provider Edge Cloud", Proceedings of the 4th Multi-Provider Edge Cloud", Proceedings of the 4th
International Workshop on Edge Systems, Analytics International Workshop on Edge Systems, Analytics and
and Networking, DOI 10.1145/3434770.3459737, April 2021, Networking, DOI 10.1145/3434770.3459737, April 2021,
<https://doi.org/10.1145/3434770.3459737>. <https://doi.org/10.1145/3434770.3459737>.
[SFC-FOG-RAN]
Bernardos, C. J. and A. Mourad, "Service Function Chaining
Use Cases in Fog RAN", Work in Progress, Internet-Draft,
draft-bernardos-sfc-fog-ran-10, 22 October 2021,
<https://datatracker.ietf.org/doc/html/draft-bernardos-
sfc-fog-ran-10>.
[Shi] Shi, W., Cao, J., Zhang, Q., Li, Y., and L. Xu, "Edge [Shi] Shi, W., Cao, J., Zhang, Q., Li, Y., and L. Xu, "Edge
Computing: Vision and Challenges", IEEE Internet of Things Computing: Vision and Challenges", IEEE Internet of Things
Journal vol. 3, no. 5, pp. 637-646, Journal, vol. 3, no. 5, pp. 637-646,
DOI 10.1109/jiot.2016.2579198, October 2016, DOI 10.1109/jiot.2016.2579198, October 2016,
<https://doi.org/10.1109/jiot.2016.2579198>. <https://doi.org/10.1109/jiot.2016.2579198>.
[SimulatingFog] [SimulatingFog]
Svorobej, S., Takako Endo, P., Bendechache, M., Filelis- Svorobej, S., Takako Endo, P., Bendechache, M., Filelis-
Papadopoulos, C., Giannoutakis, K., Gravvanis, G., Papadopoulos, C., Giannoutakis, K., Gravvanis, G.,
Tzovaras, D., Byrne, J., and T. Lynn, "Simulating Fog and Tzovaras, D., Byrne, J., and T. Lynn, "Simulating Fog and
Edge Computing Scenarios: An Overview and Research Edge Computing Scenarios: An Overview and Research
Challenges", Future Internet vol. 11, no. 3, pp. 55, Challenges", Future Internet, vol. 11, no. 3, pp. 55,
DOI 10.3390/fi11030055, February 2019, DOI 10.3390/fi11030055, February 2019,
<https://doi.org/10.3390/fi11030055>. <https://doi.org/10.3390/fi11030055>.
[Stanciu] Stanciu, V., Steen, M., Dobre, C., and A. Peter, "Privacy- [Stanciu] Stanciu, V., Steen, M., Dobre, C., and A. Peter, "Privacy-
Preserving Crowd-Monitoring Using Bloom Filters and Preserving Crowd-Monitoring Using Bloom Filters and
Homomorphic Encryption", Proceedings of the 4th Homomorphic Encryption", Proceedings of the 4th
International Workshop on Edge Systems, Analytics International Workshop on Edge Systems, Analytics and
and Networking, DOI 10.1145/3434770.3459735, April 2021, Networking, DOI 10.1145/3434770.3459735, April 2021,
<https://doi.org/10.1145/3434770.3459735>. <https://doi.org/10.1145/3434770.3459735>.
[Weiner] Weiner, M., Jorgovanovic, M., Sahai, A., and B. Nikolie, [Weiner] Weiner, M., Jorgovanovic, M., Sahai, A., and B. Nikolie,
"Design of a low-latency, high-reliability wireless "Design of a low-latency, high-reliability wireless
communication system for control applications", 2014 IEEE communication system for control applications", 2014 IEEE
International Conference on Communications (ICC), International Conference on Communications (ICC),
DOI 10.1109/icc.2014.6883918, June 2014, DOI 10.1109/icc.2014.6883918, June 2014,
<https://doi.org/10.1109/icc.2014.6883918>. <https://doi.org/10.1109/icc.2014.6883918>.
[Yangui] Yangui, S., Ravindran, P., Bibani, O., Glitho, R., Ben [Yangui] Yangui, S., Ravindran, P., Bibani, O., Glitho, R., Ben
Hadj-Alouane, N., Morrow, M., and P. Polakos, "A platform Hadj-Alouane, N., Morrow, M., and P. Polakos, "A platform
as-a-service for hybrid cloud/fog environments", 2016 IEEE as-a-service for hybrid cloud/fog environments", 2016 IEEE
International Symposium on Local and Metropolitan Area International Symposium on Local and Metropolitan Area
Networks (LANMAN), DOI 10.1109/lanman.2016.7548853, June Networks (LANMAN), DOI 10.1109/lanman.2016.7548853, June
2016, <https://doi.org/10.1109/lanman.2016.7548853>. 2016, <https://doi.org/10.1109/lanman.2016.7548853>.
[Yates] Yates, R. and S. Kaul, "The Age of Information: Real-Time [Yates] Yates, R. and S. Kaul, "The Age of Information: Real-Time
Status Updating by Multiple Sources", IEEE Transactions on Status Updating by Multiple Sources", IEEE Transactions on
Information Theory vol. 65, no. 3, pp. 1807-1827, Information Theory, vol. 65, no. 3, pp. 1807-1827,
DOI 10.1109/tit.2018.2871079, March 2019, DOI 10.1109/tit.2018.2871079, March 2019,
<https://doi.org/10.1109/tit.2018.2871079>. <https://doi.org/10.1109/tit.2018.2871079>.
[Yousefpour] [Yousefpour]
Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K.,
Jalali, F., Niakanlahiji, A., Kong, J., and J. Jue, "All Jalali, F., Niakanlahiji, A., Kong, J., and J. Jue, "All
one needs to know about fog computing and related edge one needs to know about fog computing and related edge
computing paradigms: A complete survey", Journal of computing paradigms: A complete survey", Journal of
Systems Architecture vol. 98, pp. 289-330, Systems Architecture, vol. 98, pp. 289-330,
DOI 10.1016/j.sysarc.2019.02.009, September 2019, DOI 10.1016/j.sysarc.2019.02.009, September 2019,
<https://doi.org/10.1016/j.sysarc.2019.02.009>. <https://doi.org/10.1016/j.sysarc.2019.02.009>.
[Zhang] Zhang, Q., Zhang, X., Zhang, Q., Shi, W., and H. Zhong, [Zhang] Zhang, Q., Zhang, X., Zhang, Q., Shi, W., and H. Zhong,
"Firework: Big Data Sharing and Processing in "Firework: Big Data Sharing and Processing in
Collaborative Edge Environment", 2016 Fourth IEEE Workshop Collaborative Edge Environment", 2016 Fourth IEEE Workshop
on Hot Topics in Web Systems and Technologies (HotWeb), on Hot Topics in Web Systems and Technologies (HotWeb),
DOI 10.1109/hotweb.2016.12, October 2016, DOI 10.1109/hotweb.2016.12, October 2016,
<https://doi.org/10.1109/hotweb.2016.12>. <https://doi.org/10.1109/hotweb.2016.12>.
[Zhang2] Zhang, J., Chen, B., Zhao, Y., Cheng, X., and F. Hu, "Data [Zhang2] Zhang, J., Chen, B., Zhao, Y., Cheng, X., and F. Hu, "Data
Security and Privacy-Preserving in Edge Computing Security and Privacy-Preserving in Edge Computing
Paradigm: Survey and Open Issues", IEEE Access vol. 6, pp. Paradigm: Survey and Open Issues", IEEE Access, vol. 6,
18209-18237, DOI 10.1109/access.2018.2820162, 2018, pp. 18209-18237, DOI 10.1109/access.2018.2820162, March
<https://doi.org/10.1109/access.2018.2820162>. 2018, <https://doi.org/10.1109/access.2018.2820162>.
[_60802] IEC/IEEE, "Use Cases IEC/IEEE 60802 V1.3", IEC/IEEE 60802, Acknowledgements
2018, <https://grouper.ieee.org/groups/802/1/files/public/
docs2018/60802-industrial-use-cases-0918-v13.pdf>. The authors would like to thank Joo-Sang Youn, Akbar Rahman, Michel
Roy, Robert Gazda, Rute Sofia, Thomas Fossati, Chonggang Wang, Marie-
José Montpetit, Carlos J. Bernardos, Milan Milenkovic, Dale Seed,
JaeSeung Song, Roberto Morabito, Carsten Bormann, and Ari Keränen for
their valuable comments and suggestions on this document.
Authors' Addresses Authors' Addresses
Jungha Hong Jungha Hong
ETRI ETRI
218 Gajeong-ro, Yuseung-Gu 218 Gajeong-ro, Yuseung-Gu
Daejeon Daejeon
34129 34129
Republic of Korea Republic of Korea
Email: jhong@etri.re.kr Email: jhong@etri.re.kr
skipping to change at page 37, line 4 skipping to change at line 1678
Montreal H3A 3G4 Montreal H3A 3G4
Canada Canada
Email: xavier.defoy@interdigital.com Email: xavier.defoy@interdigital.com
Matthias Kovatsch Matthias Kovatsch
Huawei Technologies Duesseldorf GmbH Huawei Technologies Duesseldorf GmbH
Riesstr. 25 C // 3.OG Riesstr. 25 C // 3.OG
80992 Munich 80992 Munich
Germany Germany
Email: ietf@kovatsch.net Email: ietf@kovatsch.net
Eve Schooler Eve Schooler
Intel University of Oxford
2200 Mission College Blvd. Parks Road
Santa Clara, CA, 95054-1537 Oxford
United States of America OX1 3PJ
United Kingdom
Email: eve.schooler@gmail.com Email: eve.schooler@gmail.com
Dirk Kutscher Dirk Kutscher
Hong Kong University of Science and Technology (Guangzhou) Hong Kong University of Science and Technology (Guangzhou)
No.1 Du Xue Rd No.1 Du Xue Rd
Guangzhou Guangzhou
China China
Email: ietf@dkutscher.net Email: ietf@dkutscher.net
 End of changes. 221 change blocks. 
731 lines changed or deleted 749 lines changed or added

This html diff was produced by rfcdiff 1.48.