# drafts

name: draft-intellinode-cats-in-network-scheduling
rev: 00
title: IntelliNode: In-Network Intelligent Scheduling Extensions for CATS
time: 2026-03-02T08:59:10Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T08:59:10Z
states: ["/api/v1/doc/state/4/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.291561+00:00
source: ietf
abstract:
  This document introduces IntelliNode, an in-network intelligent
     scheduling mechanism built upon the Computing-Aware Traffic Steering
     (CATS) framework.  Modern large-scale AI training and inference
     heavily rely on distributed heterogeneous clusters (GPU/CPU/FPGA).
     However, existing networks lack awareness of tensor semantics,
     training phases, and heterogeneous computing capabilities, leading to
     high communication latency, low resource utilization, and pipeline
     stalls.

     IntelliNode shifts away from the traditional passive scheduling
     paradigms that rely on probes and controllers.  By bypassing
     traditional paths and integrating FPGAs alongside programmable Switch
     ASICs, it constructs a rapid data-plane closed loop of "Perception-
     Inference-Decision-Execution".  This architecture performs feature
     extraction at line rate, leverages lightweight prediction models to
     infer short-term network behavior, and drives real-time heuristic
     scheduling decisions (e.g., path selection, tensor slicing, and
     compute matching).  This document defines the four core functional
     layers and extension signaling that support this architecture, laying
     the foundation for an AI-native, scalable distributed computing
     network.

full_text:
  Computing-Aware Traffic Steering                                  T. Gao
  Internet-Draft                                     Peng Cheng Laboratory
  Intended status: Informational                              1 March 2026
  Expires: 2 September 2026


     IntelliNode: In-Network Intelligent Scheduling Extensions for CATS
              draft-intellinode-cats-in-network-scheduling-00

  Abstract

     This document introduces IntelliNode, an in-network intelligent
     scheduling mechanism built upon the Computing-Aware Traffic Steering
     (CATS) framework.  Modern large-scale AI training and inference
     heavily rely on distributed heterogeneous clusters (GPU/CPU/FPGA).
     However, existing networks lack awareness of tensor semantics,
     training phases, and heterogeneous computing capabilities, leading to
     high communication latency, low resource utilization, and pipeline
     stalls.

     IntelliNode shifts away from the traditional passive scheduling
     paradigms that rely on probes and controllers.  By bypassing
     traditional paths and integrating FPGAs alongside programmable Switch
     ASICs, it constructs a rapid data-plane closed loop of "Perception-
     Inference-Decision-Execution".  This architecture performs feature
     extraction at line rate, leverages lightweight prediction models to
     infer short-term network behavior, and drives real-time heuristic
     scheduling decisions (e.g., path selection, tensor slicing, and
     compute matching).  This document defines the four core functional
     layers and extension signaling that support this architecture, laying
     the foundation for an AI-native, scalable distributed computing
     network.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."




  Gao                     Expires 2 September 2026                [Page 1]


  Internet-Draft                 IntelliNode                    March 2026


     This Internet-Draft will expire on 2 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     2.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   3
     3.  Architecture  . . . . . . . . . . . . . . . . . . . . . . . .   3
       3.1.  Feature Extraction Layer (Switch ASIC)  . . . . . . . . .   3
       3.2.  State Prediction Layer (FPGA) . . . . . . . . . . . . . .   4
       3.3.  Heuristic Scheduling Layer  . . . . . . . . . . . . . . .   4
       3.4.  Steering Plane  . . . . . . . . . . . . . . . . . . . . .   5
     4.  Security Considerations . . . . . . . . . . . . . . . . . . .   5
     5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   6
     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .   6

  1.  Introduction

     The CATS framework primarily addresses the selection of service
     instances and computing-aware traffic steering in general distributed
     systems.  However, when confronting large-scale AI training,
     distributed inference, and heterogeneous computing clusters, AI
     workloads exhibit traffic dynamics on the order of microseconds to
     milliseconds, accompanied by highly diverse tensor types (e.g.,
     gradients, activations, parameters).

     Traditional CATS models (control-plane decisions combined with
     service-level steering) are inadequate for these next-generation
     computing workloads.  IntelliNode proposes an extended architecture
     deeply embedded in the data plane.  It not only natively processes
     RoCEv2 protocol semantics but also transforms the network from a
     "passive data pipe" into an "active, computing-aware collaborative
     engine".





  Gao                     Expires 2 September 2026                [Page 2]


  Internet-Draft                 IntelliNode                    March 2026


  2.  Problem Statement

     Applying existing network scheduling mechanisms to AI training and
     heterogeneous computing networks reveals the following fundamental
     limitations:

     *  Tensor Semantic Blind Spot: Existing mechanisms cannot distinguish
        specific semantics within data streams, such as gradients,
        activations, or parameter updates.

     *  Lag in End-to-End Feedback: Mechanisms like ECN misinterpret
        "passive feedback" as "active scheduling" and assume that "rate
        reduction is the correct response."  This completely ignores the
        computing semantics in AI inference, where certain flows "cannot
        slow down, but must wait or degrade precision."

     *  Excessive Control-Plane Decision Latency: Control-plane routing
        updates, which take hundreds of milliseconds to seconds, cannot
        handle transient congestion or iterative bursts within a 1-5
        millisecond window.

     *  Conflict between Isomorphic Assumptions and Heterogeneous Reality:
        In cross-domain computing networks, node capabilities are highly
        uneven (e.g., GPU/FPGA hybrids).  The network must possess global
        state awareness to accurately match computing power with
        communication workloads rather than relying on isomorphic
        computing assumptions.

  3.  Architecture

     The IntelliNode architecture consists of four tightly coordinated
     functional layers that perfectly align with CATS's abstractions for
     information collection, decision engine, and steering plane.  This
     architecture fuses the capabilities of programmable switches, FPGAs,
     and CPUs at the local node, enabling a microsecond-level closed loop
     without interrupting the packet forwarding path.

  3.1.  Feature Extraction Layer (Switch ASIC)

     Deployed on Tofino-class programmable switch ASICs, this layer
     actively participates in RoCEv2 traffic management.  It maintains a
     high-performance Queue Pair (QP) flow state machine.  The switch
     collects and parses real-time features at line-rate, including:

     *  Basic Network Features: Ingress port, transmission rate, flow
        size, queue depth, and link utilization.





  Gao                     Expires 2 September 2026                [Page 3]


  Internet-Draft                 IntelliNode                    March 2026


     *  AI Semantic Features: Tensor type (gradient / activation / normal
        traffic), tensor position within a batch/iteration, the stage of
        the model-parallel pipeline, and whether it is cross-node
        gradient-sync traffic.

     *  Flow State Classification: The hardware identifies the flow's
        current state as UNALLOCATED, SMALL_FLOW (delay-sensitive/
        control), LARGE_FLOW (high-bandwidth parameter synchronization),
        or DRAINING (tail-end flushing).

     These features are extracted, normalized, and encoded at line-rate,
     then written into a high-speed featureFIFO to be sent directly to the
     onboard FPGA.  Simultaneously, the pipeline incorporates real-time
     checksum updates and validation logic for mutable fields in RoCEv2
     (such as ECN and TTL markings) to ensure protocol legitimacy.

  3.2.  State Prediction Layer (FPGA)

     The FPGA reads features from the featureFIFO and executes an ultra-
     low latency, lightweight prediction model (e.g., State-GNN based on
     Graph Neural Networks, linear regression, or heuristic models).  This
     layer focuses on predicting short-term network and load states within
     the next 1-5 milliseconds (ms):

     *  Network State Prediction: Imminent congestion risks on switch
        ports and the available bandwidth of candidate routing paths in
        the next window.

     *  Computing Load Prediction: The arrival time of the next batch of
        periodic tensor traffic, and the probability of queuing backlogs
        or pipeline stalls at the downstream GPU.

     These forward-looking prediction fields serve as the core input for
     the subsequent scheduling engine.

  3.3.  Heuristic Scheduling Layer

     The scheduling engine integrates the currently extracted AI semantics
     with the predicted states output by the FPGA, approximating Pareto
     Optimality amidst conflicting multi-objective goals (e.g., computing
     latency vs. communication overhead).  The decision logic is based on:

     *  Tensor type and structural priority.

     *  Operator dependency.

     *  Heterogeneous computing capabilities of target nodes.




  Gao                     Expires 2 September 2026                [Page 4]


  Internet-Draft                 IntelliNode                    March 2026


     *  1-5ms network and congestion predictions.

     The decision outputs (execution actions) include:

     *  Path and Priority: Outputs the optimal path set.  SMALL_FLOWs are
        prioritized based on arrival rate, while LARGE_FLOWs dynamically
        allocate bandwidth based on target computing power using a
        Weighted Deficit Round Robin (WDRR) policy.

     *  Tensor Slicing: Determines if tensor slicing is necessary,
        defining the number of slices and the independent routing path for
        each.

     *  Multipath Aggregation: Decides whether to enable data-plane
        multipath aggregation.

     *  In-Network Offloading: Decides whether to offload specific
        operators (e.g., Sum/Reduce) to in-network FPGAs or edge nodes.

  3.4.  Steering Plane

     The output of the heuristic scheduling must be applied to the entire
     network data plane via a lightweight signaling mechanism (potentially
     as an extension to CATS-SR or CATS-Overlay):

     *  Control Plane Interface: Triggers the local CPU to update the
        routing/forwarding tables, applying the latest policies to the
        next batch of traffic automatically.

     *  Data Plane Labels / TLVs: Pushes extended Metadata TLVs carrying
        tensor types, training phases, and compute resource requests into
        the packet header.

     *  Fragment Routing Encapsulation: Provides necessary Encapsulation
        information for traffic requiring Tensor Slicing.

  4.  Security Considerations

     Given that IntelliNode introduces granular TLV fields for tensor
     semantics and active data-plane scheduling, the system MUST:

     *  Provide integrity protection for TLV fields to prevent malicious
        nodes from tampering with "Tensor Types" to preempt high-priority
        queues.

     *  Introduce encrypted control-plane channels for telemetry and
        configuration.




  Gao                     Expires 2 September 2026                [Page 5]


  Internet-Draft                 IntelliNode                    March 2026


     *  Implement authentication to prevent unauthorized nodes from
        falsely broadcasting their Compute-Capability within the computing
        network.

  5.  IANA Considerations

     This document requests that IANA allocate new TLV types for AI-native
     CATS deployments, including but not limited to:

     *  TENSOR_TYPE TLV

     *  TRAINING_PHASE TLV

     *  COMPUTE_CAPABILITY TLV

     *  PATH_PREDICTION TLV

  Author's Address

     Teng Gao
     Peng Cheng Laboratory
     Email: gaot@pcl.ac.cn





























  Gao                     Expires 2 September 2026                [Page 6]


name: draft-li-cats-intellinode-network-scheduling
rev: 00
title: IntelliNode: In-Network Intelligent Scheduling Extensions for CATS
time: 2026-03-02T10:02:00Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T10:02:00Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.295316+00:00
source: ietf
abstract:
  This document introduces IntelliNode, an in-network intelligent
     scheduling mechanism built upon the Computing-Aware Traffic Steering
     (CATS) framework.  Modern large-scale AI training and inference
     heavily rely on distributed heterogeneous clusters (GPU/CPU/FPGA).
     However, existing networks lack awareness of tensor semantics,
     training phases, and heterogeneous computing capabilities, leading to
     high communication latency, low resource utilization, and pipeline
     stalls.

     IntelliNode shifts away from the traditional passive scheduling
     paradigms that rely on probes and controllers.  By bypassing
     traditional paths and integrating FPGAs alongside programmable Switch
     ASICs, it constructs a rapid data-plane closed loop of "Perception-
     Inference-Decision-Execution".  This architecture performs feature
     extraction at line rate, leverages lightweight prediction models to
     infer short-term network behavior, and drives real-time heuristic
     scheduling decisions (e.g., path selection, tensor slicing, and
     compute matching).  This document defines the four core functional
     layers and extension signaling that support this architecture, laying
     the foundation for an AI-native, scalable distributed computing
     network.

full_text:
  Computing-Aware Traffic Steering                                   Q. Li
  Internet-Draft                                                    T. gao
  Intended status: Informational                      Pengcheng Laboratory
  Expires: 2 September 2026                                       Y. Jiang
    Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
                                                              1 March 2026


     IntelliNode: In-Network Intelligent Scheduling Extensions for CATS
              draft-li-cats-intellinode-network-scheduling-00

  Abstract

     This document introduces IntelliNode, an in-network intelligent
     scheduling mechanism built upon the Computing-Aware Traffic Steering
     (CATS) framework.  Modern large-scale AI training and inference
     heavily rely on distributed heterogeneous clusters (GPU/CPU/FPGA).
     However, existing networks lack awareness of tensor semantics,
     training phases, and heterogeneous computing capabilities, leading to
     high communication latency, low resource utilization, and pipeline
     stalls.

     IntelliNode shifts away from the traditional passive scheduling
     paradigms that rely on probes and controllers.  By bypassing
     traditional paths and integrating FPGAs alongside programmable Switch
     ASICs, it constructs a rapid data-plane closed loop of "Perception-
     Inference-Decision-Execution".  This architecture performs feature
     extraction at line rate, leverages lightweight prediction models to
     infer short-term network behavior, and drives real-time heuristic
     scheduling decisions (e.g., path selection, tensor slicing, and
     compute matching).  This document defines the four core functional
     layers and extension signaling that support this architecture, laying
     the foundation for an AI-native, scalable distributed computing
     network.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.







  Li, et al.              Expires 2 September 2026                [Page 1]


  Internet-Draft                 IntelliNode                    March 2026


     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 2 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     2.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   3
     3.  Architecture  . . . . . . . . . . . . . . . . . . . . . . . .   3
       3.1.  Feature Extraction Layer (Switch ASIC)  . . . . . . . . .   3
       3.2.  State Prediction Layer (FPGA) . . . . . . . . . . . . . .   4
       3.3.  Heuristic Scheduling Layer  . . . . . . . . . . . . . . .   4
       3.4.  Steering Plane  . . . . . . . . . . . . . . . . . . . . .   5
     4.  Security Considerations . . . . . . . . . . . . . . . . . . .   5
     5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   6
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   6

  1.  Introduction

     The CATS framework primarily addresses the selection of service
     instances and computing-aware traffic steering in general distributed
     systems.  However, when confronting large-scale AI training,
     distributed inference, and heterogeneous computing clusters, AI
     workloads exhibit traffic dynamics on the order of microseconds to
     milliseconds, accompanied by highly diverse tensor types (e.g.,
     gradients, activations, parameters).

     Traditional CATS models (control-plane decisions combined with
     service-level steering) are inadequate for these next-generation
     computing workloads.  IntelliNode proposes an extended architecture
     deeply embedded in the data plane.  It not only natively processes



  Li, et al.              Expires 2 September 2026                [Page 2]


  Internet-Draft                 IntelliNode                    March 2026


     RoCEv2 protocol semantics but also transforms the network from a
     "passive data pipe" into an "active, computing-aware collaborative
     engine".

  2.  Problem Statement

     Applying existing network scheduling mechanisms to AI training and
     heterogeneous computing networks reveals the following fundamental
     limitations:

     *  Tensor Semantic Blind Spot: Existing mechanisms cannot distinguish
        specific semantics within data streams, such as gradients,
        activations, or parameter updates.

     *  Lag in End-to-End Feedback: Mechanisms like ECN misinterpret
        "passive feedback" as "active scheduling" and assume that "rate
        reduction is the correct response."  This completely ignores the
        computing semantics in AI inference, where certain flows "cannot
        slow down, but must wait or degrade precision."

     *  Excessive Control-Plane Decision Latency: Control-plane routing
        updates, which take hundreds of milliseconds to seconds, cannot
        handle transient congestion or iterative bursts within a 1-5
        millisecond window.

     *  Conflict between Isomorphic Assumptions and Heterogeneous Reality:
        In cross-domain computing networks, node capabilities are highly
        uneven (e.g., GPU/FPGA hybrids).  The network must possess global
        state awareness to accurately match computing power with
        communication workloads rather than relying on isomorphic
        computing assumptions.

  3.  Architecture

     The IntelliNode architecture consists of four tightly coordinated
     functional layers that perfectly align with CATS's abstractions for
     information collection, decision engine, and steering plane.  This
     architecture fuses the capabilities of programmable switches, FPGAs,
     and CPUs at the local node, enabling a microsecond-level closed loop
     without interrupting the packet forwarding path.

  3.1.  Feature Extraction Layer (Switch ASIC)

     Deployed on Tofino-class programmable switch ASICs, this layer
     actively participates in RoCEv2 traffic management.  It maintains a
     high-performance Queue Pair (QP) flow state machine.  The switch
     collects and parses real-time features at line-rate, including:




  Li, et al.              Expires 2 September 2026                [Page 3]


  Internet-Draft                 IntelliNode                    March 2026


     *  Basic Network Features: Ingress port, transmission rate, flow
        size, queue depth, and link utilization.

     *  AI Semantic Features: Tensor type (gradient / activation / normal
        traffic), tensor position within a batch/iteration, the stage of
        the model-parallel pipeline, and whether it is cross-node
        gradient-sync traffic.

     *  Flow State Classification: The hardware identifies the flow's
        current state as UNALLOCATED, SMALL_FLOW (delay-sensitive/
        control), LARGE_FLOW (high-bandwidth parameter synchronization),
        or DRAINING (tail-end flushing).

     These features are extracted, normalized, and encoded at line-rate,
     then written into a high-speed featureFIFO to be sent directly to the
     onboard FPGA.  Simultaneously, the pipeline incorporates real-time
     checksum updates and validation logic for mutable fields in RoCEv2
     (such as ECN and TTL markings) to ensure protocol legitimacy.

  3.2.  State Prediction Layer (FPGA)

     The FPGA reads features from the featureFIFO and executes an ultra-
     low latency, lightweight prediction model (e.g., State-GNN based on
     Graph Neural Networks, linear regression, or heuristic models).  This
     layer focuses on predicting short-term network and load states within
     the next 1-5 milliseconds (ms):

     *  Network State Prediction: Imminent congestion risks on switch
        ports and the available bandwidth of candidate routing paths in
        the next window.

     *  Computing Load Prediction: The arrival time of the next batch of
        periodic tensor traffic, and the probability of queuing backlogs
        or pipeline stalls at the downstream GPU.

     These forward-looking prediction fields serve as the core input for
     the subsequent scheduling engine.

  3.3.  Heuristic Scheduling Layer

     The scheduling engine integrates the currently extracted AI semantics
     with the predicted states output by the FPGA, approximating Pareto
     Optimality amidst conflicting multi-objective goals (e.g., computing
     latency vs. communication overhead).  The decision logic is based on:

     *  Tensor type and structural priority.

     *  Operator dependency.



  Li, et al.              Expires 2 September 2026                [Page 4]


  Internet-Draft                 IntelliNode                    March 2026


     *  Heterogeneous computing capabilities of target nodes.

     *  1-5ms network and congestion predictions.

     The decision outputs (execution actions) include:

     *  Path and Priority: Outputs the optimal path set.  SMALL_FLOWs are
        prioritized based on arrival rate, while LARGE_FLOWs dynamically
        allocate bandwidth based on target computing power using a
        Weighted Deficit Round Robin (WDRR) policy.

     *  Tensor Slicing: Determines if tensor slicing is necessary,
        defining the number of slices and the independent routing path for
        each.

     *  Multipath Aggregation: Decides whether to enable data-plane
        multipath aggregation.

     *  In-Network Offloading: Decides whether to offload specific
        operators (e.g., Sum/Reduce) to in-network FPGAs or edge nodes.

  3.4.  Steering Plane

     The output of the heuristic scheduling must be applied to the entire
     network data plane via a lightweight signaling mechanism (potentially
     as an extension to CATS-SR or CATS-Overlay):

     *  Control Plane Interface: Triggers the local CPU to update the
        routing/forwarding tables, applying the latest policies to the
        next batch of traffic automatically.

     *  Data Plane Labels / TLVs: Pushes extended Metadata TLVs carrying
        tensor types, training phases, and compute resource requests into
        the packet header.

     *  Fragment Routing Encapsulation: Provides necessary Encapsulation
        information for traffic requiring Tensor Slicing.

  4.  Security Considerations

     Given that IntelliNode introduces granular TLV fields for tensor
     semantics and active data-plane scheduling, the system MUST:

     *  Provide integrity protection for TLV fields to prevent malicious
        nodes from tampering with "Tensor Types" to preempt high-priority
        queues.





  Li, et al.              Expires 2 September 2026                [Page 5]


  Internet-Draft                 IntelliNode                    March 2026


     *  Introduce encrypted control-plane channels for telemetry and
        configuration.

     *  Implement authentication to prevent unauthorized nodes from
        falsely broadcasting their Compute-Capability within the computing
        network.

  5.  IANA Considerations

     This document requests that IANA allocate new TLV types for AI-native
     CATS deployments, including but not limited to:

     *  TENSOR_TYPE TLV

     *  TRAINING_PHASE TLV

     *  COMPUTE_CAPABILITY TLV

     *  PATH_PREDICTION TLV

  Authors' Addresses

     Qing Li
     Pengcheng Laboratory
     Email: liq@pcl.ac.cn


     Teng gao
     Pengcheng Laboratory
     Email: gaot@pcl.ac.cn


     Yong Jiang
     Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
     Email: jiangy@sz.tsinghua.edu.cn
















  Li, et al.              Expires 2 September 2026                [Page 6]


name: draft-wang-cats-odsi
rev: 00
title: An Open, Decentralized, and Scalable Framework for Large Language Model Inference
time: 2026-03-02T10:12:07Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T10:12:07Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.302489+00:00
source: ietf
abstract:
  Large Language Model (LLM) inference is increasingly deployed as a
     networked service, yet existing deployments rely primarily on
     centralized infrastructure and trusted operators.  Such designs limit
     openness, concentrate resource ownership, and constrain scalability
     to the capacity of individual providers.  At the same time, LLM
     inference introduces execution characteristics (e.g., strict
     sequential dependencies, large intermediate activations, and tight
     latency requirements) that are not well supported by existing
     network, transport, or coordination mechanisms in open environments.

     This document specifies an open, decentralized, and scalable
     framework for executing LLM inference across independently operated
     and mutually untrusted participants.  The framework treats inference
     as a distributed, layer-wise execution process subject to explicit
     deadlines, rather than as a monolithic computation or best-effort
     service.  It combines layer-aware activation transport and routing,
     decentralized coordination among heterogeneous compute resources, and
     security mechanisms that provide accountability and correctness
     without assuming trusted execution.

     This document focuses on the architectural framework, design
     rationale, problem definition, challenges, and solution space of the
     Open, Decentralized, and Scalable Inference framework (ODSI).  It
     does not specify concrete wire protocols, message formats, or
     protocol state machines.  Such protocol-level specifications are to
     be defined in separate documents that build upon the framework
     described herein.

full_text:
  Computing-Aware Traffic Steering                                 H. Wang
  Internet-Draft                                                     Q. Li
  Intended status: Informational                      Pengcheng Laboratory
  Expires: 3 September 2026                                       Y. Jiang
    Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
                                                                     M. Xu
                                                       Tsinghua University
                                                              2 March 2026


  An Open, Decentralized, and Scalable Framework for Large Language Model
                                 Inference
                          draft-wang-cats-odsi-00

  Abstract

     Large Language Model (LLM) inference is increasingly deployed as a
     networked service, yet existing deployments rely primarily on
     centralized infrastructure and trusted operators.  Such designs limit
     openness, concentrate resource ownership, and constrain scalability
     to the capacity of individual providers.  At the same time, LLM
     inference introduces execution characteristics (e.g., strict
     sequential dependencies, large intermediate activations, and tight
     latency requirements) that are not well supported by existing
     network, transport, or coordination mechanisms in open environments.

     This document specifies an open, decentralized, and scalable
     framework for executing LLM inference across independently operated
     and mutually untrusted participants.  The framework treats inference
     as a distributed, layer-wise execution process subject to explicit
     deadlines, rather than as a monolithic computation or best-effort
     service.  It combines layer-aware activation transport and routing,
     decentralized coordination among heterogeneous compute resources, and
     security mechanisms that provide accountability and correctness
     without assuming trusted execution.

     This document focuses on the architectural framework, design
     rationale, problem definition, challenges, and solution space of the
     Open, Decentralized, and Scalable Inference framework (ODSI).  It
     does not specify concrete wire protocols, message formats, or
     protocol state machines.  Such protocol-level specifications are to
     be defined in separate documents that build upon the framework
     described herein.

  About This Document

     This note is to be removed before publishing as an RFC.




  Wang, et al.            Expires 3 September 2026                [Page 1]


  Internet-Draft                    ODSI                        March 2026


     The latest revision of this draft can be found at
     https://kongyanye.github.io/draft-wang-cats-odsi/draft-wang-cats-
     odsi.html.  Status information for this document may be found at
     https://datatracker.ietf.org/doc/draft-wang-cats-odsi/.

     Discussion of this document takes place on the Computing-Aware
     Traffic Steering Working Group mailing list (mailto:cats@ietf.org),
     which is archived at https://mailarchive.ietf.org/arch/browse/cats/.
     Subscribe at https://www.ietf.org/mailman/listinfo/cats/.

     Source for this draft and an issue tracker can be found at
     https://github.com/kongyanye/draft-wang-cats-odsi.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 3 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   4




  Wang, et al.            Expires 3 September 2026                [Page 2]


  Internet-Draft                    ODSI                        March 2026


       1.1.  Motivation for Open and Decentralized Inference . . . . .   4
       1.2.  Scope and Non-Goals . . . . . . . . . . . . . . . . . . .   6
       1.3.  Design Principles . . . . . . . . . . . . . . . . . . . .   6
     2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.  Problem Definition  . . . . . . . . . . . . . . . . . . . . .   9
       3.1.  Layer-Dependent Execution Pattern . . . . . . . . . . . .   9
       3.2.  Stateful Token-by-Token Progression . . . . . . . . . . .   9
       3.3.  Activation Delivery with Timing Constraints . . . . . . .  10
       3.4.  Open Participation and Adversarial Behavior . . . . . . .  10
       3.5.  Limitations of Existing Execution and Transport
             Methods . . . . . . . . . . . . . . . . . . . . . . . . .  11
     4.  System and Threat Assumptions . . . . . . . . . . . . . . . .  11
       4.1.  Participants and Roles  . . . . . . . . . . . . . . . . .  11
       4.2.  Network and Execution Assumptions . . . . . . . . . . . .  12
       4.3.  Adversary and Failure Model . . . . . . . . . . . . . . .  12
     5.  ODSI Framework Overview . . . . . . . . . . . . . . . . . . .  13
       5.1.  High-Level Architecture . . . . . . . . . . . . . . . . .  13
       5.2.  Layer-Wise Distributed Execution Model  . . . . . . . . .  15
       5.3.  Open Participation and Resource Contribution  . . . . . .  15
       5.4.  Scalability Considerations  . . . . . . . . . . . . . . .  16
     6.  Execution and Coordination Mechanism  . . . . . . . . . . . .  16
       6.1.  Layer Assignment and Path Affinity  . . . . . . . . . . .  16
       6.2.  Handling Stateful Inference and KV Cache  . . . . . . . .  17
       6.3.  Heterogeneous Compute Coordination  . . . . . . . . . . .  17
       6.4.  Failure Handling and Recovery . . . . . . . . . . . . . .  18
     7.  Deadline-Driven Execution and Performance Considerations  . .  18
       7.1.  Deadline Semantics and Slack  . . . . . . . . . . . . . .  18
       7.2.  Latency Sources and Bottlenecks . . . . . . . . . . . . .  19
       7.3.  Routing and Scheduling Considerations . . . . . . . . . .  19
       7.4.  Trade-offs Between Flexibility and Timeliness . . . . . .  20
     8.  Security and Accountability Framework . . . . . . . . . . . .  20
       8.1.  Cryptographic Identity  . . . . . . . . . . . . . . . . .  20
       8.2.  Verifiable Execution Actions  . . . . . . . . . . . . . .  21
       8.3.  Stake-Based Participation and Accountability  . . . . . .  21
       8.4.  Accountability Without Trusted Parties  . . . . . . . . .  22
     9.  Incentives and Economic Considerations  . . . . . . . . . . .  22
       9.1.  Motivation for Incentive Mechanisms . . . . . . . . . . .  22
       9.2.  Costly Misbehavior and Deterrence . . . . . . . . . . . .  23
       9.3.  Reputation and Long-Term Participation  . . . . . . . . .  23
     10. Inference Path and Control Path Separation  . . . . . . . . .  24
       10.1.  Design Rationale . . . . . . . . . . . . . . . . . . . .  24
       10.2.  Inference Path Execution Properties  . . . . . . . . . .  25
       10.3.  Control Path Functions and Enforcement . . . . . . . . .  25
       10.4.  Bounded Risk and Practical Correctness . . . . . . . . .  26
     11. Scalability and Deployment Considerations . . . . . . . . . .  26
       11.1.  Open Membership and Sybil Resistance . . . . . . . . . .  27
       11.2.  Growth and Churn . . . . . . . . . . . . . . . . . . . .  27
       11.3.  Interoperability and Extensibility . . . . . . . . . . .  28



  Wang, et al.            Expires 3 September 2026                [Page 3]


  Internet-Draft                    ODSI                        March 2026


     12. Privacy Considerations  . . . . . . . . . . . . . . . . . . .  28
     13. Relationship to Existing Work . . . . . . . . . . . . . . . .  29
     14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  29
     15. References  . . . . . . . . . . . . . . . . . . . . . . . . .  29
       15.1.  Normative References . . . . . . . . . . . . . . . . . .  29
       15.2.  Informative References . . . . . . . . . . . . . . . . .  30
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  30

  1.  Introduction

     Large Language Models (LLMs) have become a foundational component of
     modern networked applications, supporting tasks such as natural
     language understanding, code generation, and interactive assistants.
     Inference for these models is typically delivered as an online
     service, where user requests are transmitted to remote servers and
     processed incrementally to generate output tokens.  Unlike
     traditional network services or offline model training workloads, LLM
     inference exhibits strict sequential dependencies, maintains per-
     request execution state, and imposes tight latency constraints on
     model response.

     Today, most large-scale LLM inference deployments rely on centralized
     infrastructure operated by a small number of providers.
     Centralization simplifies coordination, scheduling, and state
     management, but it also concentrates control, limits participation,
     and couples scalability to the capacity, geography, and policies of
     individual operators.  As model sizes and inference demand continue
     to grow, these structural limitations motivate the exploration of
     alternative execution paradigms that can support broader
     participation and more elastic scaling.

     This document introduces the Open, Decentralized, and Scalable
     Inference framework (ODSI), an architectural framework for executing
     LLM inference across independently operated and heterogeneous compute
     resources.  ODSI considers environments in which participants are
     mutually untrusted and connected only through the public Internet.
     The framework focuses on how inference execution can be coordinated,
     secured, and scaled under such conditions, without assuming
     centralized control or trusted execution environments.

  1.1.  Motivation for Open and Decentralized Inference

     Centralized inference architectures assume that compute resources,
     network paths, and execution environments are under common
     administrative control.  While these assumptions enable tight
     optimization and simplified coordination, they impose structural
     constraints that become increasingly pronounced as model sizes
     continue to grow.  Modern LLMs require substantial compute throughput



  Wang, et al.            Expires 3 September 2026                [Page 4]


  Internet-Draft                    ODSI                        March 2026


     and memory capacity, and deploying a single model instance often
     exceeds the capabilities of an individual server.  As a result,
     inference deployments increasingly rely on multiple servers to host
     and execute a single model, raising operational complexity and cost.

     At the same time, a large volume of distributed compute resources
     remains underutilized.  Many devices and servers possess limited
     memory capacity or modest compute performance, preventing them from
     hosting complete model instances despite having available compute
     resources.  These constraints lead to fragmented and wasted capacity,
     particularly at the network edge or within smaller organizations,
     where individual nodes cannot independently support large models even
     though aggregate resources may be sufficient.

     Centralized cloud-based inference also introduces data movement and
     privacy concerns.  User inputs must be transmitted to remote data
     centers for processing, increasing exposure to data leakage and
     raising privacy risks in sensitive applications.  In addition,
     aggregating large volumes of inference traffic at centralized
     endpoints places sustained pressure on network bandwidth, increases
     transmission and queuing delays, and creates service bottlenecks that
     limit horizontal scalability.  As demand grows, scaling inference
     capacity requires proportional expansion of centralized
     infrastructure and network provisioning, which may not be
     economically or operationally sustainable.

     An open and decentralized inference model seeks to address these
     limitations by allowing independently operated participants to
     contribute partial compute resources without requiring prior trust
     relationships or centralized admission.  By distributing inference
     execution across many nodes, including resource-constrained and edge
     devices, this paradigm enables inference capacity to scale
     elastically with participation.  Placing computation closer to data
     sources can also reduce data movement, mitigate bandwidth
     bottlenecks, and improve responsiveness in certain deployment
     scenarios.

     However, decentralization fundamentally changes the inference
     execution environment.  Participants may vary widely in compute
     performance, memory capacity, network connectivity, and availability,
     and some participants may behave maliciously or rationally rather
     than altruistically.  Moreover, LLM inference is inherently stateful,
     i.e., the generation of each output token depends on all previous
     tokens, commonly represented through cached intermediate values such
     as key–value (KV) caches.  These characteristics make inference
     sensitive to delays, failures, and inconsistencies, and prevent it
     from being treated as a stateless or best-effort distributed task.




  Wang, et al.            Expires 3 September 2026                [Page 5]


  Internet-Draft                    ODSI                        March 2026


     ODSI is motivated by the need to support open participation and
     elastic scaling while preserving the correctness, timeliness, and
     reliability required for practical LLM inference.  The framework
     addresses how inference can be executed across decentralized and
     heterogeneous resources while remaining usable as an interactive
     network service.

  1.2.  Scope and Non-Goals

     This document defines the architectural framework, problem
     formulation, and design considerations for decentralized LLM
     inference under open participation.  It identifies the key challenges
     introduced by decentralization, including state management, latency
     constraints, heterogeneity, and adversarial behavior, and describes
     the high-level mechanisms used to address these challenges within the
     ODSI framework.

     This document does not specify concrete network protocols, wire
     formats, message encodings, or protocol state machines.  It also does
     not mandate specific model architectures, execution platforms,
     hardware accelerators, or economic systems.  Where cryptographic,
     incentive, or coordination mechanisms are discussed, they are
     described at an abstract level to illustrate design intent rather
     than to prescribe particular implementations.

     Protocol-level specifications, interoperability requirements, and
     implementation details are to be defined in other documents that
     build upon the framework presented here.

  1.3.  Design Principles

     The ODSI framework is guided by the following design principles:

     *  Open Participation: Any independently operated participant may
        contribute compute resources without requiring centralized
        admission or prior trust, subject to mechanisms that provide
        accountability and abuse resistance.

     *  Decentralized Coordination: Inference execution is coordinated
        without assuming a single trusted controller, relying instead on
        distributed mechanisms that tolerate heterogeneity, failures, and
        adversarial behavior.

     *  State-Aware Execution: The framework explicitly accounts for the
        stateful and sequential nature of LLM inference, including the
        management of intermediate execution state across tokens and
        layers.




  Wang, et al.            Expires 3 September 2026                [Page 6]


  Internet-Draft                    ODSI                        March 2026


     *  Deadline Sensitivity: Inference execution is treated as a latency-
        sensitive process, where intermediate steps are subject to
        explicit timing constraints rather than best-effort delivery.

     *  Scalability Through Composition: The framework is designed to
        scale by composing many independent contributors, allowing overall
        inference capacity to grow with participation rather than
        centralized provisioning.

     These principles inform the architectural choices and mechanisms
     described in the remainder of this document.

  2.  Terminology

     This section defines the terminology used throughout this document.
     Phrases in upper-case refer to other defined terms.

     ACTIVATION

     Intermediate numerical data produced by executing a model layer
     during inference.  ACTIVATIONS are consumed by subsequent layers and
     may be transmitted across the network between participants.

     CONTROL PLANE

     The non-latency-critical path responsible for coordination,
     verification, and enforcement functions, including cryptographic
     IDENTITY registration, STAKE management, SLASHING, REWARD settlement,
     and REPUTATION updates.

     DEADLINE

     A time constraint by which a specific inference step, such as a layer
     execution or ACTIVATION delivery, must complete to preserve end-to-
     end responsiveness.

     EXECUTION COMMITMENT

     A cryptographically signed declaration by a PARTICIPANT indicating
     intent to execute a specific inference step under defined inputs and
     DEADLINES, enabling later verification.

     IDENTITY

     A persistent, self-generated identifier bound to a PARTICIPANT,
     typically realized as a public–private key pair.

     INFERENCE PLANE



  Wang, et al.            Expires 3 September 2026                [Page 7]


  Internet-Draft                    ODSI                        March 2026


     The latency-critical path responsible for performing inference-
     related work, including LAYER execution, ACTIVATION transport,
     ROUTING, RE-ROUTING, and failure signaling.

     LAYER

     A discrete computation stage within an inference pipeline.  LAYERS
     are executed sequentially for each token step and may be assigned to
     different execution participants.

     PARTICIPANT

     An independently operated entity that contributes compute, memory, or
     network resources to ODSI and participates using a cryptographic
     IDENTITY.

     REPUTATION

     A persistent performance signal associated with a PARTICIPANT,
     derived from historical correctness, DEADLINE adherence,
     availability, and throughput.  REPUTATION influences task assignment
     and reward rates.

     RE-ROUTING

     The process of dynamically changing the ROUTING or LAYER assignment
     of an ongoing inference request in response to failures, performance
     degradation, or DEADLINE pressure.

     REWARD

     An economic payment issued to a PARTICIPANT for successfully
     completing an assigned inference task.

     ROUTING

     The selection of execution participants and network paths for
     ACTIVATION transport and LAYER execution, informed by DEADLINES and
     observed performance.

     SLACK

     The difference between an allocated DEADLINE budget and the expected
     execution time for an inference step.  SLACK represents tolerance to
     variability and delay.

     SLASHING




  Wang, et al.            Expires 3 September 2026                [Page 8]


  Internet-Draft                    ODSI                        March 2026


     An enforced economic penalty applied to a PARTICIPANT when verifiable
     misbehavior is detected, such as incorrect output, missed DEADLINES,
     or commitment violations.

     STAKE

     A quantity of economic value locked by a PARTICIPANT as collateral
     for participation.  STAKE enables accountability, economic
     deterrence, and Sybil resistance

  3.  Problem Definition

     This section defines the core problem addressed by the ODSI
     framework.  The goal is to formalize the execution constraints and
     failure modes that arise when LLM inference is performed across open,
     distributed, and independently operated compute resources.  These
     constraints collectively define what it means for decentralized
     inference to be correct, timely, and usable.

  3.1.  Layer-Dependent Execution Pattern

     LLM inference executes as a fixed sequence of model layers applied
     repeatedly for each generated token.  The output of each layer
     constitutes an intermediate activation that is required as input to
     the next layer in the sequence.  Execution therefore forms a strict
     dependency chain at layer granularity.

     In a decentralized setting, different layers may be executed on
     different nodes.  This introduces an explicit requirement that
     intermediate activations be transferred between nodes in the correct
     order and without duplication or omission.  Any execution framework
     must preserve layer ordering and ensure that each layer operates on
     the correct input corresponding to a specific inference request and
     token position.

  3.2.  Stateful Token-by-Token Progression

     Inference proceeds incrementally, generating tokens one at a time.
     Each token depends on an execution state accumulated from all
     previous tokens, commonly including cached intermediate values such
     as KV caches.  This state is logically persistent over the lifetime
     of the inference request.









  Wang, et al.            Expires 3 September 2026                [Page 9]


  Internet-Draft                    ODSI                        March 2026


     As a result, inference requests exhibit execution affinity, i.e.,
     successive tokens must either be processed by nodes that already
     possess the relevant state or incur the cost of state transfer or
     reconstruction.  Failures, delays, or inconsistencies in state
     handling directly affect correctness and latency.  The problem
     therefore includes maintaining coherent per-request state across a
     sequence of distributed execution steps.

  3.3.  Activation Delivery with Timing Constraints

     For interactive applications, inference must progress under tight
     latency constraints.  Each layer execution contributes both
     computation delay and communication delay, and delays in earlier
     layers propagate to all subsequent layers within the same token
     generation step.

     This creates implicit per-layer timing constraints, i.e.,
     intermediate activations must be delivered within bounded time to
     sustain acceptable end-to-end response latency.  The problem is not
     merely reliable delivery, but timely delivery under variable network
     conditions and heterogeneous execution speeds.

     The activation delivery problem is defined as follows.  Given a
     sequence of layer-dependent computations distributed across multiple
     nodes, how can intermediate activations be delivered and processed in
     order, within time bounds, despite variability in network latency,
     compute throughput, and node availability?

  3.4.  Open Participation and Adversarial Behavior

     ODSI assumes an open execution environment in which nodes may join or
     leave without centralized admission and are operated by independent
     parties.  Participants may differ significantly in performance,
     reliability, and incentives, and some may behave maliciously or
     rationally rather than cooperatively.

     The problems to solve therefore includes:

     *  Detecting incorrect or inconsistent execution results,

     *  Attributing actions to specific participants,

     *  Preventing abuse such as equivocation, free-riding, or denial of
        service,

     *  Enabling accountability without assuming trusted execution
        environments.




  Wang, et al.            Expires 3 September 2026               [Page 10]


  Internet-Draft                    ODSI                        March 2026


     Any viable solution must address correctness and liveness under these
     conditions while remaining compatible with open participation.

  3.5.  Limitations of Existing Execution and Transport Methods

     Existing execution and transport methods do not directly address the
     problem defined above.  Best-effort networking does not account for
     execution dependencies or timing constraints.  Traditional
     distributed computation frameworks assume stable membership, trusted
     operators, or coarse-grained tasks.  Centralized schedulers do not
     extend naturally to environments without common administrative
     control.

     The problem addressed by ODSI is therefore not solved by simply
     distributing computation or improving transport performance.  It
     requires a framework that explicitly integrates execution
     dependencies, state management, timing constraints, and participant
     accountability into the design.

  4.  System and Threat Assumptions

     This section specifies the assumptions under which the ODSI framework
     operates.  It defines the participating entities, communication and
     execution conditions, and the classes of failures and adversarial
     behavior the framework is designed to tolerate.  No centralized
     trust, privileged operators, or trusted execution environments are
     assumed.

  4.1.  Participants and Roles

     The system consists of a set of independently operated nodes that
     participate in inference execution.  Nodes may assume one or more of
     the following roles:

     *  *Clients* initiate inference requests and receive generated
        outputs.  A client may or may not also participate in execution.

     *  *Execution Nodes* perform inference computation, typically
        executing one or more model layers for specific inference
        requests.  Execution nodes may differ in compute capacity, memory
        availability, and supported hardware.

     *  *Coordination Nodes* participate in control-plane functions such
        as identity management, stake accounting, verification, and
        settlement.  These roles may be co-located with execution nodes or
        operated separately.





  Wang, et al.            Expires 3 September 2026               [Page 11]


  Internet-Draft                    ODSI                        March 2026


     All participants are identified by persistent cryptographic
     identities.  No global trust relationships are assumed among
     participants, and no role is restricted to a fixed or privileged set
     of operators.

  4.2.  Network and Execution Assumptions

     Nodes communicate over the public Internet using unreliable,
     asynchronous networks.  Message delivery may experience variable
     latency, reordering, duplication, or loss.  No assumptions are made
     about bounded network delay or synchronized clocks, except where
     explicitly stated by higher-layer mechanisms.

     Execution nodes are heterogeneous.  They may differ in:

     *  Compute throughput and supported instruction sets,

     *  Available memory and storage,

     *  Network bandwidth and latency,

     *  Availability and uptime.

     Nodes may join or leave the system at arbitrary times.  Execution may
     be interrupted due to failures, preemption, or voluntary withdrawal.
     The system does not assume trusted execution environments or
     hardware-based attestation, and correctness cannot be inferred solely
     from successful message delivery.

  4.3.  Adversary and Failure Model

     The framework assumes the presence of faulty, rational, and malicious
     participants.  Nodes may deviate arbitrarily from prescribed
     behavior, including but not limited to:

     *  Returning incorrect or fabricated inference outputs,

     *  Withholding results or responding after deadlines,

     *  Equivocating by providing inconsistent results to different peers,

     *  Attempting to free-ride without performing assigned computation,

     *  Launching denial-of-service or resource exhaustion attacks.

     In addition to malicious behavior, the system must tolerate non-
     malicious failures such as crashes, network partitions, and transient
     performance degradation.



  Wang, et al.            Expires 3 September 2026               [Page 12]


  Internet-Draft                    ODSI                        March 2026


     The adversary is not assumed to control a majority of system
     resources globally, but may control multiple identities unless
     mitigated by Sybil-resistance mechanisms.  The framework does not
     assume confidentiality of intermediate activations unless explicitly
     provided by higher-layer mechanisms.

     Under these assumptions, the framework aims to provide:

     *  Safety: incorrect inference results can be detected and
        attributed,

     *  Liveness: inference can make progress despite failures and churn,

     *  Accountability: misbehavior can be penalized without relying on
        trusted authorities.

  5.  ODSI Framework Overview

     This section provides a high-level overview of the ODSI framework.
     The framework defines how large-scale inference execution can be
     coordinated across open, heterogeneous, and independently operated
     resources while preserving correctness, timeliness, and
     accountability.

     ODSI is structured as a layered framework that integrates execution,
     coordination, and incentive mechanisms.  It does not mandate a
     specific implementation or protocol stack, but instead defines
     architectural components and their interactions.

  5.1.  High-Level Architecture





















  Wang, et al.            Expires 3 September 2026               [Page 13]


  Internet-Draft                    ODSI                        March 2026


                      +----------------------+
                      |        Client        |
                      |  (Inference Request) |
                      +----------+-----------+
                                 |
                                 v
     =========================================================
     |                    INFERENCE PLANE                    |
     |   (Deadline-Critical, Peer-to-Peer Inference Path)    |
     |                                                       |
     |   +-----------+     +-----------+     +-----------+   |
     |   |  Layer i  | --> | Layer i+1 | --> | Layer i+2 |   |
     |   | Exec Node |     | Exec Node |     | Exec Node |   |
     |   +-----------+     +-----------+     +-----------+   |
     |       |                   |                   |       |
     |       |        Intermediate Activations       |       |
     |       +---------------------------------------+       |
     |                                                       |
     =========================================================
                                 |
                                 |   Execution Events
                                 |   (commit, result, timing)
                                 v
     ---------------------------------------------------------
     |                     CONTROL PLANE                     |
     |     (Asynchronous Coordination and Accountability)    |
     |                                                       |
     |   +----------------+   +--------------------------+   |
     |   | Identity and   |   | Verification and Dispute |   |
     |   | Stake Manage.  |   | Resolution               |   |
     |   +----------------+   +--------------------------+   |
     |            |                     |                    |
     |            v                     v                    |
     |   +----------------+   +--------------------------+   |
     |   | Incentives and |   | Reputation and Routing   |   |
     |   | Settlement     |   | Feedback                 |   |
     |   +----------------+   +--------------------------+   |
     |                                                       |
     ---------------------------------------------------------

     At a high level, ODSI separates inference execution into two
     logically distinct planes:

     *  Execution Plane: Responsible for performing inference computation
        and delivering intermediate results under latency constraints.

     *  Control Plane: Responsible for identity management, coordination,
        verification, accounting, and enforcement.



  Wang, et al.            Expires 3 September 2026               [Page 14]


  Internet-Draft                    ODSI                        March 2026


     The execution plane operates in a peer-to-peer fashion and is
     optimized for low-latency, deadline-sensitive computation.  The
     control plane operates asynchronously and does not block inference
     progress, allowing execution to proceed even in the presence of
     coordination delays.

     This separation enables inference to remain responsive while still
     supporting accountability and correctness in an open environment.

  5.2.  Layer-Wise Distributed Execution Model

     ODSI adopts a layer-wise execution model in which inference
     computation is decomposed into sequential stages corresponding to the
     layers of the inference graph.  Each stage may be executed by a
     different execution node, and intermediate results are transferred
     between nodes as needed.

     Execution proceeds incrementally for each inference request, with
     explicit association between:

     *  A specific inference request,

     *  A specific token generation step,

     *  A specific computation layer.

     This explicit structuring allows the framework to reason about
     execution dependencies, timing constraints, and correctness at layer
     granularity.  It also enables flexible placement of computation
     across nodes with varying capabilities, without requiring any single
     node to host the entire inference workload.

  5.3.  Open Participation and Resource Contribution

     ODSI is designed to support open participation without centralized
     admission control.  Any node may contribute resources to inference
     execution by assuming execution or coordination roles, subject to
     framework-defined requirements for identity, accountability, and
     correctness.

     Resource contribution is flexible and may include:

     *  Compute capacity for executing specific computation stages,

     *  Memory for maintaining execution state,

     *  Network capacity for transporting intermediate results.




  Wang, et al.            Expires 3 September 2026               [Page 15]


  Internet-Draft                    ODSI                        March 2026


     Nodes are not required to support complete inference execution.
     Instead, they may contribute partial resources aligned with their
     capabilities.  This allows resource-constrained or edge nodes to
     participate meaningfully, while enabling aggregate inference capacity
     to scale with the number of participants.

  5.4.  Scalability Considerations

     Scalability in ODSI is achieved through decentralization,
     decomposition, and asynchronous coordination.  By distributing
     execution across many independently operated nodes, the framework
     avoids reliance on centralized bottlenecks.

     Key scalability properties include:

     *  Horizontal scaling: Inference capacity increases with the number
        of participating nodes.

     *  Elastic participation: Nodes may join or leave without global
        reconfiguration.

     *  Local decision-making: Execution placement and routing decisions
        can be made using local observations rather than global state.

     The framework is designed to tolerate heterogeneity and churn while
     maintaining bounded coordination overhead.  As inference demand
     grows, scalability is achieved by expanding participation rather than
     by increasing the capacity of centralized infrastructure.

  6.  Execution and Coordination Mechanism

     This section describes how inference execution is coordinated across
     distributed participants in the ODSI framework.  It focuses on how
     computation is assigned, how execution state is preserved, how
     heterogeneous resources are coordinated, and how failures are handled
     during inference execution.

  6.1.  Layer Assignment and Path Affinity

     Inference execution in ODSI is organized as a sequence of layer
     executions.  For each inference request, layers are assigned to
     execution nodes based on their capabilities, availability, and
     observed performance characteristics.








  Wang, et al.            Expires 3 September 2026               [Page 16]


  Internet-Draft                    ODSI                        March 2026


     ODSI introduces the notion of execution path affinity, whereby
     successive layers and token steps of the same inference request
     preferentially follow a consistent sequence of nodes.  Path affinity
     reduces the need to transfer execution state and intermediate data,
     thereby improving latency and reducing network overhead.

     Layer assignment decisions may be adapted dynamically in response to
     changing network conditions or node availability.  However,
     reassignment is performed conservatively to avoid excessive state
     migration or disruption of ongoing inference execution.  Under
     typical operating conditions, stable execution paths are expected to
     dominate, and recovery-related latency remains within acceptable
     bounds for interactive inference.

  6.2.  Handling Stateful Inference and KV Cache

     Inference execution maintains per-request state across token
     generation steps.  This state commonly includes cached intermediate
     values such as KV caches, which are required for efficient generation
     of subsequent tokens.

     ODSI treats execution state as logically associated with an execution
     path rather than with a single node.  State may be:

     *  Retained locally by execution nodes across successive steps,

     *  Transferred explicitly when execution is reassigned,

     *  Reconstructed when transfer is infeasible or too costly.

     This document does not mandate a specific state representation or
     transfer mechanism.  Instead, it defines coordination requirements
     that ensure state consistency and correctness across execution steps.

  6.3.  Heterogeneous Compute Coordination

     Execution nodes in ODSI are heterogeneous in compute performance,
     memory capacity, and network connectivity.  Coordination mechanisms
     must account for this heterogeneity when assigning computation and
     routing intermediate results.

     Nodes may advertise capabilities and performance metrics, such as
     execution throughput or observed latency.  Coordination decisions may
     incorporate these metrics to balance load, avoid bottlenecks, and
     satisfy timing constraints.






  Wang, et al.            Expires 3 September 2026               [Page 17]


  Internet-Draft                    ODSI                        March 2026


     The framework supports partial participation, allowing nodes to
     execute only those computation stages that align with their
     capabilities.  This enables broad participation without requiring
     uniform hardware or resource provisioning.

  6.4.  Failure Handling and Recovery

     ODSI is designed to tolerate failures during inference execution,
     including node crashes, network disruptions, and missed execution
     deadlines.

     Failure handling strategies include:

     *  Detecting stalled or failed execution steps through timeout or
        absence of expected outputs,

     *  Reassigning computation to alternative nodes when failures occur,

     *  Reconstructing execution state when necessary to resume inference.

     The framework prioritizes forward progress and bounded recovery cost.
     Failures may result in degraded performance or recomputation, but
     should not compromise correctness or global system stability.

     Recovery mechanisms are coordinated without assuming centralized
     control and do not require halting unrelated inference requests.

  7.  Deadline-Driven Execution and Performance Considerations

     Interactive inference services impose strict latency requirements
     that shape execution, coordination, and resource selection decisions.
     This section describes how ODSI reasons about deadlines, identifies
     sources of latency, and manages trade-offs between flexibility and
     timely execution.

  7.1.  Deadline Semantics and Slack

     In ODSI, inference execution is associated with explicit or implicit
     deadlines derived from application-level responsiveness requirements.
     Deadlines apply not only to end-to-end inference requests but also to
     intermediate execution steps, such as individual layer computations
     within a token generation cycle.









  Wang, et al.            Expires 3 September 2026               [Page 18]


  Internet-Draft                    ODSI                        March 2026


     Each execution step may be assigned a deadline budget, representing
     the maximum allowable time from input availability to output
     production.  The difference between the allocated budget and the
     expected execution time is referred to as slack.  Slack captures
     tolerance to variability in computation and communication and serves
     as a key signal for execution planning.

     Slack is consumed as inference progresses.  Delays incurred at
     earlier steps reduce the slack available to downstream steps, making
     subsequent execution increasingly time-sensitive.  The framework
     therefore prioritizes maintaining positive slack throughout execution
     to preserve responsiveness.

  7.2.  Latency Sources and Bottlenecks

     End-to-end inference latency arises from multiple sources, including:

     *  Computation time for executing individual layers,

     *  Data transfer time for delivering intermediate results,

     *  Queuing delays caused by contention for compute or network
        resources,

     *  Coordination overhead for assignment and verification.

     In decentralized environments, these latency components are highly
     variable and may fluctuate over short time scales.  Bottlenecks may
     shift dynamically due to changes in network conditions, node
     availability, or workload distribution.

     ODSI does not assume that any single latency source dominates.
     Instead, it treats latency as a composite effect and seeks to
     minimize the risk of deadline violations by accounting for both
     computation and communication delays when coordinating execution.

  7.3.  Routing and Scheduling Considerations

     Routing and scheduling decisions in ODSI are informed by deadline
     constraints and observed performance.  Execution steps may be routed
     through nodes that offer favorable trade-offs between compute
     throughput, network latency, and reliability.

     Scheduling decisions may incorporate:

     *  Estimated computation time based on historical performance,

     *  Network round-trip latency and variability,



  Wang, et al.            Expires 3 September 2026               [Page 19]


  Internet-Draft                    ODSI                        March 2026


     *  Current queue occupancy or load,

     *  Remaining slack for the inference request.

     The framework favors execution paths that preserve slack and reduce
     the probability of deadline violations.  However, routing decisions
     are made using incomplete and potentially stale information, and must
     therefore tolerate uncertainty.

  7.4.  Trade-offs Between Flexibility and Timeliness

     ODSI balances execution flexibility against the need for timely
     completion.  Allowing frequent reassignment or dynamic
     reconfiguration can improve robustness and load balancing but may
     introduce additional coordination overhead and state transfer costs.

     Conversely, favoring stable execution paths improves predictability
     and reduces overhead but may limit the system’s ability to respond to
     failures or performance degradation.

     The framework does not prescribe a single optimal balance.  Instead,
     it defines a design space in which implementations may tune the
     degree of flexibility based on workload characteristics, deployment
     conditions, and performance objectives.  The guiding principle is to
     favor timely execution in the common case, while retaining sufficient
     adaptability to preserve progress under adverse conditions.

  8.  Security and Accountability Framework

     This section describes the security and accountability mechanisms
     assumed by the ODSI framework.  The framework operates in an open
     environment with mutually untrusted participants and therefore relies
     on cryptographic mechanisms and economic accountability rather than
     trusted operators or centralized enforcement.

  8.1.  Cryptographic Identity

     Each participant in the system is associated with a persistent
     cryptographic identity, typically represented by a public–private key
     pair [RFC6979].  This identity serves as the basis for
     authentication, attribution, and accountability across all
     interactions.

     All inference-related actions, including execution commitments,
     result submissions, and coordination messages, are bound to the
     participant’s identity through digital signatures.  This binding
     ensures that actions can be reliably attributed to specific
     participants without relying on centralized identity providers.



  Wang, et al.            Expires 3 September 2026               [Page 20]


  Internet-Draft                    ODSI                        March 2026


     Identities are self-generated and do not imply trust.  The framework
     assumes that a single entity may control multiple identities unless
     constrained by additional mechanisms such as economic bonding or
     resource-based admission.

  8.2.  Verifiable Execution Actions

     ODSI requires that execution-related actions be verifiable, meaning
     that they can be independently checked for consistency, correctness,
     or policy compliance after the fact.

     Verifiable actions may include:

     *  Commitments to execute specific computation stages,

     *  Submission of intermediate or final execution outputs,

     *  Timing assertions related to execution deadlines.

     Verification does not require continuous oversight during execution.
     Instead, it relies on cryptographic commitments, hashes, and signed
     messages that allow third parties to reconstruct and evaluate
     execution behavior when disputes arise.

     This approach enables detection of incorrect execution, equivocation,
     or deadline violations without imposing synchronous verification on
     the critical execution path [Byzantine].

  8.3.  Stake-Based Participation and Accountability

     ODSI employs stake-based participation as a foundational mechanism
     for accountability and Sybil resistance.  To serve inference
     requests, participants are required to lock a quantity of economic
     value as stake, which acts as collateral against misbehavior.

     Stake introduces a real economic cost to participation and creates
     persistent consequences for incorrect or unreliable behavior.  When
     misbehavior is verified—such as incorrect execution, commitment
     violations, or repeated deadline failures—stake may be partially or
     fully forfeited according to predefined rules.

     Stake directly enables accountability by ensuring that identities are
     bound to economic risk.  It also provides an economic basis for Sybil
     resistance: while identities are inexpensive to create, meaningful
     participation and influence require proportional stake.  As a result,
     large-scale Sybil attacks require substantial capital commitment and
     expose the adversary to high risk of loss.




  Wang, et al.            Expires 3 September 2026               [Page 21]


  Internet-Draft                    ODSI                        March 2026


     The influence and opportunities afforded to a participant may be
     proportional to both locked stake and historical performance.  This
     ensures that influence reflects sustained contribution rather than
     identity count alone.

  8.4.  Accountability Without Trusted Parties

     The framework is designed to provide accountability without assuming
     trusted coordinators, validators, or execution environments.
     Accountability is achieved by combining identity-bound actions with
     verifiable evidence and enforceable consequences.

     When misbehavior is detected, responsibility can be attributed to
     specific identities based on signed execution records.  Consequences,
     such as penalties or exclusion from future participation, can then be
     applied according to defined rules.

     This design ensures that participants are held accountable for their
     actions while preserving open participation and decentralization.
     Trust is replaced by verification and consequence, allowing the
     system to operate securely in adversarial environments.

  9.  Incentives and Economic Considerations

     ODSI operates in an open environment where participation is voluntary
     and participants are assumed to act in their own interest.  As a
     result, correct and timely execution cannot be assumed to arise from
     cooperation alone.  This section outlines the economic mechanisms
     that align participant incentives with system objectives, ensuring
     reliable execution under decentralized operation.

  9.1.  Motivation for Incentive Mechanisms

     In a decentralized inference environment, participants contribute
     compute, memory, and network resources that incur real costs.
     Without explicit incentives, participants may decline execution
     assignments, deprioritize inference workloads, or abandon execution
     paths when conditions become unfavorable.

     ODSI therefore associates inference execution with explicit economic
     rewards [Bitcoin].  Each successfully executed layer earns a payment,
     creating a direct linkage between contributed work and compensation.
     Reward levels may vary based on execution conditions, including:

     *  Tighter execution deadlines, which impose higher performance
        requirements,





  Wang, et al.            Expires 3 September 2026               [Page 22]


  Internet-Draft                    ODSI                        March 2026


     *  Placement on latency-critical or bottleneck segments of an
        execution path,

     *  Historical reliability and performance of the executing
        participant.

     By rewarding each completed execution unit, ODSI enables fine-grained
     accounting and encourages participants to contribute resources
     proportionally to their capabilities.

  9.2.  Costly Misbehavior and Deterrence

     For incentives to be effective, misbehavior must be economically
     disadvantageous.  ODSI assumes that participants may behave
     strategically, including submitting incorrect activation outputs,
     violating execution commitments, or missing assigned deadlines.

     Misbehavior triggers penalties through the control path.  Slashing
     events may occur in response to:

     *  Incorrect or invalid activation outputs,

     *  Mismatches between committed execution inputs and revealed
        results,

     *  Failure to meet agreed execution deadlines.

     Penalties are calibrated to exceed the expected gains from cheating
     or shirking, ensuring that rational participants cannot profit from
     misbehavior even if detection is probabilistic.  Slashing may involve
     forfeiture of locked stake, loss of accrued rewards, or other
     economically meaningful consequences.

     Penalties are applied only when sufficient cryptographic and
     execution evidence exists to attribute responsibility to a specific
     identity.  This ensures that deterrence is precise and does not
     require centralized trust or continuous supervision.

  9.3.  Reputation and Long-Term Participation

     While per-layer payments and penalties shape short-term behavior,
     long-term reliability is reinforced through reputation mechanisms.
     Reputation reflects a participant’s historical performance across
     multiple dimensions, including:

     *  Deadline adherence rate,

     *  Output correctness and consistency,



  Wang, et al.            Expires 3 September 2026               [Page 23]


  Internet-Draft                    ODSI                        March 2026


     *  Availability and responsiveness,

     *  Sustained execution throughput over time.

     Reputation directly influences future participation opportunities.
     Participants with strong reputations may receive higher task volumes,
     preferential assignment to latency-critical execution paths, higher
     reward rates, or access to tighter deadlines.  Conversely,
     participants with poor or unstable reputations may receive fewer
     assignments, reduced compensation, or eventual exclusion.

     Reputation complements direct economic incentives by encouraging
     sustained, honest participation across many execution sessions.
     Together, per-layer rewards, slashing-based deterrence, and
     reputation-driven coordination create a self-reinforcing environment
     in which rational participants are motivated to behave reliably,
     enabling scalable and decentralized inference execution.

  10.  Inference Path and Control Path Separation

     ODSI separates inference execution into an inference path and a
     control path in order to reconcile strict latency requirements with
     the need for security, accountability, and open participation
     (analogy to Lightning [Lightning]).  The inference path is
     responsible for latency-critical execution and data movement required
     to generate inference outputs.  The control path is responsible for
     identity management, economic coordination, verification, and
     enforcement.  Together, they form a coherent architecture that
     decouples performance-sensitive execution from governance and
     accountability functions.

  10.1.  Design Rationale

     Interactive inference requires execution progress within tight and
     predictable time bounds.  Operations such as cryptographic identity
     registration, stake management, global verification, or reward
     settlement cannot be placed directly on the critical execution path
     without violating these constraints.

     At the same time, an open and decentralized environment requires
     mechanisms to deter misbehavior, attribute responsibility, and
     enforce incentives.  These mechanisms inherently involve
     coordination, state persistence, and potentially delayed resolution.

     The separation between the inference path and the control path
     addresses this tension by allowing inference execution to proceed
     optimistically and independently, while ensuring that execution
     remains observable, attributable, and enforceable.  Lightweight



  Wang, et al.            Expires 3 September 2026               [Page 24]


  Internet-Draft                    ODSI                        March 2026


     checks on the inference path reduce the likelihood that incorrect
     results propagate to users, while the control path provides
     definitive validation, economic settlement, and long-term
     accountability.

  10.2.  Inference Path Execution Properties

     The inference path carries all latency-critical activities required
     for inference execution.  This includes:

     *  Layer-wise computation across participating nodes,

     *  Transport of intermediate activations and execution state,

     *  Routing and re-routing decisions based on network and execution
        conditions,

     *  Failure detection and signaling to enable timely recovery.

     The inference path is optimized for low latency, minimal coordination
     overhead, and predictable progress under Internet variability.

     Inference path execution is optimistic but constrained.  Participants
     are not blindly trusted, and incorrect execution is mitigated through
     the following mechanisms:

     *  Execution commitments: Participants commit to execution inputs,
        layer identifiers, and deadlines before revealing outputs,
        preventing adaptive or inconsistent behavior.

     *  Selective redundancy: For high-impact layers or execution steps,
        multiple participants may perform the same computation, allowing
        mismatches to be detected through lightweight comparison.

     *  State-local execution: Execution path affinity minimizes state
        transfer and confines errors to limited execution segments.

     These mechanisms allow many incorrect executions to be detected
     within the same token step or shortly thereafter, enabling rapid
     fallback or localized re-execution before incorrect results propagate
     to the user.

  10.3.  Control Path Functions and Enforcement

     The control path operates asynchronously and is responsible for
     system-wide coordination and accountability.  Its functions include,
     but are not limited to:




  Wang, et al.            Expires 3 September 2026               [Page 25]


  Internet-Draft                    ODSI                        March 2026


     *  Cryptographic identity registration and authentication,

     *  Stake locking, management, and release,

     *  Validation of execution commitments and revealed results,

     *  Slashing decisions for incorrect execution or missed deadlines,

     *  Reward calculation and settlement,

     *  Maintenance of long-term reputation or eligibility signals.

     Because control path operations are not latency-critical, they can
     perform thorough verification and policy enforcement without delaying
     inference execution.  Control path decisions are driven by signed
     execution records and verifiable evidence produced during inference
     path execution.

     While some violations may be detected only after inference has
     progressed or completed, economic deterrence and reputational
     consequences make sustained misbehavior irrational for participants
     seeking long-term participation.

  10.4.  Bounded Risk and Practical Correctness

     The separation between the inference path and the control path
     introduces bounded risk rather than absolute prevention of incorrect
     execution.  However, the combination of early detection on the
     inference path, rapid fallback mechanisms, and strong economic
     deterrence on the control path ensures that incorrect results are
     rare, localized, and unlikely to persist undetected.

     This design follows a well-established distributed systems principle:
     optimistic execution combined with eventual verification.  ODSI
     applies this principle to open and decentralized inference, enabling
     high-throughput, low-latency execution while preserving
     accountability and system integrity.

  11.  Scalability and Deployment Considerations

     ODSI is designed to scale across large numbers of independently
     operated participants and to operate under dynamic network and
     resource conditions.  This section discusses considerations related
     to open participation, participant churn, and long-term extensibility
     of the framework.






  Wang, et al.            Expires 3 September 2026               [Page 26]


  Internet-Draft                    ODSI                        March 2026


  11.1.  Open Membership and Sybil Resistance

     ODSI supports open membership, allowing any participant to contribute
     computational resources without requiring centralized admission or
     prior trust relationships.  This openness is essential for elastic
     scaling and for harnessing widely distributed and heterogeneous
     compute capacity.

     However, open membership introduces the risk that a single entity may
     create many identities to gain disproportionate influence or rewards.
     To mitigate this risk, ODSI relies on Sybil-resistance mechanisms
     implemented within the control plane, where Sybil resistance can only
     be achieved economically rather than administratively.  While
     identity creation is unrestricted, meaningful participation requires
     stake-backed commitment.  Stake ensures that influence, task volume,
     and rewards are proportional to economic risk and historical
     performance rather than identity count.

     This approach preserves openness while preventing adversaries from
     cheaply amplifying influence through large numbers of identities.
     Large-scale Sybil attacks require correspondingly large capital
     commitments and carry a high risk of economic loss.

  11.2.  Growth and Churn

     Participants in ODSI may join or leave the system at any time, either
     intentionally or due to failures, mobility, or network conditions.
     The framework assumes continuous churn and does not require long-
     lived availability from individual participants.

     Scalability under churn is achieved by:

     *  Decentralized participant discovery and coordination,

     *  Adaptive assignment of inference execution based on observed
        availability and performance,

     *  Conservative state migration and localized recovery when execution
        paths change.

     The inference plane prioritizes timely progress in the presence of
     transient failures, while the control plane provides longer-term
     stability by discouraging unreliable behavior through economic and
     reputational consequences.  Together, these mechanisms allow the
     system to scale with participation while remaining robust under
     dynamic conditions.





  Wang, et al.            Expires 3 September 2026               [Page 27]


  Internet-Draft                    ODSI                        March 2026


  11.3.  Interoperability and Extensibility

     ODSI is defined as an architectural framework rather than a single
     monolithic protocol.  It is intended to accommodate multiple protocol
     instantiations, execution environments, and incentive mechanisms.

     Interoperability is supported by:

     *  Clear separation between inference execution and verification
        functions,

     *  Well-defined interfaces between planes,

     *  Use of cryptographic primitives and message formats that can be
        standardized independently.

     Extensibility is a key design goal.  New execution strategies,
     verification techniques, or incentive schemes can be introduced
     without disrupting existing deployments, provided they respect the
     framework’s core principles.  This modularity allows ODSI to evolve
     alongside advances in inference techniques, hardware capabilities,
     and decentralized coordination mechanisms.

  12.  Privacy Considerations

     Inference execution in ODSI involves the transmission of user-
     provided inputs and intermediate activations across independently
     operated participants.  By default, the framework does not provide
     confidentiality guarantees for such data beyond transport-level
     protection.

     Decentralized execution may reduce the need to transmit user data to
     centralized endpoints and can enable computation to occur closer to
     data sources.  However, distributing execution across multiple
     participants may also increase the number of entities that observe
     portions of the execution state.

     Privacy risks include exposure of user inputs, partial activations,
     execution patterns, or metadata such as timing and routing
     information.  These risks vary depending on deployment context,
     participant selection policies, and execution strategies.

     ODSI is designed to be compatible with additional privacy-enhancing
     mechanisms, including data minimization, execution on trusted
     hardware, encrypted computation, or differential privacy techniques.
     Such mechanisms are considered out of scope for this document but may
     be incorporated by specific protocol instantiations or deployment
     profiles.



  Wang, et al.            Expires 3 September 2026               [Page 28]


  Internet-Draft                    ODSI                        March 2026


     Operators and users should carefully evaluate privacy requirements
     and select appropriate configurations and extensions when deploying
     ODSI in sensitive environments.

  13.  Relationship to Existing Work

     ODSI draws inspiration from multiple areas of distributed systems and
     decentralized computing, while addressing challenges that are
     specific to large-scale, interactive inference workloads.

     Centralized inference platforms provide tightly optimized execution
     but rely on trusted operators and centralized infrastructure.  ODSI
     departs from this model by enabling open participation and
     decentralized execution while preserving interactivity.

     Prior work on distributed machine learning focuses primarily on
     training or batch-oriented computation, where execution is less
     sensitive to strict per-step latency and state continuity.  In
     contrast, ODSI targets online inference with strong sequential
     dependencies and deadline constraints.

     Peer-to-peer and decentralized computation frameworks enable open
     resource contribution but typically assume best-effort execution or
     stateless tasks.  ODSI extends these ideas by incorporating stateful
     execution, deadline awareness, and economic accountability.

     Blockchain and decentralized ledger systems provide mechanisms for
     identity, verification, and incentive alignment, but are generally
     unsuitable for latency-critical execution.  ODSI adopts similar
     accountability principles while decoupling execution from
     verification to meet performance requirements.

     By integrating concepts from these domains, ODSI defines a distinct
     architectural approach for open, decentralized inference that
     complements rather than replaces existing systems.

  14.  Acknowledgments

     The authors would like to thank colleagues and reviewers in the
     community who provided feedback on the early version of this draft.

  15.  References

  15.1.  Normative References







  Wang, et al.            Expires 3 September 2026               [Page 29]


  Internet-Draft                    ODSI                        March 2026


     [RFC6979]  Pornin, T., "Deterministic Usage of the Digital Signature
                Algorithm (DSA) and Elliptic Curve Digital Signature
                Algorithm (ECDSA)", RFC 6979, DOI 10.17487/RFC6979, August
                2013, <https://www.rfc-editor.org/rfc/rfc6979>.

  15.2.  Informative References

     [Bitcoin]  Nakamoto, S., "Bitcoin: A peer-to-peer electronic cash
                system", 2008.

     [Byzantine]
                Castro, M. and B. Liskov, "Practical byzantine fault
                tolerance", OSDI, February 1999, <https://www.usenix.org/l
                egacy/publications/library/proceedings/osdi99/full_papers/
                castro/castro.ps>.

     [Lightning]
                Joseph Poon and T. Dryja, "The bitcoin lightning network:
                Scalable off-chain instant payments", 2008.

  Authors' Addresses

     Hanling Wang
     Pengcheng Laboratory
     Email: wanghl03@pcl.ac.cn


     Qing Li
     Pengcheng Laboratory
     Email: liq@pcl.ac.cn


     Yong Jiang
     Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
     Email: jiangy@sz.tsinghua.edu.cn


     Mingwei Xu
     Tsinghua University
     Email: xumw@tsinghua.edu.cn











  Wang, et al.            Expires 3 September 2026               [Page 30]


name: draft-li-cats-aisemantic-contract
rev: 00
title: Semantic-Driven Traffic Shaping Contract for AI Networks
time: 2026-03-02T10:02:44Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T10:02:44Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.306922+00:00
source: ietf
abstract:
  This document defines a "Semantic-Driven Shaping Contract".
     Traditional network protocols treat AI training and inference traffic
     as opaque byte streams, leading to highly inefficient scheduling.
     This contract allows applications or distributed training frameworks
     to explicitly pass "minimum necessary semantics" to the underlying
     network.  In exchange, the network commits to executing fine-grained,
     differentiated forwarding and resource allocation actions for tensor
     flows with diverse semantics, based on predefined rules and global
     real-time states.  This model significantly improves overall resource
     utilization and task completion times in heterogeneous computing
     networks, cross-domain intelligent computing centers, and integrated
     training-inference scenarios.

full_text:
  Computing-Aware Traffic Steering                                   Q. Li
  Internet-Draft                                                    T. gao
  Intended status: Standards Track                    Pengcheng Laboratory
  Expires: 2 September 2026                                       Y. Jiang
    Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
                                                              1 March 2026


          Semantic-Driven Traffic Shaping Contract for AI Networks
                    draft-li-cats-aisemantic-contract-00

  Abstract

     This document defines a "Semantic-Driven Shaping Contract".
     Traditional network protocols treat AI training and inference traffic
     as opaque byte streams, leading to highly inefficient scheduling.
     This contract allows applications or distributed training frameworks
     to explicitly pass "minimum necessary semantics" to the underlying
     network.  In exchange, the network commits to executing fine-grained,
     differentiated forwarding and resource allocation actions for tensor
     flows with diverse semantics, based on predefined rules and global
     real-time states.  This model significantly improves overall resource
     utilization and task completion times in heterogeneous computing
     networks, cross-domain intelligent computing centers, and integrated
     training-inference scenarios.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 2 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.




  Li, et al.              Expires 2 September 2026                [Page 1]


  Internet-Draft          Semantic Shaping Contract             March 2026


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Problem Statement: Limitations of Existing Network
             Mechanisms  . . . . . . . . . . . . . . . . . . . . . . .   2
     2.  Cross-Domain Amplification of Challenges  . . . . . . . . . .   3
     3.  The Semantic-Driven Mapping Loop: The Contract  . . . . . . .   3
       3.1.  Semantic Information Model (Metadata Model) . . . . . . .   3
       3.2.  Network Policy / Action Set . . . . . . . . . . . . . . .   4
     4.  Extended Use Case: Top-K Routing Semantics for MoE
             Architecture  . . . . . . . . . . . . . . . . . . . . . .   5
     5.  Deployment Considerations . . . . . . . . . . . . . . . . . .   5
       5.1.  Decision Location: Why In-Network?  . . . . . . . . . . .   5
       5.2.  RDMA / RoCEv2 Integration . . . . . . . . . . . . . . . .   5
     6.  Security Considerations . . . . . . . . . . . . . . . . . . .   5
     7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   6
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   6

  1.  Problem Statement: Limitations of Existing Network Mechanisms

     In the era of large AI models, the "importance" of traffic
     dynamically shifts with the model's phase and exhibits a high degree
     of computability.  Existing traffic control and Quality of Service
     (QoS) mechanisms suffer from fundamental flaws in this context:

     *  *Coarse QoS Granularity and Invalid Implicit Assumptions:*
        Traditional QoS assumes that traffic within the same class has
        negligible variance and its importance remains stable at the
        session level.  However, in AI scenarios, QoS fails to
        differentiate between "early-layer activations" and "late-layer
        activations," nor can it distinguish between the "KV Cache of
        early tokens" and "tail tokens."

     *  *Static and Incomputable DiffServ Semantics:* Differentiated
        Services (DiffServ/DSCP) rely on static markings that the network
        blindly executes.  It cannot express dynamic computing semantics,
        such as "this flow is quantizable during congestion," "this flow
        tolerates a 5ms store-and-forward delay," or "this flow requires
        absolute preemption."




  Li, et al.              Expires 2 September 2026                [Page 2]


  Internet-Draft          Semantic Shaping Contract             March 2026


     *  *Passive and Dimensionless ECN Feedback:* Existing Explicit
        Congestion Notification (ECN) mechanisms operate on the assumption
        that "end-systems know best how to respond to congestion" and that
        "rate reduction is the only correct response."  It possesses zero
        understanding of computing semantics, treating activations,
        gradients, blocking, and non-blocking operations equally.  In AI
        inference, the correct response to congestion is often "precision
        degradation (quantization)" or "prioritizing the draining of
        critical tensors," rather than blind rate reduction.

  2.  Cross-Domain Amplification of Challenges

     In cross-domain intelligent computing networks characterized by
     multi-tasking, multi-tenancy, and integrated training and inference,
     the aforementioned flaws are severely amplified:

     *  *Time-scale Mismatch:* Cross-domain Round-Trip Times (RTT) reach
        the millisecond level, easily exceeding the "effective value
        window" of sensitive tensors like late-layer activations.  The
        network MUST make differentiation and routing decisions
        instantaneously during forwarding; post-facto congestion control
        feedback is entirely ineffective.

     *  *Resource & Path Asymmetry:* Cross-domain links are scarce, high-
        cost resources.  Delay-tolerant and compressible intermediate
        activations absolutely MUST NOT compete equally for cross-domain
        bandwidth with critical gradients that require immediate delivery.

     *  *Tight Compute-Network Coupling:* Traffic steering is no longer
        merely about "delivery to a fixed destination."  It requires
        dynamic selection based on compute heterogeneity (e.g., local GPU
        vs. remote FPGA).  A lack of semantic understanding leads to a
        severe mismatch between computing power and network resources.

  3.  The Semantic-Driven Mapping Loop: The Contract

     The core of this draft is to establish a closed-loop mapping
     mechanism from "application-layer semantic input" to "network-side
     action commitment."

  3.1.  Semantic Information Model (Metadata Model)

     The application layer MUST expose "exchangeable Semantic Metadata" to
     the network.  Based on the commonalities and specifics of training
     and inference tasks, this is categorized as follows:






  Li, et al.              Expires 2 September 2026                [Page 3]


  Internet-Draft          Semantic Shaping Contract             March 2026


     *  *Traffic Class:* Explicitly identifies the data type (e.g.,
        Activation, Gradient, KV Cache, Parameter, Collaborative State
        Synchronization).

     *  *Urgency & Dependency:* Provides coarse-grained dependency hints
        (e.g., Early-token vs. Late-token) and the current layer or stage
        of the model (Layer ID / Pipeline Stage).

     *  *Tolerance & Sensitivity:*

        -  *Fidelity/Accuracy Sensitivity:* Indicates whether in-network
           low-precision quantization is permitted.

        -  *Loss/Latency Tolerance:* Indicates whether the flow permits
           buffering (store-and-forward) or dropping.

     *  *Compute Affinity:* Indicates the preferred characteristics of the
        underlying computing power (e.g., GPU, FPGA, CPU, or specific
        operator acceleration hardware).

  3.2.  Network Policy / Action Set

     Upon receiving the aforementioned semantics, network nodes with
     global state awareness can execute a set of policies that transcend
     traditional routing:

     *  *Queueing / Scheduling:* Identifies flow states to guarantee
        absolute preemption for highly time-sensitive traffic.

     *  *Buffering / Store-and-forward:* Utilizes the storage resources of
        network devices to temporarily delay flows with high latency
        tolerance (e.g., large-block parameter pulls); it also implements
        cache multiplexing for inference requests from different users,
        directly optimizing hardware throughput without altering the model
        structure.

     *  *Shaping & In-network Quantization:* Triggers in-network low-
        precision quantization and sparsity strategies during congestion,
        rather than relying on simple packet dropping.

     *  *Steering:* Intelligently guides task flows to the most
        appropriate heterogeneous computing nodes based on Compute
        Affinity.








  Li, et al.              Expires 2 September 2026                [Page 4]


  Internet-Draft          Semantic Shaping Contract             March 2026


  4.  Extended Use Case: Top-K Routing Semantics for MoE Architecture

     For dynamic computing architectures like Mixture-of-Experts (MoE),
     this contract supports the definition of more complex routing
     metadata for intelligent scheduling in the network data plane:

     *  *Model Router Metadata:* Carries Token ID / Query vector
        summaries, Top-K candidate expert lists, weights/confidence
        levels, and positional markers (Token_pos).

     *  *System State Semantics:* Network nodes maintain real-time metrics
        for each expert node, including backlog queues, computing load,
        network latency, and bandwidth utilization.

     By matching these two semantics, the network can instantaneously
     determine which Expert node with the lightest load should receive the
     Token flow at the moment of forwarding.

  5.  Deployment Considerations

  5.1.  Decision Location: Why In-Network?

     Compared to edge devices (GPUs/NICs) that only possess local queuing
     information, in-network nodes (e.g., Core/Spine Switches) maintain a
     global perspective.  The network can perceive concurrent multi-tenant
     tasks and real-time multipath congestion states.  Crucially, it can
     make immediate decisions to buffer, slice, or reroute cross-domain
     traffic before it enters high-cost bottleneck links.

  5.2.  RDMA / RoCEv2 Integration

     Intelligent computing centers rely heavily on RDMA.  The Semantic
     Header defined in this contract will be designed as Extension Headers
     for RoCEv2/UDP packets, or carried using specific reserved fields.
     This enables supporting hardware (such as the FPGA and parsing
     pipelines in the IntelliNode architecture) to extract metadata and
     execute policies at line rate (e.g., 400Gbps).

  6.  Security Considerations

     To ensure the integrity of the Semantic-Driven Shaping Contract, the
     system MUST:

     *  *Authentication and Anti-Spoofing:* Prevent malicious tenants from
        tampering with the Urgency level or forging network states to
        unfairly preempt high-priority queues.





  Li, et al.              Expires 2 September 2026                [Page 5]


  Internet-Draft          Semantic Shaping Contract             March 2026


  7.  IANA Considerations

     This document requests that IANA allocate specific protocol numbers
     or RoCEv2 option type spaces for the AI Semantic Header to facilitate
     standardized deployment.

  Authors' Addresses

     Qing Li
     Pengcheng Laboratory
     Email: liq@pcl.ac.cn


     Teng gao
     Pengcheng Laboratory
     Email: gaot@pcl.ac.cn


     Yong Jiang
     Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
     Email: jiangy@sz.tsinghua.edu.cn






























  Li, et al.              Expires 2 September 2026                [Page 6]


name: draft-li-individual-inip
rev: 00
title: In-Network Inference Protocol
time: 2026-03-02T13:54:34Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T13:54:34Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.310407+00:00
source: ietf
abstract:
  This document specifies the In-Network Inference Protocol (INIP), a
     lightweight protocol designed specifically for implementing high-
     speed in-network inference in data center internal networks.  INIP
     utilizes data plane devices (such as switches, DPUs, and SmartNICs)
     to perform lightweight inference tasks while ensuring that core
     network forwarding functions are not affected.  The protocol operates
     based on the IPv4 protocol and adopts a fixed, lightweight packet
     format.

     INIP adopts a two-tier architecture of "centralized control plane
     adaptation and scheduling, and minimal data plane execution".  The
     control plane stores all inference models, deploys model rules to
     data plane devices using a CDN-like scheduling method, and assumes
     the responsibility of degraded fallback inference; the data plane
     performs packet parsing and match action table-based inference.  This
     document details INIP's core logic, packet format, data plane device
     constraints, model expression specifications, control plane
     responsibilities, CDN-like scheduling mechanism, dynamic model
     popularity replacement, and overall execution process.

full_text:
  Individual Submission                                              Q. Li
  Internet-Draft                                                    G. Xie
  Intended status: Experimental                       Pengcheng Laboratory
  Expires: 3 September 2026                                       Y. Jiang
                           Tsinghua Shenzhen International Graduate School
                                                                     M. Xu
                                                       Tsinghua University
                                                              2 March 2026


                       In-Network Inference Protocol
                        draft-li-individual-inip-00

  Abstract

     This document specifies the In-Network Inference Protocol (INIP), a
     lightweight protocol designed specifically for implementing high-
     speed in-network inference in data center internal networks.  INIP
     utilizes data plane devices (such as switches, DPUs, and SmartNICs)
     to perform lightweight inference tasks while ensuring that core
     network forwarding functions are not affected.  The protocol operates
     based on the IPv4 protocol and adopts a fixed, lightweight packet
     format.

     INIP adopts a two-tier architecture of "centralized control plane
     adaptation and scheduling, and minimal data plane execution".  The
     control plane stores all inference models, deploys model rules to
     data plane devices using a CDN-like scheduling method, and assumes
     the responsibility of degraded fallback inference; the data plane
     performs packet parsing and match action table-based inference.  This
     document details INIP's core logic, packet format, data plane device
     constraints, model expression specifications, control plane
     responsibilities, CDN-like scheduling mechanism, dynamic model
     popularity replacement, and overall execution process.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.







  Li, et al.              Expires 3 September 2026                [Page 1]


  Internet-Draft                    INIP                        March 2026


     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 3 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
       1.1.  Background  . . . . . . . . . . . . . . . . . . . . . . .   3
       1.2.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . .   3
       1.3.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   3
       1.4.  Requirements Language . . . . . . . . . . . . . . . . . .   5
     2.  Core Logic Overview . . . . . . . . . . . . . . . . . . . . .   5
     3.  INIP Packet Format  . . . . . . . . . . . . . . . . . . . . .   6
     4.  Data Plane Device Constraints . . . . . . . . . . . . . . . .   7
       4.1.  Heterogeneous Device Support  . . . . . . . . . . . . . .   7
       4.2.  Data Plane Inference Table Structure  . . . . . . . . . .   8
       4.3.  Fixed Inference Parameters  . . . . . . . . . . . . . . .   8
       4.4.  Data Plane Execution Logic  . . . . . . . . . . . . . . .   9
     5.  Model Expression Specifications . . . . . . . . . . . . . . .  10
     6.  Core Responsibilities of the Control Plane  . . . . . . . . .  10
       6.1.  Data Plane Device Capability Management . . . . . . . . .  10
       6.2.  Model Logic Conversion  . . . . . . . . . . . . . . . . .  11
       6.3.  Rule Issuance Verification  . . . . . . . . . . . . . . .  11
       6.4.  CDN-like Scheduling and Model Popularity Management . . .  11
         6.4.1.  CDN-like Scheduling Principles  . . . . . . . . . . .  11
         6.4.2.  Model Popularity Considerations . . . . . . . . . . .  11
         6.4.3.  Model Resource Reuse  . . . . . . . . . . . . . . . .  12
       6.5.  Degraded Fallback Inference . . . . . . . . . . . . . . .  12
       6.6.  Responding to User Requests . . . . . . . . . . . . . . .  13
     7.  Overall Process . . . . . . . . . . . . . . . . . . . . . . .  13
     8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  15
     9.  Security Considerations . . . . . . . . . . . . . . . . . . .  15
     10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  15
       10.1.  Normative References . . . . . . . . . . . . . . . . . .  15



  Li, et al.              Expires 3 September 2026                [Page 2]


  Internet-Draft                    INIP                        March 2026


       10.2.  Informative References . . . . . . . . . . . . . . . . .  16
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  16

  1.  Introduction

  1.1.  Background

     With the rapid development of data center networks and artificial
     intelligence, the demand for low-latency and high-throughput
     inference services within data centers continues to grow.  Typical
     scenarios include real-time traffic classification, malicious traffic
     detection, as well as lightweight computer vision/natural language
     applications (rapid product defect detection, simple face
     recognition, short text sentiment polarity judgment, device control
     command recognition, etc.).

     Traditional inference deployment (deploying inference services on
     dedicated servers) can not use existing data plane resources for
     green networking and often introduces additional transmission
     latency.  To address this, this document proposes the In-Network
     Inference Protocol (INIP).  This protocol enables data plane devices
     to undertake fast inference tasks while ensuring their forwarding
     function.  INIP makes the control plane store all models, implements
     in-network deployment of model rules using a CDN-like scheduling
     method, simplifies the adaptation of models to heterogeneous data
     plane devices through model distillation and cross-matching mode
     conversion, and optimizes data plane utilization through dynamic
     replacement based on model popularity.  The INIP control plane also
     assumes the responsibility of fallback inference to ensure the normal
     execution of inference tasks.

  1.2.  Scope

     This document defines INIP's core logic, packet format, data plane
     constraints, model expression specifications, control plane
     responsibilities, and overall process.  The protocol is applicable to
     data center internal network environments, focusing on real-time,
     high-throughput inference scenarios that avoid large models with
     heavy computing.  INIP does not define a new network layer protocol
     but operates based on the IPv4 protocol [RFC791], and model query
     messages initiated by users to the control plane are implemented
     using the lightweight DNS protocol [RFC1035] to ensure standardized
     adaptation.

  1.3.  Terminology

     The key terms used in this document are defined as follows:




  Li, et al.              Expires 3 September 2026                [Page 3]


  Internet-Draft                    INIP                        March 2026


     Data Plane Device:
        A network device (such as a Tofino switch [TOFINO], DPU, or
        SmartNIC) that undertakes packet forwarding.  A general inference
        table may be reserved during P4 [P4] code compilation (depending
        on resource availability) to execute user inference.

     Inference Table
        A general table reserved on a data plane device, populated with
        inference rules issued by the control plane.  It adopts a "key
        field + action field" structure to perform fast match-based
        inference.

     Feature
        A core component of the key fields in the inference table, i.e.,
        the feature data required for model inference.  It corresponds
        one-to-one with the features in the INIP packet feature area,
        arranged with a fixed bit width, and serves as the core basis for
        the data plane to perform inference (table lookup and matching).

     Matching Mode
        The matching method for sample features during the operation of
        the data plane device's inference table, including exact matching,
        range matching, and ternary matching [P4].  It is determined by
        the remaining hardware resources after allocation to network
        functions.

     Control Plane
        The core scheduling and adaptation node of INIP, responsible for
        data plane device capability management, model logic conversion,
        rule issuance, CDN-like scheduling, model popularity statistics,
        dynamic model replacement, and degraded fallback inference.  It
        also acts as a DNS server to receive and respond to model query
        messages sent by users.

     CDN-like Scheduling
        A scheduling method that determines the deployment location of
        models based on topological proximity, load balancing, data plane
        device capability, and model popularity.

     Model Popularity
        An indicator measuring the frequency of model queries (only
        counting model queries initiated by users to the control plane via
        the DNS protocol), used to determine whether a model is high-
        frequency or low-frequency, and to provide a basis for the dynamic
        replacement of data plane models.

     General Decision Tree




  Li, et al.              Expires 3 September 2026                [Page 4]


  Internet-Draft                    INIP                        March 2026


        A unified model format obtained by distilling various original
        models (deep learning models, traditional machine learning models)
        [DTDISTILL].

     Fallback Inference
        A degradation scheme where the control plane independently
        executes inference tasks to ensure business continuity when there
        are no suitable data plane devices.

  1.4.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in
     BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  2.  Core Logic Overview

     The core design goal of INIP is to use data plane devices to enable
     lightweight, fast inference in data center internal networks while
     ensuring that core network forwarding functions (such as routing,
     ACL, firewall) are not compromised.  The protocol operates based on
     the IPv4 protocol [RFC791], with INIP packets encapsulated in IPv4
     packets for transmission.

     INIP adopts a two-tier architecture, with the core logic as follows:

     1.  Control Plane: Centrally manages all inference models (including
         original models and their distilled general decision trees)
         [DTDISTILL] [DLDISTILL], undertaking core responsibilities such
         as model logic conversion after distillation, inference rule
         issuance, device capability management, CDN-like scheduling,
         model popularity statistics and dynamic replacement, and fallback
         inference.  It also receives two types of messages sent by
         users—model queries (implemented using the DNS protocol [RFC1035]
         to obtain the model's data plane deployment location) and INIP-
         format inference requests (to obtain fallback inference results).
         The control plane acts as a DNS server to parse users' DNS query
         requests and return the corresponding IP addresses of data plane
         devices that hold the specific in-network inference tasks.










  Li, et al.              Expires 3 September 2026                [Page 5]


  Internet-Draft                    INIP                        March 2026


     2.  Data Plane: Performs normal network functions and can receive
         model inference rules issued by the control plane.  However, it
         only executes "packet parsing + matching-based inference" when
         its own IP is the destination IP of the INIP packet; after a
         successful inference, it needs to modify the inference result
         field and swap the source and destination IP pairs to ensure the
         packet is returned to the user.

     3.  CDN-like Deployment and Popularity Adaptation: Based on CDN-like
         scheduling logic and combined with model popularity, the control
         plane prioritizes deploying general decision tree model rules for
         high-frequency queries to nearby, suitable data plane devices;
         the same model can be deployed on multiple data plane devices to
         achieve multi-node redundant deployment/load balancing, and
         optimize the utilization of the data plane inference table
         through dynamic popularity replacement.

     4.  Fallback Mechanism: When there are no suitable data plane
         devices, no corresponding model rules are deployed on the data
         plane, or an interruption occurs during model rule replacement,
         the control plane automatically executes fallback inference.

     5.  Request-Result Association: By setting a request identifier in
         the INIP packet, a one-to-one correspondence between inference
         requests and return results is achieved, ensuring that the user
         can clearly distinguish the inference results of different
         requests.

  3.  INIP Packet Format

     An INIP packet consists of a fixed-length header (64 bits) and a
     variable-length feature area.  The following is a visual
     representation of the INIP packet format (excluding the IPv4 header):

         0               1               2               3
         0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |        Model ID (16bit)       |      Request ID (16bit)     |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         | Inference Res. | Feature Cnt. |                             |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                             +
         |                                                             |
         |              Feature Area (Variable Length)                 |
         |                                                             |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Detailed descriptions of each INIP packet field (all 8 bits) are as
     follows:



  Li, et al.              Expires 3 September 2026                [Page 6]


  Internet-Draft                    INIP                        March 2026


     *  *Model ID (16 bits)*: Uniquely identifies the inference model
        requested by the user, assigned by the control plane, and
        corresponds one-to-one with the model rules issued by the control
        plane to the data plane.  It is the core index for users to
        initiate inference requests and for the data plane to perform
        matching-based inference; when users initiate a model query via
        the DNS, they need to use this model ID as the query key in the
        format "[Model ID].inip.local" (a local domain, resolvable only
        within the data center).

     *  *Request ID (16 bits)*: generated and carried by the user when
        initiating an inference request, which must be unique within a
        certain period.  When the data plane or control plane returns the
        inference result, it must carry this field unchanged to ensure
        that the user can associate the returned inference result with the
        corresponding request.

     *  *Inference Result (8 bits)*: This field is valid when the Most
        Significant Bit (MSB) is set to 1; when valid, it carries the
        inference result returned to the user by the data plane device or
        control plane.  When the MSB is 0, this field is a reserved bit,
        and the sender (user) must set it to 0, while the receiver (data
        plane/control plane) ignores this field.  After a successful
        inference, the MSB of this field must be set to 1, and the
        corresponding inference result must be filled in.

     *  *Feature Count (8 bits)*: Indicates the total number of features
        (N) carried by the inference request, informing the data plane
        device of the number of features to be parsed.

     *  *Feature Area (Variable Length)*: Stores N feature data required
        for inference.  Each feature adopts a globally fixed bit width
        (e.g., 32 bits, uniformly specified by the control plane).

  4.  Data Plane Device Constraints

  4.1.  Heterogeneous Device Support

     INIP supports heterogeneous data plane devices.  Regardless of the
     physical type (switch, DPU, smartNIC), manufacturer, or hardware
     specifications of the device, all data plane devices can reserve a
     general inference table (optional, only considered after meeting
     normal networking functions).  Inference operations are completed
     through this single inference table, and hardware differences between
     devices do not affect protocol execution.






  Li, et al.              Expires 3 September 2026                [Page 7]


  Internet-Draft                    INIP                        March 2026


  4.2.  Data Plane Inference Table Structure

     Data plane devices only reserve one general inference table, which is
     populated by the control plane with inference rules converted from
     general decision trees.  The structure is simple and fixed, adapting
     to the data plane's line-rate table lookup requirements.  The
     inference table adopts a fixed "key field + action field" structure:

     *  *Key Fields*: Core matching fields, fixedly including two types of
        fields: ① Model ID (8 bits): Exactly the same as the Model ID
        field in the INIP packet, used to locate the model rule
        corresponding to the current inference; ② Inference Features (n
        fixed-bit-width features): Corresponding to the number of
        resolvable features n of the data plane, and features beyond the N
        features required by the model should be regarded as meaningless;
        the bit width of each feature is consistent with the INIP packet
        feature area (uniformly specified by the control plane).

     *  *Action Fields*: Action instructions executed after a successful
        table lookup, fixedly including one type of core field, namely the
        Inference Result (8 bits): the inference conclusion corresponding
        to the set of key fields (Model ID + Features), used to fill the
        INIP packet.

  4.3.  Fixed Inference Parameters

     The fixed inference-related parameters of the inference table
     include:

     *  *Resolvable Feature Count (n)*: The number of fixed-bit-width
        features that the device can resolve.  The bit width of all
        features is globally uniform, which needs to adapt to the value
        range (0-255) of the feature count (8 bits).  The control plane
        will select suitable devices to deploy model rules based on this
        parameter, ensuring that n ≥ the number of features N required by
        the model.

     *  *Inference Table Matching Mode*: The matching mode (exact
        matching, range matching, ternary matching) of each key field in
        the inference table, determined by the allocation of the device's
        hardware resources.  For example, when the TCAM resources of the
        Tofino chip are prioritized for core network functions and SRAM
        has redundancy, the inference table will adopt an SRAM-based exact
        matching mode.

     *  *Maximum Number of Inference Table Entries*: The maximum number of
        inference rules that the device can carry, determined by the
        hardware resources of the inference table (such as SRAM capacity).



  Li, et al.              Expires 3 September 2026                [Page 8]


  Internet-Draft                    INIP                        March 2026


     The inference-related parameters of data plane devices are globally
     fixed after P4 code compilation [P4].  If inference-related
     parameters need to be adjusted, the control plane can modify and
     recompile the P4 code as needed, and synchronously update the
     corresponding data plane device information in the capability
     database to ensure the accuracy of subsequent model replacement and
     adaptation.

  4.4.  Data Plane Execution Logic

     Only the destination of the INIP packet (which may be a data plane
     device, control plane, or user) will execute "packet parsing + table-
     based inference (if it is a data plane/control plane) + post-hit
     processing" operations; network devices with non-destination IPs
     (including non-destination data plane devices) only perform behaviors
     consistent with network forwarding logic.  When the destination is a
     data plane device, the specific execution logic is as follows:

     1.  Packet Parsing: Parse the INIP packet encapsulated in the IPv4
         packet, extract the Model ID, Request ID, Feature Count N, and N
         inference features, and record the source IP address and
         destination IP address;

     2.  Matching-based Inference: Perform table lookup operations
         according to the inference table matching rules based on the
         extracted Model ID and inference features, distinguishing between
         hit and miss scenarios;

     3.  Miss Handling: If there is no hit in the inference table, forward
         the complete packet to the control plane (while modifying the
         destination IP in the IPv4 header to the control plane IP), and
         the control plane executes fallback inference;

     4.  Post-hit Processing (Core Operation): If there is a hit in the
         inference table, perform two key operations: ① Modify the
         "Inference Result" field of the INIP packet: Set the Most
         Significant Bit (MSB) of this field to 1 to identify that the
         inference result is valid, and fill the matched inference result
         from the inference table into this field; ② Adjust the IPv4
         header: Swap the source IP address and destination IP address,
         i.e., change the original source IP (user IP) in the IPv4 header
         to the destination IP, and the original destination IP (data
         plane device IP) to the source IP, ensuring that the data packet
         can be returned to the requesting user along the original path;

     5.  Packet Forwarding: Send the modified IPv4 packet according to the
         networking forwarding logic.




  Li, et al.              Expires 3 September 2026                [Page 9]


  Internet-Draft                    INIP                        March 2026


  5.  Model Expression Specifications

     All models in INIP are represented in the form of general decision
     trees, which are obtained by distilling various original models
     (including deep learning models and traditional machine learning
     models) through knowledge distillation technology [DTDISTILL].  This
     ensures the unity and adaptability of model expression, facilitating
     the control plane to convert them into inference rules supported by
     data plane devices.

     A general decision tree only defines the business logic judgments
     required for inference (such as feature interval judgment, equality
     judgment) and is not bound to the matching mode of any data plane
     device.  This allows models from different sources and types to be
     flexibly adapted to devices with different matching modes after
     distillation.  The control plane stores all original models and their
     corresponding general decision trees, while the data plane only
     stores the inference table rules converted by the control plane; the
     control plane uniformly assigns Model IDs to ensure their uniqueness,
     and this Model ID will serve as the core key for users to initiate
     queries.

  6.  Core Responsibilities of the Control Plane

     The control plane is the core adaptation and scheduling node of INIP,
     undertaking the following key responsibilities, including core
     functions such as CDN-like scheduling, model popularity management,
     and fallback inference, to ensure the efficient operation of the
     protocol:

  6.1.  Data Plane Device Capability Management

     The control plane maintains a dynamically updatable device capability
     database, which records the fixed parameters (n, matching mode,
     maximum number of entries), topological location, and remaining
     inference table capacity of each data plane device.  After a data
     plane device deploys or deletes model rules, or updates inference
     parameters, the control plane updates the corresponding device
     information to support CDN-like scheduling and model popularity
     replacement.











  Li, et al.              Expires 3 September 2026               [Page 10]


  Internet-Draft                    INIP                        March 2026


  6.2.  Model Logic Conversion

     The control plane converts the business logic of the general decision
     tree into inference rules according to the matching mode of the
     target data plane device.  It also assigns a unique Model ID to each
     model, associating the model with the inference rules.  The
     conversion process must follow the complete logical equivalence
     principle and allow cross-matching mode conversion (such as
     converting the range logic of the decision tree into exact matching
     or ternary matching rules) to ensure the accuracy of the inference
     results; the converted rules must adapt to the "key field + action
     field" structure of the data plane inference table.

  6.3.  Rule Issuance Verification

     Inference rules will be issued to the target data plane device only
     if both of the following conditions are met:

     1.  The number of resolvable features n of the device ≥ the number of
         features N required by the model; the control plane will only
         issue the corresponding model rules to devices that meet this
         condition;

     2.  The number of rules generated after conversion does not exceed
         the remaining number of entries in the device's inference table.

  6.4.  CDN-like Scheduling and Model Popularity Management

  6.4.1.  CDN-like Scheduling Principles

     During scheduling, the four-dimensional principles of "topological
     proximity + load balancing + capability adaptability + popularity
     priority" are followed.  Priority is given to selecting devices that
     are close to the user, have low load, and meet the adaptation
     conditions (n≥N, number of rules ≤ remaining capacity) to deploy
     high-frequency popular model rules to data plane devices; the same
     model can be deployed on multiple suitable data plane devices to
     achieve near-user response and load sharing of inference requests,
     consistent with the core logic of CDN.

  6.4.2.  Model Popularity Considerations

     The control plane statistics the model query messages initiated by
     users to it (only counting model query messages sent by users, not
     INIP-format inference requests sent by users), and determines the
     model popularity level and replacement rules in combination with data
     plane device constraints.  The specific consideration dimensions are
     as follows:



  Li, et al.              Expires 3 September 2026               [Page 11]


  Internet-Draft                    INIP                        March 2026


     *  *Core Consideration Indicators*: ① Model Query Frequency: Count
        the number of queries for each model within a fixed period (e.g.,
        1 hour) to distinguish between high-frequency (popular) and low-
        frequency models (example: ≥1000 queries per hour are high-
        frequency, <1000 queries per hour are low-frequency); ② Request
        Latency Requirement: Prioritize high-frequency models that are
        "latency-sensitive" (such as device control command recognition,
        real-time traffic classification), which need to be deployed to
        the data plane first; ③ Model Resource Occupation: Consider the
        number of data plane inference table entries occupied by model
        rules, and prioritize models with "high popularity + low resource
        occupation" to improve resource reuse rate.

     *  *Popularity Judgment and Dynamic Replacement Rules*: The control
        plane maintains a "model popularity ranking", which is updated
        every fixed period.  The core scenario triggering dynamic
        replacement is "low-frequency models occupying resources while
        high-frequency models have no deployment space".  During
        replacement, the principles of "high-frequency replacing low-
        frequency, low-resource-occupation replacing high-resource-
        occupation, and latency-sensitive models being retained first" are
        followed; if an inference request interruption occurs during
        replacement, the control plane automatically executes fallback
        inference to ensure business continuity; after the replacement is
        completed, inference requests are automatically switched to data
        plane response.

  6.4.3.  Model Resource Reuse

     If a data plane device still has remaining capacity in the inference
     table after deploying one model, it can continue to deploy other
     models, prioritizing high-frequency popular models to achieve
     efficient reuse of device resources; the same model can be deployed
     on multiple suitable data plane devices to further improve near-user
     response efficiency, achieve load sharing, and reduce the fallback
     inference pressure on the control plane.

  6.5.  Degraded Fallback Inference

     If there are no suitable data plane devices (e.g., no devices meet
     n≥N or the number of rules exceeds the limit), no corresponding model
     rules are deployed on the data plane (no corresponding Model ID), or
     an interruption occurs during model rule replacement, the control
     plane receives the INIP-format inference request message sent by the
     user and independently executes degraded fallback inference to ensure
     the inference task is not interrupted.  Fallback inference has no
     data plane hardware acceleration, and is only used as a degradation
     scheme, not replacing the deployment of high-frequency models on the



  Li, et al.              Expires 3 September 2026               [Page 12]


  Internet-Draft                    INIP                        March 2026


     data plane.  When the control plane returns the inference result, it
     must carry the Request ID and Model ID from the packet unchanged, set
     the MSB of the Inference Result field to 1, fill in the inference
     result, and swap the source IP and destination IP to ensure the
     packet is returned to the user.

  6.6.  Responding to User Requests

     The control plane acts as a DNS server to respond to model query
     messages sent by users via the DNS protocol [RFC1035].  The message
     uses "[Model ID].inip.local" (a local domain, resolvable only within
     the data center) as the query key, carrying the Model ID; if the
     model has multiple data plane deployment points (i.e., rules have
     been successfully issued to multiple suitable data plane devices),
     only the IP address of the data plane device closest to the user's
     topological location is returned; if the model has no data plane
     deployment points (no suitable data plane devices or no rules
     issued), the control plane's own IP address is returned.  After
     obtaining the IP address, the user can cache the mapping relationship
     between the Model ID and the corresponding IP address locally, with
     the cache validity period uniformly configured by the control plane
     (set via the DNS TTL field).  During the cache validity period, the
     user does not need to repeatedly send model query messages and can
     directly send corresponding messages based on the cached IP address:
     send INIP-format inference request messages to data plane devices,
     and send INIP-format inference request messages to the control plane
     (for fallback inference); after the cache expires, the user resends a
     DNS model query message to the control plane.

  7.  Overall Process

     The overall execution process of the INIP steps is as follows:

     1.   The control plane compiles P4 code for each data plane device
          (the code first includes network forwarding behaviors and an
          inference table when device resources are sufficient); when
          there is an inference table, it updates the corresponding
          information in the device capability database.  If the inference
          table parameters (number of features, table capacity) need to be
          adjusted later, the control plane must recompile the P4 code and
          update the device capability database.

     2.   The control plane distills and converts the original learning
          models into general decision trees, assigns a unique Model ID to
          the model, and stores all original models, corresponding general
          decision trees, and Model IDs; it also configures DNS services
          to support users to initiate queries with "[Model
          ID].inip.local" as the key.



  Li, et al.              Expires 3 September 2026               [Page 13]


  Internet-Draft                    INIP                        March 2026


     3.   The control plane parses the general decision tree, and screens
          out multiple suitable target data plane devices (the same model
          can be deployed on multiple suitable devices) in combination
          with CDN-like scheduling logic (topological proximity + load
          balancing + capability adaptability + popularity priority).

     4.   The control plane converts the decision tree logic into
          inference rules supported by the target device in an equivalent
          manner (i.e., adapting to the "key field + action field"
          structure of the data plane inference table), and checks whether
          the number of rules exceeds the limit based on the current
          remaining capacity of the device's inference table; if the
          remaining capacity is sufficient, completes the rule issuance,
          and updates the remaining capacity of the device in the device
          capability database, as well as the deployed model rules and
          Model ID information.

     5.   The control plane statistics the frequency of user DNS model
          query messages corresponding to each Model ID, updates the model
          popularity ranking every fixed period; for high-frequency
          popular models, they can be deployed on multiple suitable data
          plane devices.  If dynamic replacement of data plane model rules
          is needed, the replacement operation is performed.  During the
          replacement, the control plane receives the INIP-format
          inference request message sent by the user and executes fallback
          inference.

     6.   The user sends a model query message to the control plane via
          the DNS protocol, using "[Model ID].inip.local" as the query key
          to obtain the corresponding IP address: if the model has
          multiple data plane deployment points, the control plane returns
          the IP address of the data plane device closest to the user; if
          there are no data plane deployment points, the control plane
          returns its own IP.  After obtaining the IP address, the user
          caches the mapping relationship between the Model ID and the IP
          address locally (the cache validity period is controlled by the
          DNS TTL field).  During the cache validity period, the user
          directly sends an INIP inference request message to the
          corresponding node based on the cached IP address; after the
          cache expires, the user resends a DNS model query message to the
          control plane.

     7.   Only the destination of the INIP packet (data plane device,
          control plane, or user) parses the packet; network devices with
          non-destination IPs only forward the packet normally, performing
          behaviors consistent with network logic.





  Li, et al.              Expires 3 September 2026               [Page 14]


  Internet-Draft                    INIP                        March 2026


     8.   If the destination is a data plane device, extract the Model ID,
          Request ID, and feature data, and perform table-based inference;
          if there is a hit in the inference table, modify the Inference
          Result field (set MSB to 1 + fill in the inference result), swap
          the source and destination IP pairs in the IPv4 header, and
          forward the packet back to the user; if there is no hit in the
          inference table, forward the original INIP packet to the control
          plane for degraded fallback inference (the packet destination is
          modified to the control plane at this time).

     9.   As a possible destination IP of the INIP packet, the control
          plane receives the INIP-format inference request message sent by
          the user, executes fallback inference, sets the MSB of the
          Classification Result field to 1, fills in the inference result,
          swaps the source and destination IP pairs in the IPv4 header,
          carries the Request ID and Model ID unchanged, and forwards the
          packet back to the user.

     10.  When the user is the destination IP, parse and receive the
          inference result; network devices with non-destination IPs only
          forward the INIP packet normally.

  8.  IANA Considerations

     This document does not require any resource allocation from the
     Internet Assigned Numbers Authority (IANA).  INIP reuses the private
     allocation value of the IPv4 Protocol field (range 128-255), which is
     a private use range reserved by IANA, and no IANA public allocation
     number is required; when users initiate model queries via the DNS
     protocol, the local domain "inip.local" (non-public domain) is used,
     which does not require domain name allocation from IANA and only
     needs to be configured for resolution within the data center.

  9.  Security Considerations

     INIP is specifically designed for data center internal network
     environments.  Users should ensure that the environment has security
     protection measures such as physical isolation and access control.
     Therefore, INIP packets are transmitted in plain text to prioritize
     low latency processing performance.  For model query messages
     initiated by users via the DNS protocol, access control can be
     implemented through the internal DNS server of the data center to
     restrict only authorized users from initiating queries, preventing
     the leakage of Model IDs and device IP information.

  10.  References

  10.1.  Normative References



  Li, et al.              Expires 3 September 2026               [Page 15]


  Internet-Draft                    INIP                        March 2026


     [RFC791]   Postel, J., "Internet Protocol", STD 5, RFC 791,
                DOI 10.17487/RFC0791, September 1981,
                <https://www.rfc-editor.org/rfc/rfc791>.

     [RFC1035]  Mockapetris, P., "Domain names - implementation and
                specification", STD 13, RFC 1035, DOI 10.17487/RFC1035,
                November 1987, <https://www.rfc-editor.org/rfc/rfc1035>.

     [DTDISTILL]
                Xie, G., Li, Q., Dong, Y., Duan, G., Jiang, Y., and J.
                Duan, "Mousika: Enable General In-Network Intelligence in
                Programmable Switches by Knowledge Distillation",
                INFOCOM IEEE Conference on Computer Communications,
                DOI 10.1109/INFOCOM48880.2022.9796856, 2022,
                <https://doi.org/10.1109/INFOCOM48880.2022.9796856>.

     [P4]       The P4 Language Consortium, "P4_16 Language
                Specification", Version 1.2.5, 2024.

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/rfc/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

  10.2.  Informative References

     [TOFINO]   Intel/Barefoot Networks, "Tofino™ Programmable Switch Chip
                Datasheet", 2018.

     [DLDISTILL]
                Hinton, G., Vinyals, O., and J. Dean, "Distilling the
                Knowledge in a Neural Network", NIPS Deep Learning
                Workshop, 2015.

  Authors' Addresses

     Qing Li
     Pengcheng Laboratory
     Email: liq@pcl.ac.cn


     Guorui Xie
     Pengcheng Laboratory
     Email: xgr19@tsinghua.org.cn



  Li, et al.              Expires 3 September 2026               [Page 16]


  Internet-Draft                    INIP                        March 2026


     Yong Jiang
     Tsinghua Shenzhen International Graduate School
     Email: jiangy@sz.tsinghua.edu.cn


     Mingwei Xu
     Tsinghua University
     Email: xumw@tsinghua.edu.cn











































  Li, et al.              Expires 3 September 2026               [Page 17]


name: draft-xu-sidrops-asrank-vulnerabilities
rev: 00
title: Structural Vulnerabilities in ASRank under Adversarial Conditions
time: 2026-03-02T20:32:50Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T20:32:50Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.314512+00:00
source: ietf
abstract:
  This document analyzes the structural vulnerabilities of ASRank, a
     widely used algorithm for inferring Autonomous System (AS) business
     relationships from BGP routing data.  ASRank plays a key role in
     security research and BGP operation, yet its inference process is
     highly sensitive to small changes in input data.  This sensitivity
     introduces risks in adversarial conditions, where inference results
     may be manipulated without detection.  This document outlines the
     design of ASRank, identifies its structural vulnerabilities, analyzes
     a minimal manipulation example, and discusses the security
     implications and potential countermeasures.

full_text:
  SIDROPS Working Group                                              Y. Xu
  Internet-Draft                                                   Y. Chen
  Intended status: Informational                                     K. Xu
  Expires: 3 September 2026                                          Q. Li
                                                                     J. Wu
                                                       Tsinghua University
                                                              2 March 2026


     Structural Vulnerabilities in ASRank under Adversarial Conditions
                 draft-xu-sidrops-asrank-vulnerabilities-00

  Abstract

     This document analyzes the structural vulnerabilities of ASRank, a
     widely used algorithm for inferring Autonomous System (AS) business
     relationships from BGP routing data.  ASRank plays a key role in
     security research and BGP operation, yet its inference process is
     highly sensitive to small changes in input data.  This sensitivity
     introduces risks in adversarial conditions, where inference results
     may be manipulated without detection.  This document outlines the
     design of ASRank, identifies its structural vulnerabilities, analyzes
     a minimal manipulation example, and discusses the security
     implications and potential countermeasures.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 3 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.





  Xu, et al.              Expires 3 September 2026                [Page 1]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     2.  Background and AS Relationship Inference  . . . . . . . . . .   3
     3.  ASRank Overview . . . . . . . . . . . . . . . . . . . . . . .   3
     4.  Vulnerabilities in ASRank Inference . . . . . . . . . . . . .   4
     5.  A Minimal Example of Manipulations  . . . . . . . . . . . . .   5
     6.  Security Implications . . . . . . . . . . . . . . . . . . . .   6
     7.  Mitigation Considerations . . . . . . . . . . . . . . . . . .   6
     8.  Conclusion  . . . . . . . . . . . . . . . . . . . . . . . . .   7
     9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   7
     10. Security Considerations . . . . . . . . . . . . . . . . . . .   7
     11. References  . . . . . . . . . . . . . . . . . . . . . . . . .   8
       11.1.  Normative References . . . . . . . . . . . . . . . . . .   8
       11.2.  Informative References . . . . . . . . . . . . . . . . .   8
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   8

  1.  Introduction

     The Internet comprises over 80,000 ASes interconnected via the Border
     Gateway Protocol (BGP).  These interconnections are governed by
     business relationships, primarily provider-to-customer (P2C) and
     peer-to-peer (P2P), which directly influence routing policies and
     global path selection.

     Since AS business relationships are not publicly disclosed, operators
     and researchers rely on inference algorithms to identify them from
     observed BGP routing data.  Among these, CAIDA's ASRank algorithm
     [ASRank] is the most widely used methods that underpins CAIDA's
     public AS relationship dataset [CAIDA_Dataset] and has supported
     hundreds of scientific studies and operational tools.

     Despite its critical role, the security of ASRank under adversarial
     conditions remains largely unexplored.  ASRank's inference process
     relies on AS triplets, which are sequences of three consecutive
     Autonomous Systems extracted from BGP paths, and applies a greedy
     ordering based on transit degree.  This design introduces sensitivity
     to small input perturbations, which can alter inference order and
     lead to cascading misclassifications.



  Xu, et al.              Expires 3 September 2026                [Page 2]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     The objective of this document is to highlight the security risks
     introduced by structural vulnerabilities in ASRank, which may allow
     adversaries to manipulate AS relationship inference results at scale.
     Given ASRank's widespread use in both operational networks and
     research, such manipulation can undermine the reliability of critical
     datasets and downstream analyses.  This document also discusses the
     security implications of these vulnerabilities and outlines potential
     countermeasures.

     The rest of this document is structured as follows: Section 2
     provides background on AS relationships and inference techniques.
     Section 3 presents an overview of the ASRank algorithm.  Section 4
     analyzes structural vulnerabilities in ASRank's inference process.
     Section 5 introduces a minimal manipulation example to illustrate the
     impact of input perturbations.  Section 6 discusses the security
     implications of these vulnerabilities.  Section 7 outlines potential
     mitigation strategies.  Finally, Section 8 concludes the document.

  2.  Background and AS Relationship Inference

     ASes interconnect under business agreements that define how traffic
     is exchanged between them.  The two primary types of AS relationships
     are P2C and P2P.  In a P2C relationship, the provider offers transit
     services to the customer in exchange for payment.  In a P2P
     relationship, two ASes exchange traffic without settlement, typically
     to reach each other's customers.

     These relationships play a central role in shaping BGP routing
     policies, including route selection and export behavior.  However, AS
     relationships are generally considered proprietary and are not
     publicly disclosed.  As a result, researchers and operators rely on
     inference algorithms to estimate these relationships from observed
     BGP routing data.

     Most inference methods operate on BGP paths collected from public
     vantage points and apply heuristic or statistical techniques to
     classify AS links.  Among these, CAIDA's ASRank algorithm is the most
     widely used and influential approaches.

  3.  ASRank Overview

     ASRank is a multi-stage algorithm developed by CAIDA to infer AS
     relationships from empirical BGP routing data.  It is used to
     generate CAIDA's public AS relationship datasets and has become a
     foundational tool in BGP researches and operations.






  Xu, et al.              Expires 3 September 2026                [Page 3]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     The algorithm begins by sanitizing BGP paths, removing those that
     contain artifacts such as loops, reserved AS numbers, or Internet
     Exchange Points (IXPs).  It then sorts ASes in descending order of
     transit degree, computed from counts of AS triplets observed in the
     data.  Based on this ordering, ASRank first identifies a clique of
     top-level ASes and infers the corresponding P2P relationships within
     the clique.  After filtering poisoned paths further, it infers P2C
     relationships in a greedy, top-down manner.  During this process,
     newly inferred relationships serve as a basis for subsequent
     inferences, thereby progressively determining P2C relationships until
     no additional ones can be resolved.  The algorithm then handles
     special cases, such as ASes with atypical transit patterns.  Finally,
     any remaining unresolved links are inferred as P2P relationships.

     While ASRank's design enables scalable and automated inference, its
     reliance on triplet-based statistics and order-sensitive heuristics
     introduces structural vulnerabilities, which are discussed in the
     following sections.

  4.  Vulnerabilities in ASRank Inference

     A central component of ASRank's inference process is the P2C
     classification phase, which accounts for the majority of inferred
     relationships.  In this phase, ASRank processes ASes in descending
     order of their transit degree, a metric defined as the number of
     distinct neighbors adjacent to an AS when it appears in the middle
     position of observed AS triplets (X, Y, Z).  For each AS, ASRank
     examines relevant triplets (X, Y, Z) and applies a heuristic: if the
     relationship between AS X and AS Y is P2P (denoted as X - Y) or P2C
     (denoted as X > Y), the algorithm infers a P2C relationship AS Y > AS
     Z.

     A key characteristic of this process is its order-dependence: the
     relationships inferred earlier influence which triplets can trigger
     further inferences, shaping the final output.  In practice, many ASes
     have similar transit degrees.  As a result, even small changes in the
     input data can perturb the processing order, leading to different
     inference sequences and potentially altering the resulting AS
     relationships.












  Xu, et al.              Expires 3 September 2026                [Page 4]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     This sensitivity introduces a structural vulnerability.  Minor
     perturbations in the input data can trigger cascading inference
     changes, resulting in qualitatively different relationship graphs.
     In open environments where BGP data is collected from public vantage
     points, this behavior creates an opportunity for inference results to
     be influenced or manipulated without directly modifying the
     algorithm.  These characteristics raise concerns about the robustness
     of ASRank in adversarial or noisy settings, especially given its
     widespread use in BGP researches and operations.

  5.  A Minimal Example of Manipulations

     Consider the following observed BGP paths, where we highlight only
     the contiguous AS-path segments that are relevant for inference:

         path 1:  ... M N A B C D
         path 2:  ... M N A B E
         path 3:  ... M N A F
         path 4:  ... M N A G

     These paths represent a typical scenario where multiple downstream
     ASes are accessible through a shared chain of transit providers.
     ASRank initially infers a top-level clique of major transit ASes.
     For simplicity, we assume that M and N are part of this clique, with
     their mutual relationship already inferred as P2P (M-N).

     From these paths, ASRank extracts AS triplets and computes the
     transit degrees (denoted by TD).  AS A appears as the middle AS in
     four triplets and has four distinct neighbors, yielding TD(A)=4.
     Similarly, TD(B)=3 and TD(C)=2, while D, E, F, and G all have a
     transit degree of zero.  ASRank thus visits ASes in the following
     order:

         A -> B -> C -> D -> E -> F -> G

     Starting with A, ASRank infers N>A from (M,N,A) as M-N is already
     known.  It then moves to B, inferring A>B from the triplet (N,A,B),
     based on the previously inferred N>A.  Following this sequence,
     ASRank infers the following P2C relationships:

         {N > A, A > B, B > C, C > D, B > E, A > F, A > G}

     Now consider adding two additional BGP paths:

         path 5:  ... X A B D C Y
         path 6:  ... X F D G Y





  Xu, et al.              Expires 3 September 2026                [Page 5]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     Due to the new introduced AS triplets, the transit degree of D
     increases to TD(D)=4, surpassing that of C, whose transit degree
     becomes TD(C)=3.  As a result, D is visited before C, resulting in
     the updated inference order:

         A -> B -> D -> C -> ...

     This reordered traversal leads to a different inference chain.  When
     visiting D, the relationship A>B has already been inferred, and
     (A,B,D) in path 5 therefore satisfies the heuristic and implies B>D.
     Later, when visiting C, (B,D,C) in path 5 together with the newly
     inferred B>D again matches the heuristic and implies D>C.
     Consequently, the final set of inferred relationships includes:

         {N > A, A > B, B > D, D > C, B > C, B > E, A > F, A > G}

     Compared to the original outcome, C>D is replaced by D>C, and a new
     P2C relationship B>D is inferred.

  6.  Security Implications

     Because ASRank-derived datasets are widely used in both academic
     studies and operational tools, inaccuracies in inferred AS
     relationships can propagate into downstream systems, undermining the
     reliability of Internet measurements and topology analysis, and
     leading to incorrect assumptions about routing behavior and
     connectivity.

     Manipulated or unstable inferences can lead to misleading views of
     Internet structure and behavior.  For instance, incorrect
     relationships may distort the AS hierarchy or misrepresent network
     reachability.  Such errors can reduce the accuracy of systems that
     depend on AS relationship data for tasks like anomaly detection,
     route validation, and security monitoring.

     Moreover, inaccurate ASRank inferences can unfairly elevate an AS's
     perceived importance, giving it undue visibility, influence, or
     business advantage.  This can also distort peering decisions and
     mislead customers.

  7.  Mitigation Considerations

     To address ASRank's vulnerabilities, two areas require attention:
     improving inference robustness and limiting the injection of forged
     AS paths.






  Xu, et al.              Expires 3 September 2026                [Page 6]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     At the inference level, robustness can be enhanced by using
     organizational or business-affiliation data to flag relationships
     that don't align with expected economic logic.  Rather than making
     early, fixed decisions, ASRank could analyze the full relationship
     graph to reduce the impact of inference order.  Since forged paths
     may coexist with valid ones, detecting internal inconsistencies, like
     unexpected reversals in relationships, can help identify and verify
     suspicious inferences.

     At the routing layer, deploying path validation mechanisms can reduce
     the risk of forged triplets.  BGPsec [RFC8205] provides cryptographic
     protection for AS paths but is limited by low adoption and reduced
     effectiveness under partial deployment.  Newer approaches like ASPA
     [I-D.ietf-sidrops-aspa-verification] and ASRA
     [I-D.geng-sidrops-asra-profile] offer more practical alternatives by
     verifying legitimate AS connections, though they are still in early
     stages and face known limitations.

  8.  Conclusion

     This document analyzes the structural vulnerabilities of ASRank, a
     widely used algorithm for inferring AS business relationships from
     BGP routing data.  We describe how ASRank’s order-sensitive inference
     process can lead to unstable or incorrect relationship
     classifications under small input changes.  A minimal example
     demonstrates how minor perturbations in BGP paths can trigger
     cascading inference shifts.  These vulnerabilities raise concerns
     about the reliability of ASRank-derived datasets, especially in
     adversarial or noisy conditions.  Strengthening inference robustness
     and deploying path validation mechanisms are essential steps toward
     improving the security and trustworthiness of AS relationship
     inference.

  9.  IANA Considerations

     This document includes no request to IANA.

  10.  Security Considerations

     The structural weaknesses in ASRank's inference process may be
     exploited to manipulate AS relationship outputs without altering the
     algorithm itself.  Such manipulation can distort Internet topology
     views, mislead routing decisions, and undermine downstream systems
     that rely on inferred AS relationships.  While this document does not
     describe specific attack strategies, it highlights the risks posed by
     inference sensitivity and the potential for undetected influence.
     Mitigation strategies are discussed in Section 7.  Operators and
     researchers are encouraged to interpret ASRank-derived data with



  Xu, et al.              Expires 3 September 2026                [Page 7]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     caution and consider complementary validation mechanisms.

  11.  References

  11.1.  Normative References

     [RFC8205]  Lepinski, M., Ed. and K. Sriram, Ed., "BGPsec Protocol
                Specification", RFC 8205, DOI 10.17487/RFC8205, September
                2017, <https://www.rfc-editor.org/info/rfc8205>.

  11.2.  Informative References

     [ASRank]   Luckie, M., Huffaker, B., Dhamdhere, A., Giotsas, V., and
                K. Claffy, "AS relationships, customer cones, and
                validation", IMC 2013, Proceedings of the 2013 Conference
                on Internet Measurement Conference, pp. 243–256 , 23
                October 2013, <https://doi.org/10.1145/2504730.2504747>.

     [CAIDA_Dataset]
                CAIDA, "AS Relationships Dataset", 2025,
                <https://www.caida.org/catalog/datasets/as-
                relationships/>.

     [I-D.ietf-sidrops-aspa-verification]
                Azimov, A., Bogomazov, E., Bush, R., Patel, K., Snijders,
                J., and K. Sriram, "BGP AS_PATH Verification Based on
                Autonomous System Provider Authorization (ASPA) Objects",
                Work in Progress, Internet-Draft, draft-ietf-sidrops-aspa-
                verification-24, 19 October 2025,
                <https://datatracker.ietf.org/doc/html/draft-ietf-sidrops-
                aspa-verification-24>.

     [I-D.geng-sidrops-asra-profile]
                Geng, N., Sriram, K., and M. Huang, "A Profile for
                Autonomous System Relationship Authorization (ASRA)", Work
                in Progress, Internet-Draft, draft-geng-sidrops-asra-
                profile-02, 17 October 2025,
                <https://datatracker.ietf.org/doc/html/draft-geng-sidrops-
                asra-profile-02>.

  Authors' Addresses

     Yi Xu
     Tsinghua University
     30 Shuangqing Road
     Beijing
     100084
     China



  Xu, et al.              Expires 3 September 2026                [Page 8]


  Internet-Draft  Structural Vulnerabilities in ASRank und      March 2026


     Email: y-xu22@mails.tsinghua.edu.cn


     Yihao Chen
     Tsinghua University
     30 Shuangqing Road
     Beijing
     100084
     China
     Email: yh-chen21@mails.tsinghua.edu.cn


     Ke Xu
     Tsinghua University
     30 Shuangqing Road
     Beijing
     100084
     China
     Email: xuke@tsinghua.edu.cn


     Qi Li
     Tsinghua University
     30 Shuangqing Road
     Beijing
     100084
     China
     Email: qli01@tsinghua.edu.cn


     Jianping Wu
     Tsinghua University
     30 Shuangqing Road
     Beijing
     100084
     China
     Email: jianping@cernet.edu.cn














  Xu, et al.              Expires 3 September 2026                [Page 9]


name: draft-cgfabk-nmrg-ibn-generative-ai
rev: 01
title: Generative AI for Intent-Based Networking
time: 2025-10-17T09:47:19Z
group_uri: /api/v1/group/group/1027/
expires: 2026-04-20T09:47:19Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.317731+00:00
source: ietf
abstract:
  This document describes how to specialize AI models in order to offer
     a scalable, efficient path to creating highly targeted generative
     models for Intent-Based Networking.

full_text:
  Network Management                                           P. Cassara'
  Internet-Draft                                                  A. Gotta
  Intended status: Informational                                  CNR-ISTI
  Expires: 20 April 2026                                       G. Fioccola
                                                              A. Artigiani
                                                       Huawei Technologies
                                                                 R. Burrai
                                                                  E. Kolaj
                                                         SIRIUS Technology
                                                               M. Martalo'
                                                                V. Pilloni
                                                    University of Cagliari
                                                           17 October 2025


                 Generative AI for Intent-Based Networking
                   draft-cgfabk-nmrg-ibn-generative-ai-01

  Abstract

     This document describes how to specialize AI models in order to offer
     a scalable, efficient path to creating highly targeted generative
     models for Intent-Based Networking.

  About This Document

     This note is to be removed before publishing as an RFC.

     The latest revision of this draft can be found at
     https://giuseppefioccola.github.io/draft-cgfabk-nmrg-ibn-generative-
     ai/draft-cgfabk-nmrg-ibn-generative-ai.html.  Status information for
     this document may be found at https://datatracker.ietf.org/doc/draft-
     cgfabk-nmrg-ibn-generative-ai/.

     Discussion of this document takes place on the Network Management
     Research Group mailing list (mailto:nmrg@irtf.org), which is archived
     at https://mailarchive.ietf.org/arch/browse/nmrg.  Subscribe at
     https://www.ietf.org/mailman/listinfo/nmrg/.

     Source for this draft and an issue tracker can be found at
     https://github.com/giuseppefioccola/draft-cgfabk-nmrg-ibn-generative-
     ai.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.




  Cassara', et al.          Expires 20 April 2026                 [Page 1]


  Internet-Draft              ibn-generative-ai               October 2025


     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 20 April 2026.

  Copyright Notice

     Copyright (c) 2025 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
       1.1.  Overview of IBN . . . . . . . . . . . . . . . . . . . . .   4
       1.2.  Role of Generative AI in IBN  . . . . . . . . . . . . . .   4
       1.3.  Motivation for AI Model Specialization  . . . . . . . . .   5
       1.4.  Transfer Learning . . . . . . . . . . . . . . . . . . . .   5
     2.  Conventions and Definitions . . . . . . . . . . . . . . . . .   6
     3.  Background Concepts . . . . . . . . . . . . . . . . . . . . .   6
       3.1.  Generative AI and LLMs in Networking  . . . . . . . . . .   6
       3.2.  Intent Parsing and Translation via Generative Models  . .   6
       3.3.  Fundamentals of LoRA  . . . . . . . . . . . . . . . . . .   7
     4.  Specializing Generative AI with LoRA  . . . . . . . . . . . .   7
       4.1.  The Need for Model Specialization in IBN  . . . . . . . .   7
       4.2.  LoRA for Lightweight Fine-Tuning  . . . . . . . . . . . .   7
       4.3.  Comparison with Traditional Fine-Tuning and Pruning . . .   7
       4.4.  Efficiency and Performance Gains in Network Models  . . .   7
     5.  Hub: Repository of Specialized Models . . . . . . . . . . . .   7
       5.1.  Concept and Architecture of a Hub . . . . . . . . . . . .   7
       5.2.  Organizing and Indexing Adapters for Networking Tasks . .   8
       5.3.  Examples of IBN-Targeted LoRA Adapters  . . . . . . . . .   8
       5.4.  Benefits of Modular and Shareable Specializations . . . .   8



  Cassara', et al.          Expires 20 April 2026                 [Page 2]


  Internet-Draft              ibn-generative-ai               October 2025


     6.  Flow: Model Fusion and Evolution  . . . . . . . . . . . . . .   8
       6.1.  Concept of Flow for Adapter Composition . . . . . . . . .   8
       6.2.  Fusion of Multiple Adapters to Generate New Models  . . .   8
       6.3.  Workflow for Dynamic Specialization Based on Intent . . .   9
       6.4.  Case Study: Multi-Domain Network Management . . . . . . .   9
     7.  Lifecycle of Specialized Models in IBN  . . . . . . . . . . .   9
       7.1.  Model Generation, Evaluation, and Deployment  . . . . . .   9
       7.2.  Feedback Loops and Continuous Adaptation  . . . . . . . .  10
       7.3.  Benchmarking Specialized Models: Accuracy, Latency, and
             Resource Consumption  . . . . . . . . . . . . . . . . . .  10
     8.  Practical Frameworks and Tools  . . . . . . . . . . . . . . .  10
       8.1.  LoRA Fine-Tuning Libraries  . . . . . . . . . . . . . . .  10
       8.2.  LoRA Hub Implementations  . . . . . . . . . . . . . . . .  10
       8.3.  Toolchains for LoRA Flow and Adapter Fusion . . . . . . .  10
     9.  Logical Architecture for Model Management . . . . . . . . . .  10
       9.1.  Overview  . . . . . . . . . . . . . . . . . . . . . . . .  11
       9.2.  Logical Architecture Diagram  . . . . . . . . . . . . . .  11
       9.3.  Detailed Workflow . . . . . . . . . . . . . . . . . . . .  12
       9.4.  Models Adaptation, and Orchestration  . . . . . . . . . .  13
     10. Network Digital Twin Ontology . . . . . . . . . . . . . . . .  13
     11. Challenges and Research Directions  . . . . . . . . . . . . .  14
       11.1.  Model Interoperability in LoRA Flow  . . . . . . . . . .  14
       11.2.  Ensuring Security and Robustness in Specialized
              Adapters . . . . . . . . . . . . . . . . . . . . . . . .  14
       11.3.  Governance of LoRA Repositories  . . . . . . . . . . . .  14
       11.4.  Towards Autonomous Model Lifecycle Management in IBN . .  14
     12. Security Considerations . . . . . . . . . . . . . . . . . . .  14
     13. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  14
     14. References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
       14.1.  Normative References . . . . . . . . . . . . . . . . . .  14
       14.2.  Informative References . . . . . . . . . . . . . . . . .  14
     Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  15
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  15

  1.  Introduction

     This document describes how transfer learning techniques can be
     adopted to design generative AI specialized models for Intent-Based
     Networking (IBN).  It also describes tools, such as Low-rank
     Adaptation (LoRA), for achieving efficient and scalable transfer
     learning in data networks for designing targeted generative models
     for Intent-Based Networking (IBN).

     The objective of this document is define a framework for the rapid
     adaptation and composition of specialized knowledge, addressing the
     challenges identified in [AI-Challenges] and enabling the dynamic,
     multi-domain use cases of [IBN-UseCases].




  Cassara', et al.          Expires 20 April 2026                 [Page 3]


  Internet-Draft              ibn-generative-ai               October 2025


     Future work should focus on interoperability, governance, and
     autonomous management to fully realize the potential of this
     approach.

  1.1.  Overview of IBN

     IBN represents a paradigm shift in network management, aiming to
     bridge the gap between business objectives and network
     configurations.  Unlike traditional networking, which requires
     manual, device-level configurations, IBN allows operators to specify
     high-level intents (e.g., "ensure low latency for video traffic"),
     which the system then automatically translates, enforces, and
     continuously validates.  Key references include [IBN-UseCases], which
     outlines IBN use cases across enterprise, data center, and service
     provider environments.

     IBN fundamentally changes the role of network administrators by
     focusing on desired outcomes instead of manual command-line
     configurations.  The system must be capable of not only translating
     intents into actionable policies but also of continuously validating
     whether those intents are met.  This requires advanced mechanisms for
     policy enforcement, telemetry collection, and feedback loops that
     ensure alignment between high-level business goals and actual network
     performance.  IBN can reduce operational complexity, improve network
     agility, and significantly lower the time required to deploy new
     services.

  1.2.  Role of Generative AI in IBN

     Generative AI, particularly Large Language Models (LLMs), can enhance
     IBN by automating intent parsing, policy generation, and network
     troubleshooting.  LLMs can understand natural language intents,
     generate low-level configurations, and adapt policies in real-time.
     This aligns with the challenges presented in [AI-Challenges], which
     stresses the need for models that can dynamically adapt to context
     and scale efficiently.

     Generative AI can extend its role to include auto-remediation,
     dynamic configuration adjustments, and the creation of context-aware
     policies based on real-time network conditions.  These capabilities
     are critical in environments with rapidly shifting traffic patterns,
     such as 5G networks, multi-cloud environments, and large-scale SDN
     deployments.  By leveraging fine-tuned generative models, IBN systems
     can become self-configuring, self-optimizing, and self-healing,
     moving closer to the vision of fully autonomous networks.






  Cassara', et al.          Expires 20 April 2026                 [Page 4]


  Internet-Draft              ibn-generative-ai               October 2025


  1.3.  Motivation for AI Model Specialization

     Generic LLMs often lack the precision required for domain-specific
     applications like IBN.  Specialization is critical to improve
     accuracy, reduce inference time, and minimize resource consumption.
     LoRA provides an efficient method to fine-tune large models using
     minimal computational resources, enabling targeted model
     specialization for distinct network domains or intent categories.

     Specialization ensures that the AI system correctly interprets and
     applies complex domain-specific policies, such as those for Quality
     of Service (QoS), security compliance, and multi-vendor
     interoperability.  Without specialization, generic models risk
     generating suboptimal or even invalid configurations.  LoRA-based
     specialization enables the rapid creation and deployment of these
     focused models, supporting agile, intent-driven network operations
     while maintaining the scalability and cost-effectiveness required for
     wide deployment across heterogeneous infrastructures.

  1.4.  Transfer Learning

     Transfer learning enables pretrained models to adapt to specific
     tasks with significantly less data and computational resources.  In
     the context of IBN, this approach offers a dual advantage: enhancing
     the efficiency of model training and improving the reliability of
     intent recognition and execution.  Indeed, it avoids the need for
     building models from scratch, which is not only resource-intensive
     but also prone to overfitting due to limited labeled network-specific
     data.  By fine-tuning models on domain-specific datasets, developers
     can quickly create robust models that interpret network intents with
     high accuracy.  Furthermore, it facilitates cross-domain knowledge
     integration, allowing the model to generalize better across various
     networking environments and topologies.  Additionally, continual
     learning mechanisms can be integrated into the LLM-based IBN
     framework to refine performance over time, incorporating feedback
     loops that learn from real-world interactions and user corrections.

     In conclusion, transfer learning significantly boosts the viability
     of LLM-based solutions in IBN by offering a cost-effective, accurate,
     and scalable method for intent interpretation and execution.  This
     fusion of LLMs with transfer learning not only propels the automation
     of network operations but also ensures that the AI systems remain
     adaptive, interpretable, and aligned with organizational objectives
     in dynamic digital infrastructures.







  Cassara', et al.          Expires 20 April 2026                 [Page 5]


  Internet-Draft              ibn-generative-ai               October 2025


  2.  Conventions and Definitions

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in
     BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  3.  Background Concepts

  3.1.  Generative AI and LLMs in Networking

     LLMs like GPT and BERT can process complex, unstructured inputs and
     generate contextually relevant outputs.  In IBN, LLMs are used to
     translate high-level intents into vendor-agnostic policies and
     device-specific configurations.  The integration of these models into
     network management systems is emerging as a critical research area,
     as outlined in [AI-Challenges].

  3.2.  Intent Parsing and Translation via Generative Models

     An example flow in Figure 1 illustrates how an intent like "minimize
     latency for video traffic" is parsed by a generative model and
     translated into specific QoS policies.

     +---------------------+
     |  User High-Level    |
     |       Intent        |
     +---------------------+
                |
                v
     +---------------------+
     |  Generative Model   |
     | (Intent Parser +    |
     |  Policy Generator)  |
     +---------------------+
                |
                v
     +-----------------------+
     | Network Configuration |
     +-----------------------+

            Figure 1: Example flow of intent parsing and translation








  Cassara', et al.          Expires 20 April 2026                 [Page 6]


  Internet-Draft              ibn-generative-ai               October 2025


  3.3.  Fundamentals of LoRA

     LoRA [LoRA] introduces trainable low-rank matrices into the layers of
     pre-trained models, allowing fine-tuning with significantly fewer
     parameters.  This approach reduces the storage and computational
     footprint, enabling deployment of specialized models even on
     resource-constrained systems.

  4.  Specializing Generative AI with LoRA

  4.1.  The Need for Model Specialization in IBN

     IBN requires high domain accuracy.  For example, a generic model may
     misinterpret a telecommunications-specific intent.  Specializing
     models using LoRA ensures they are finely tuned to the nuances of
     networking language, policies, and protocols.

  4.2.  LoRA for Lightweight Fine-Tuning

     LoRA fine-tunes only small, low-rank matrices, leaving the rest of
     the model unchanged.  For instance, adapting a generic LLM to focus
     on BGP policy generation could be achieved by training a small
     adapter on BGP-related datasets, reducing the compute requirements
     compared to full fine-tuning.

  4.3.  Comparison with Traditional Fine-Tuning and Pruning

     Unlike traditional fine-tuning, which adjusts the entire model, or
     pruning, which removes less important weights, LoRA introduces new
     parameters without overwriting existing knowledge.  This allows
     multiple specialized LoRA adapters to coexist and be swapped or
     combined as needed.

  4.4.  Efficiency and Performance Gains in Network Models

     Empirical studies show that LoRA adapters can reduce memory
     consumption by over 70% compared to full fine-tuning.  For example,
     fine-tuning a model for IPv6 routing policy generation using LoRA
     reduces the required GPU VRAM from 16 GB to approximately 5 GB while
     maintaining performance.

  5.  Hub: Repository of Specialized Models

  5.1.  Concept and Architecture of a Hub

     A Hub is a structured repository where specialized adapters are
     stored, indexed, and shared.  It enables efficient reuse and
     modularity.  The architecture typically includes:



  Cassara', et al.          Expires 20 April 2026                 [Page 7]


  Internet-Draft              ibn-generative-ai               October 2025


     *  Adapter storage

     *  Metadata index (domain, use case, version)

     *  Dependency tracking for composite models

  5.2.  Organizing and Indexing Adapters for Networking Tasks

     Adapters can be categorized by networking domain (e.g., security,
     QoS, routing) and intent granularity (e.g., traffic type, SLA
     constraint).  Cross-indexing improves discoverability and
     composability.

  5.3.  Examples of IBN-Targeted LoRA Adapters

     *  Adapter A: IPv6 Segment Routing configuration

     *  Adapter B: QoS policy for video prioritization

     *  Adapter C: Security baseline for zero-trust architectures

  5.4.  Benefits of Modular and Shareable Specializations

     Operators can quickly compose or replace adapters based on current
     intents, improving agility and reducing the cost of specialization.
     LoRA Hub supports collaboration and community-driven model evolution.

  6.  Flow: Model Fusion and Evolution

  6.1.  Concept of Flow for Adapter Composition

     Flow refers to the systematic combination of multiple adapters to
     form a new, composite model capable of handling complex, multi-domain
     intents.

  6.2.  Fusion of Multiple Adapters to Generate New Models

     Example: Combining an adapter for SLA enforcement with another for
     security compliance to address intents like "guarantee low latency
     for critical applications within zero-trust boundaries."











  Cassara', et al.          Expires 20 April 2026                 [Page 8]


  Internet-Draft              ibn-generative-ai               October 2025


          +-------------------------+
          |        Base LLM         |
          +-------------------------+
            |        |    ...   \
            |        |           \
     +---------+ +----------+   +----------+
     | Adapter | | Adapter  |   | Adapter  |
     |   QoS   | | Security |...|     j    |
     +---------+ +----------+   +----------+
             \        |            /
              \       |    ...    /
              +-------------------+
              |     Composite     |
              |    Specialized    |
              |       Model       |
              +-------------------+

          Figure 2: Fusion of multiple adapters to generate new models

  6.3.  Workflow for Dynamic Specialization Based on Intent

     1.  Parse user intent

     2.  Query LoRA Hub for relevant adapters

     3.  Dynamically load and fuse adapters

     4.  Generate configuration

  6.4.  Case Study: Multi-Domain Network Management

     A service provider managing both MPLS and SDN domains can fuse LoRA
     adapters for each to generate cross-domain policies automatically in
     response to high-level intents.

  7.  Lifecycle of Specialized Models in IBN

  7.1.  Model Generation, Evaluation, and Deployment

     *  Generation: Fine-tuning on domain datasets

     *  Evaluation: Accuracy, latency, resource profiling

     *  Deployment: Containerized adapters for on-demand loading







  Cassara', et al.          Expires 20 April 2026                 [Page 9]


  Internet-Draft              ibn-generative-ai               October 2025


  7.2.  Feedback Loops and Continuous Adaptation

     In-network telemetry provides real-time feedback on policy
     effectiveness, enabling further fine-tuning or adapter updates,
     aligning with the adaptive model requirements discussed in
     [AI-Challenges].

  7.3.  Benchmarking Specialized Models: Accuracy, Latency, and Resource
        Consumption

     Key benchmarks include:

     *  Policy generation latency (< 50 ms target)

     *  Memory footprint reduction (70% over full models)

     *  Domain-specific intent accuracy (> 90%)

  8.  Practical Frameworks and Tools

  8.1.  LoRA Fine-Tuning Libraries

     *  PEFT (Parameter-Efficient Fine-Tuning) [HF-PEFT]

     *  Hugging Face LoRA integration

  8.2.  LoRA Hub Implementations

     *  Model repositories using Hugging Face Hub structure

     *  Metadata indexing via JSON-LD for semantic search

  8.3.  Toolchains for LoRA Flow and Adapter Fusion

     *  Adapter composition APIs

     *  Intent parsing frontends

     *  Deployment via Kubernetes with sidecar loading of adapters

  9.  Logical Architecture for Model Management










  Cassara', et al.          Expires 20 April 2026                [Page 10]


  Internet-Draft              ibn-generative-ai               October 2025


  9.1.  Overview

     Drawing inspiration from the Analytics Logical Function (AnLF) and
     Model Training Logical Function (MTLF) architectures defined in 3GPP
     NWDAF, this section proposes a logical architecture to manage, store,
     and orchestrate AI models for Intent-Based Networking.  The
     architecture introduces the following logical components:

     *  Model Repository Function (MRF): Stores adapters and base models.

     *  Model Training and Specialization Function (MTSF): Handles fine-
        tuning, evaluation, and versioning.

     *  Model Fusion and Composition Function (MFCF): Manages flow
        processes to create composite models.

     *  Intent Processing and Adapter Orchestration Function (IPOF):
        Dynamically selects, composes, and deploys adapters based on
        parsed intents.

     *  Telemetry Feedback Function (TFF): Continuously monitors
        performance and triggers model re-specialization when required.

  9.2.  Logical Architecture Diagram



























  Cassara', et al.          Expires 20 April 2026                [Page 11]


  Internet-Draft              ibn-generative-ai               October 2025


     +----------------------------+
     |      Intent Interface      |
     +----------------------------+
                   |
                   v
     +-------------------------------+
     |    Intent Processing and      |
     |  Adapter Orchestration (IPOF) |
     +-------------------------------+
                   |
                   v
     +------------------------------+
     | Model Fusion and Composition |
     |         (MFCF)               |
     +------------------------------+
                   |
                   v
     +-----------------------------+
     |  Model Repository (MRF)     |
     +-----------------------------+
                   ^
                   |
     +---------------------------------+
     | Model Training & Specialization |
     |          (MTSF)                 |
     +---------------------------------+
                   ^
                   |
     +-----------------------------------+
     | Telemetry Feedback Function (TFF) |
     +-----------------------------------+

                     Figure 3: Logical Architecture Diagram

  9.3.  Detailed Workflow

     1.  Intent Reception: User or application submits a high-level intent
         through the Intent Interface.

     2.  Adapter Orchestration: IPOF parses the intent, queries the MRF
         for relevant adapters, and orchestrates dynamic fusion via MFCF.

     3.  Model Fusion: MFCF composes adapters into a composite model
         tailored to the intent.

     4.  Deployment: The composite model is deployed into the IBN system
         to generate specific configurations.




  Cassara', et al.          Expires 20 April 2026                [Page 12]


  Internet-Draft              ibn-generative-ai               October 2025


     5.  Telemetry Feedback: The TFF monitors the applied configurations'
         impact on network KPIs and feeds insights back to IPOF and MTSF.

     6.  Re-Specialization: If performance deviates from expected
         thresholds, MTSF initiates LoRA re-training or fine-tuning using
         new data collected via TFF.

  9.4.  Models Adaptation, and Orchestration

     This architecture closely follows the modularity principles of
     NWDAF's AnLF for real-time analytics and MTLF for model training and
     distribution, enabling scalability, continuous learning, and rapid
     specialization across distributed edge and core network nodes.  Each
     function can be independently scaled and containerized, ensuring
     fault tolerance and efficient resource utilization.  This logical
     separation allows efficient model lifecycle management, supports
     federated learning, and can integrate with standard IBN orchestrators
     via REST APIs or 3GPP-compatible interfaces.

  10.  Network Digital Twin Ontology

     The proposed generative AI–enabled Intent-Based Networking (IBN)
     architecture can be empowered with a Network Digital Twin (NDT)
     ontology, which serves as the semantic backbone that allows the
     system to represent, reason, and act upon the network’s physical and
     logical states in a unified way.  Since the adapters are categorized
     by networking domain and intent granularity with different quality
     parameters, the ontology has to be quality-driven, meaning that its
     core entities and relationships are structured around quality
     indicators such as latency, throughput, reliability, and energy
     efficiency.  The ontology relies on a semantic representation so that
     every network element (node, link, function) is described by its
     functional attributes together with quantifiable quality parameters.
     This enables intelligent alignment between service requirements and
     underlying resource capabilities through IBN.  Within the overall
     generative AI-enabled IBN model, the ontology provides a multi-
     layered abstraction that connects service requirements with traffic
     parameters and physical infrastructure, supporting both top-down
     intent translation and bottom-up feedback from telemetry data.  This
     ontology enhances the adapter fusion and orchestration processes by
     guiding AI models to interpret intents in the context of network
     quality objectives, select appropriate policy actions, and evaluate
     trade-offs between performance and efficiency.  By embedding these
     semantic and quality-aware relationships, the NDT ontology enables
     the digital twin to operate as a continuously learning system,
     capable of dynamically optimizing network behavior, validating
     intents before deployment, and adapting to evolving conditions while
     maintaining compliance with quality and energy-efficiency goals.



  Cassara', et al.          Expires 20 April 2026                [Page 13]


  Internet-Draft              ibn-generative-ai               October 2025


  11.  Challenges and Research Directions

  11.1.  Model Interoperability in LoRA Flow

     Ensuring consistent parameterization and preventing interference when
     fusing adapters.

  11.2.  Ensuring Security and Robustness in Specialized Adapters

     Preventing malicious adapter injection and ensuring adapters respect
     security constraints.

  11.3.  Governance of LoRA Repositories

     Managing trust, versioning, and deprecation of adapters.

  11.4.  Towards Autonomous Model Lifecycle Management in IBN

     Developing self-optimizing pipelines for adapter generation,
     validation, and retirement.

  12.  Security Considerations

     TODO

  13.  IANA Considerations

     This document has no IANA actions.

  14.  References

  14.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/rfc/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

  14.2.  Informative References

     [AI-Challenges]
                François, J., Clemm, A., Papadimitriou, D., Fernandes, S.,
                and S. Schneider, "Research Challenges in Coupling
                Artificial Intelligence and Network Management", Work in



  Cassara', et al.          Expires 20 April 2026                [Page 14]


  Internet-Draft              ibn-generative-ai               October 2025


                Progress, Internet-Draft, draft-irtf-nmrg-ai-challenges-
                05, 18 March 2025, <https://datatracker.ietf.org/doc/html/
                draft-irtf-nmrg-ai-challenges-05>.

     [HF-PEFT]  "Hugging Face PEFT", n.d.,
                <https://github.com/huggingface/peft>.

     [IBN-UseCases]
                Yao, K., Chen, D., Jeong, J. P., Wu, Q., Yang, C.,
                Contreras, L. M., and G. Fioccola, "Use Cases and
                Practices for Intent-Based Networking", Work in Progress,
                Internet-Draft, draft-irtf-nmrg-ibn-usecases-00, 28 March
                2025, <https://datatracker.ietf.org/doc/html/draft-irtf-
                nmrg-ibn-usecases-00>.

     [LoRA]     "*** BROKEN REFERENCE ***".

  Acknowledgments

     TODO

  Authors' Addresses

     Pietro Cassara'
     CNR-ISTI
     Pisa
     Italy
     Email: pietro.cassara@isti.cnr.it


     Alberto Gotta
     CNR-ISTI
     Pisa
     Italy
     Email: alberto.gotta@isti.cnr.it


     Giuseppe Fioccola
     Huawei Technologies
     Vimodrone (Milan)
     Italy
     Email: giuseppe.fioccola@huawei.com


     Aldo Artigiani
     Huawei Technologies
     Turin
     Italy



  Cassara', et al.          Expires 20 April 2026                [Page 15]


  Internet-Draft              ibn-generative-ai               October 2025


     Email: aldo.artigiani@huawei.com


     Riccardo Burrai
     SIRIUS Technology
     Prato
     Italy
     Email: riccardo.burrai@siriustec.it


     Emiljan Kolaj
     SIRIUS Technology
     Prato
     Italy
     Email: emiljan.kolaj@siriustec.it


     Marco Martalo'
     University of Cagliari
     Cagliari
     Italy
     Email: marco.martalo@unica.it


     Virginia Pilloni
     University of Cagliari
     Cagliari
     Italy
     Email: virginia.pilloni@unica.it






















  Cassara', et al.          Expires 20 April 2026                [Page 16]


name: draft-song-tsvwg-camp
rev: 00
title: Consistency-Aware Multipath Transport (CAMP) toward Interactive Multimodal LLM-Based Systems
time: 2026-03-02T11:52:13Z
group_uri: /api/v1/group/group/1027/
expires: 2026-09-03T11:52:13Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.321311+00:00
source: ietf
abstract:
  With the prosperity of generative large language models (LLMs),
     interactive LLM-based services, such as digital humans, have imposed
     new stringent requirements on low latency and high multimodal
     consistency.  Traditional interactive LLM-based systems typically
     transmit multimodal content over a single network path, thereby
     failing to exploit the advantages offered by multipath networks.
     Even when multipath transport mechanisms are adopted, single-stream
     encapsulation does not enable differentiated management of
     heterogeneous modalities.  However, naively separating modalities
     into multiple streams further introduces inter-modal arrival
     inconsistency.  To address these challenges, this document specifies
     CAMP, a consistency-aware multipath transport design over the
     Multipath QUIC (MPQUIC) protocol.  First, CAMP defines a three-stream
     separation encapsulation format to support modality-differentiated
     transmission.  Second, it incorporates a transport-layer consistency-
     aware multipath scheduler to reduce inter-modal arrival time
     deviation across network paths.  Third, it specifies a client-side
     application-layer alignment mechanism that operates in coordination
     with the transport scheduler.  To the best of our knowledge, this is
     the first specification to address multipath-enabled multimodal
     consistency guarantees for interactive LLM-based systems.

full_text:
  TSVWG                                                            Y. Song
  Internet-Draft                                      Pengcheng Laboratory
  Intended status: Informational                                     C. Li
  Expires: 3 September 2026                   East China Normal University
                                                                     Q. Li
                                                      Pengcheng Laboratory
                                                              2 March 2026


      Consistency-Aware Multipath Transport (CAMP) toward Interactive
                        Multimodal LLM-Based Systems
                          draft-song-tsvwg-camp-00

  Abstract

     With the prosperity of generative large language models (LLMs),
     interactive LLM-based services, such as digital humans, have imposed
     new stringent requirements on low latency and high multimodal
     consistency.  Traditional interactive LLM-based systems typically
     transmit multimodal content over a single network path, thereby
     failing to exploit the advantages offered by multipath networks.
     Even when multipath transport mechanisms are adopted, single-stream
     encapsulation does not enable differentiated management of
     heterogeneous modalities.  However, naively separating modalities
     into multiple streams further introduces inter-modal arrival
     inconsistency.  To address these challenges, this document specifies
     CAMP, a consistency-aware multipath transport design over the
     Multipath QUIC (MPQUIC) protocol.  First, CAMP defines a three-stream
     separation encapsulation format to support modality-differentiated
     transmission.  Second, it incorporates a transport-layer consistency-
     aware multipath scheduler to reduce inter-modal arrival time
     deviation across network paths.  Third, it specifies a client-side
     application-layer alignment mechanism that operates in coordination
     with the transport scheduler.  To the best of our knowledge, this is
     the first specification to address multipath-enabled multimodal
     consistency guarantees for interactive LLM-based systems.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.





  Song, et al.            Expires 3 September 2026                [Page 1]


  Internet-Draft                    camp                        March 2026


     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 3 September 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.  Background and Challenges . . . . . . . . . . . . . . . . . .   4
       2.1.  Characteristics of LLM-generated Multimodal Data  . . . .   4
       2.2.  Traditional Streaming Protocol  . . . . . . . . . . . . .   4
       2.3.  Multipath Transport Basics  . . . . . . . . . . . . . . .   5
       2.4.  Key Challenges  . . . . . . . . . . . . . . . . . . . . .   5
     3.  Problem Statement and Design Principles . . . . . . . . . . .   6
       3.1.  Problem Statement . . . . . . . . . . . . . . . . . . . .   6
         3.1.1.  Multimodal Data Model . . . . . . . . . . . . . . . .   6
         3.1.2.  Multipath Network Model . . . . . . . . . . . . . . .   7
         3.1.3.  Scheduling Function . . . . . . . . . . . . . . . . .   7
         3.1.4.  Performance Metrics . . . . . . . . . . . . . . . . .   7
         3.1.5.  Overall Problem Definition  . . . . . . . . . . . . .   7
       3.2.  Design Principles . . . . . . . . . . . . . . . . . . . .   8
     4.  Consistency-Aware Multipath Transport (CAMP) Design . . . . .   8
       4.1.  Overall Architecture  . . . . . . . . . . . . . . . . . .   8
         4.1.1.  Server Side . . . . . . . . . . . . . . . . . . . . .  10
         4.1.2.  Multipath Transport Network . . . . . . . . . . . . .  10
         4.1.3.  Client Side . . . . . . . . . . . . . . . . . . . . .  11
       4.2.  Three-Stream Separation Format  . . . . . . . . . . . . .  11
       4.3.  Consistency-Aware Scheduler . . . . . . . . . . . . . . .  12
       4.4.  Client-Side Alignment Mechanism . . . . . . . . . . . . .  12
     5.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
     6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  14
     7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  14



  Song, et al.            Expires 3 September 2026                [Page 2]


  Internet-Draft                    camp                        March 2026


       7.1.  Normative References  . . . . . . . . . . . . . . . . . .  14
       7.2.  Informative References  . . . . . . . . . . . . . . . . .  14
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  15

  1.  Introduction

     Generative large language models (LLMs) have enabled a new generation
     of interactive systems, including digital humans, conversational
     agents, and multimodal assistants.  These systems continuously
     generate and transmit multimodal (i.e., text, audio, and video)
     content in response to user inputs.  Unlike traditional media
     streaming or static content delivery, interactive LLM-based systems
     operate under strict real-time constraints and require strong
     semantic and temporal coupling across modalities.

     Existing transport approaches for interactive LLM-based systems
     predominantly rely on single-path transmission, which fails to
     leverage the bandwidth aggregation, redundancy, and latency diversity
     benefits offered by multipath networks.  In addition, current
     implementations typically encapsulate multiple modalities within a
     single transport stream, which preventing fine-grained multimodal
     data management.  For example, text and audio generally require
     reliable and in-order delivery, whereas video streams may tolerate
     controlled frame loss to reduce latency.

     A straightforward improvement is to separate text, audio, and video
     into independent streams to enable differentiated transmission.
     However, this may occur significant inter-modal arrival time,
     introducing data incosistency probelm.  Existing consistency
     mitigation mechanisms are primarily implemented at the client-side
     application layer through post-arrival timestamp alignment.  However,
     they operate reactively and lack coordination with transport-layer
     multipath scheduler, leading sub-optimal performance.  Furtheremore,
     existing multipath transport mechanisms primarily optimize for
     throughput or per-stream latency; they do not consider multimodal
     consistency requirements intrinsic to interactive LLM-based systems.

     Based on the above challenges, this document specifies Consistency-
     Aware Multipath Transport (CAMP), a transport-layer design built on
     Multipath QUIC (MPQUIC).  CAMP introduces structured modality
     separation, a consistency-aware scheduler, and a coordinated client-
     side alignment mechanism.  The design does not modify the underlying
     MPQUIC protocol and is deployable over existing MPQUIC
     implementations.







  Song, et al.            Expires 3 September 2026                [Page 3]


  Internet-Draft                    camp                        March 2026


     The remainder of this document is organized as follows.  Section 2
     provides background and summarizes the key challenges.  Section 3
     defines the problem and presents the design principles.  Section 4
     specifies the CAMP architecture and mechanisms.  Section 5 discusses
     security considerations, and Section 6 covers IANA considerations.

  2.  Background and Challenges

  2.1.  Characteristics of LLM-generated Multimodal Data

     Interactive LLM-based systems generate heterogeneous yet semantically
     coupled data streams, typically including text, audio, and video.
     These modalities differ significantly in their traffic patterns,
     reliability requirements, delay sensitivity, and tolerance to loss.

     Text data is discrete, low-bandwidth, and generally requires reliable
     and in-order delivery to preserve semantic correctness.  Audio data
     is continuous and delay-sensitive, requiring bounded jitter and
     reliable delivery within strict playout deadlines.  Video data is
     bandwidth-intensive and latency-sensitive but can tolerate controlled
     frame loss or quality adaptation in exchange for reduced delay.

     Although these modalities exhibit distinct transport requirements,
     they are derived from a unified generative process and therefore
     require tight temporal alignment.  For example, synthesized speech
     must correspond to generated text, and video frames representing
     facial animation must remain synchronized with spoken content.
     Consequently, multimodal transmission for interactive LLM systems
     requires both differentiated per-modality management and bounded
     inter-modal arrival skew.

  2.2.  Traditional Streaming Protocol

     Traditional real-time streaming protocols, such as the Real-Time
     Messaging Protocol (RTMP), encapsulate audio and video data within
     logical channels over a single transport connection, typically based
     on TCP.  These protocols were primarily designed for media
     broadcasting and content distribution rather than interactive,
     generative workloads.

     Such designs exhibit two fundamental limitations in the context of
     interactive LLM-based systems.  First, single-path transport cannot
     leverage multipath bandwidth aggregation, redundancy, or latency
     diversity.  Second, multiple modalities are often multiplexed within
     a single transport stream, preventing differentiated congestion
     control, retransmission strategies, and scheduling policies.  As a
     result, heterogeneous modality requirements cannot be independently
     satisfied.



  Song, et al.            Expires 3 September 2026                [Page 4]


  Internet-Draft                    camp                        March 2026


     These limitations make traditional streaming approaches unsuitable
     for interactive LLM-based systems that demand both real-time
     responsiveness and modality-aware management.

  2.3.  Multipath Transport Basics

     Multipath transport protocols enable a single logical connection to
     utilize multiple network paths concurrently.  Multipath QUIC (MPQUIC,
     as defined in [RFC9221]) extends QUIC (as defined in [RFC9000]) by
     allowing multiple paths under a single connection context, each with
     independent path characteristics.  Multipath transport can provide
     bandwidth aggregation, path redundancy, and potentially reduced
     latency through dynamic path selection.

     However, existing multipath scheduling algorithms primarily optimize
     metrics such as throughput maximization, congestion window
     utilization, or per-stream delivery latency.  They do not explicitly
     account for cross-stream temporal relationships.  When independent
     streams are distributed across heterogeneous paths with different
     delay and jitter characteristics, arrival-time divergence across
     streams may increase.

     Therefore, while multipath transport improves network utilization and
     resilience, it does not inherently guarantee multimodal consistency.
     Moreover, existing multipath schedulers do not consider multimodal
     inconsistency problem.

  2.4.  Key Challenges

     Based on the above background, four key challenges are concluded in
     transporting LLM-generated multimodal data:

     1.  *Inability of Single-Path Transmission to Exploit Multipath
         Advantages:* Traditional single-path transport cannot leverage
         multipath bandwidth aggregation, redundancy, or latency
         diversity.  This limitation constrains achievable performance in
         real-time interactive environments.

     2.  *Lack of Differentiated Multimodal Management under Single-Stream
         Encapsulation:* Encapsulating text, audio, and video within a
         single transport stream prevents modality-specific congestion
         control, reliability policies, and scheduling strategies.  As
         interactive LLM systems impose heterogeneous requirements across
         modalities, single-stream designs cannot satisfy differentiated
         quality-of-service objectives.






  Song, et al.            Expires 3 September 2026                [Page 5]


  Internet-Draft                    camp                        March 2026


     3.  *Inter-Modal Arrival Inconsistency under Naive Stream Separation
         with Multipath:* Separating modalities into independent streams
         enables differentiated management.  However, when such streams
         are scheduled independently across heterogeneous paths,
         significant inter-modal arrival skew may occur.  This timing
         divergence degrades perceptual synchronization and interactive
         quality.

     4.  *Insufficiency of Pure Application-Layer Alignment without
         Transport-Layer Coordination:* Existing multimodal consistency
         mechanisms are primarily implemented at the client-side
         application layer through post-arrival timestamp alignment.
         While such approaches can partially compensate for skew, they
         operate reactively and lack coordination with transport-layer
         multipath scheduling decisions.  Consequently, latency and
         consistency cannot be jointly optimized.  Moreover, existing
         transport-layer schedulers are not designed to be consistency-
         aware and do not consider cross-modal timing constraints.

     These challenges motivate the need for a consistency-aware multipath
     transport design tailored to interactive multimodal LLM-based
     systems.

  3.  Problem Statement and Design Principles

  3.1.  Problem Statement

     In interactive LLM-based systems, multimodal outputs (i.e., text,
     audio, video) are generated continuously and transmitted over
     heterogeneous multipath networks.  We aim to achieve low latency and
     cross-modal consistency under path heterogeneity.

  3.1.1.  Multimodal Data Model

     Let M = {T, A, V} denote the set of modalities.

     For each modality m in M, let S_m = {s_m,1, s_m,2, ...} denote the
     ordered sequence of generated data units.

     Each data unit s_m,k is associated with:

     *  generation time g_m,k

     *  size sigma_m,k

     *  deadline delta_m

     *  reliability indicator r_m, where r_m is either 0 or 1



  Song, et al.            Expires 3 September 2026                [Page 6]


  Internet-Draft                    camp                        March 2026


     For a given semantic synchronization index k, the set {s_T,k, s_A,k,
     s_V,k} represents cross-modal correlated data units.

  3.1.2.  Multipath Network Model

     Let P = {p_1, p_2, ..., p_N} denote the set of available paths.

     Each path p_i is characterized by:

     *  delay function D_i(t)

     *  available bandwidth b_i(t)

     *  loss probability l_i(t)

     We assume heterogeneous paths, i.e., there exist i != j such that
     D_i(t) is not equal to D_j(t).

  3.1.3.  Scheduling Function

     Define a scheduling function pi(.) such that: pi(s_m,k) = p_i
     indicates that data unit s_m,k is transmitted over path p_i.

     The arrival time of s_m,k is defined as: a_m,k = g_m,k + D_i(g_m,k).

  3.1.4.  Performance Metrics

     End-to-end latency L is defined as: L = average(a_m,k - g_m,k).

     Inter-modal arrival skew for synchronization index k is defined as:
     Delta_k = max_m(a_m,k) - min_m(a_m,k).

     Expected cross-modal skew C is defined as: C = average(Delta_k).  A
     smaller C indicates better cross-modal consistency.

  3.1.5.  Overall Problem Definition

     Given:

     *  modality set M

     *  multimodal data streams {S_m}

     *  heterogeneous path set P

     *  path dynamics {D_i(t), b_i(t), l_i(t)}

     Find a scheduling function pi(.) such that:



  Song, et al.            Expires 3 September 2026                [Page 7]


  Internet-Draft                    camp                        March 2026


     *  L is minimized

     *  C is minimized

     *  Reliability constraint holds: If r_m = 1, the loss probability of
        s_m,k MUST be zero

     *  Deadline constraint holds: a_m,k - g_m,k <= delta_m

     *  Bandwidth constraint holds for each path p_i: The total
        transmitted size assigned to p_i at time t MUST NOT exceed b_i(t)

  3.2.  Design Principles

     Based on the above problem statement, the consistency-aware multipath
     transport (CAMP) design for LLM-generated multimodal data is guided
     by the following principles:

     1.  *Modality Differentiation:* Different modalities (text, audio,
         and video) exhibit heterogeneous requirements in reliability,
         latency sensitivity, and rate characteristics.  The transport
         layer MUST support differentiated handling across modalities
         rather than enforcing a single homogeneous policy.

     2.  *Consistency Awareness*: In interactive multimodal systems,
         cross-modal arrival consistency is a first-order requirement.
         Scheduling decisions MUST consider inter-modal arrival skew in
         addition to per-flow latency.  Minimizing latency alone is
         insufficient.

     3.  *Coordination Between Transport and Application Layers*: Cross-
         modal consistency is an end-to-end property that depends on both
         transport-layer scheduling and receiver-side application-level
         alignment mechanisms.  Transport-layer multipath scheduling
         decisions influence arrival ordering and skew, while the receiver
         performs timestamp-based alignment and buffering.  These
         mechanisms SHOULD be designed to cooperate, rather than operate
         independently.

  4.  Consistency-Aware Multipath Transport (CAMP) Design

  4.1.  Overall Architecture

     CAMP adopts an end-to-end architecture that integrates multimodal
     generation, transport-layer consistency-aware scheduling, and
     receiver-side alignment.  The high-level architecture is illustrated
     below.




  Song, et al.            Expires 3 September 2026                [Page 8]


  Internet-Draft                    camp                        March 2026


                            +------------------------------------+
                            |              Server                |
                            |------------------------------------|
                            |          LLM Application           |
                            |  (Multimodal Generation Engine)    |
                            +----------------+-------------------+
                                             |
                                             v
                            +------------------------------------+
                            |   Server Side: Application Layer   |
                            |------------------------------------|
                            |          - Text Output             |
                            |          - Audio Output            |
                            |          - Video Output            |
                            +----------------+-------------------+
                                             |
                                             v
                            +------------------------------------+
                            | Server Side: Transport Layer (CAMP)|
                            |------------------------------------|
                            |  - Three Logical Streams           |
                            |      * Text Stream                 |
                            |      * Audio Stream                |
                            |      * Video Stream                |
                            |  - Consistency-Aware Scheduler     |
                            |  - Multipath Manager               |
                            +----------------+-------------------+
                                             |
                             ======================================
                                    Heterogeneous Paths
                             (p1, p2, ..., pN with diverse settings)
                             ======================================
                                             |
                                             v
                            +------------------------------------+
                            | Client Side: Transport Layer (CAMP)|
                            |------------------------------------|
                            |        - Stream Reassembly         |
                            |        - Path Reordering           |
                            +----------------+-------------------+
                                             |
                                             v
                            +------------------------------------+
                            |  Client Side: Application Layer    |
                            |------------------------------------|
                            |  - Timestamp-Based Alignment       |
                            |  - Cross-Modal Synchronization     |
                            +----------------+-------------------+



  Song, et al.            Expires 3 September 2026                [Page 9]


  Internet-Draft                    camp                        March 2026


                                             |
                                             v
                            +------------------------------------+
                            |               Client               |
                            |   (Multimodal Rendering Engine)    |
                            +------------------------------------+

     The above architecture consists of two primary components: Server
     Side and Client Side, with clear separation of responsibilities
     between the transport layer and the application layer at both ends.
     The end-to-end design ensures that the transport layer and
     application layer work in coordination to meet the low-latency and
     consistency requirements of multimodal interactive systems.

  4.1.1.  Server Side

     At the server side, the system is responsible for generating
     multimodal data, including text, audio, and video, which are produced
     by the LLM-based application.  These modalities are generated by the
     LLM Application (Multimodal Generation Engine), which processes user
     inputs and produces multimodal outputs in real-time.

     Once the data is generated, the Application Layer handles the
     preparation of each modality for transmission.  This layer separates
     the modalities (text, audio, video) into different logical streams,
     enabling modality-specific handling.  For example, while text and
     audio may require reliable transmission, video might be able to
     tolerate some packet loss, and thus can be transmitted with a
     different reliability strategy.

     The Transport Layer (CAMP) then encapsulates each modality into its
     own logical stream.  The transport layer is responsible for
     consistency-aware scheduling, managing how data is sent over multiple
     paths.  This layer uses a Consistency-Aware Scheduler to make
     scheduling decisions that minimize inter-modal arrival skew and
     optimize the use of available bandwidth.  It takes into account the
     heterogeneity of the transport paths and the varying needs of
     different data types (modalities).

  4.1.2.  Multipath Transport Network

     The network is composed of multiple heterogeneous paths (denoted as
     p1, p2, ..., pN) that offer diverse characteristics such as delay,
     bandwidth, and packet loss.  These paths are utilized by the CAMP
     Transport Layer to maximize the available bandwidth and optimize
     latency, while ensuring cross-modal consistency.





  Song, et al.            Expires 3 September 2026               [Page 10]


  Internet-Draft                    camp                        March 2026


     CAMP uses multipath transport to enable the aggregation of bandwidth
     and path redundancy, but the scheduling of data across these paths is
     done in a way that minimizes the inter-modal arrival skew.  The
     transport layer takes the different path characteristics into
     account, ensuring that data from different modalities arrives at the
     receiver in a synchronized manner.

  4.1.3.  Client Side

     On the client side, the data streams are received and reassembled by
     the Transport Layer (CAMP).  This layer is responsible for reordering
     data packets that may have arrived out of order due to the diverse
     paths used for transmission.  The Multipath Manager ensures that the
     transport layer maintains synchronization across all streams and
     guarantees consistency in the arrival times of the different
     modalities.

     Once the data is reassembled, the Application Layer at the client
     side performs timestamp-based alignment and cross-modal
     synchronization.  The timestamp-based alignment mechanism ensures
     that the data from all modalities (text, audio, video) is correctly
     synchronized before being presented to the user.  This
     synchronization is essential for interactive multimodal applications
     like digital humans or virtual assistants, where inconsistencies in
     the timing of different modalities would disrupt the user experience.

     The Client then renders the synchronized multimodal data, ensuring a
     seamless experience for the user.  The interaction between the
     transport layer and application layer ensures that latency is
     minimized, and consistency between modalities is maintained
     throughout the system.

  4.2.  Three-Stream Separation Format

     The Three-Stream Separation Format is designed to handle multimodal
     data (text, audio, and video) in a manner that ensures independent
     management and optimized transmission for each modality.  In this
     format, each modality is encapsulated in its own logical stream
     within the CAMP protocol.  By separating these modalities, we can
     apply different transmission policies tailored to the unique needs of
     each type of data, such as reliability, latency, and throughput.










  Song, et al.            Expires 3 September 2026               [Page 11]


  Internet-Draft                    camp                        March 2026


     Each stream is identified by a unique stream identifier (e.g., T for
     text, A for audio, V for video), allowing for differentiation and
     independent handling.  The encapsulation of each modality also
     includes important metadata that is used to ensure the correct
     delivery and synchronization of the streams.  This metadata can
     include generation timestamp, modality-specific parameters,
     synchronization index, etc.

     By using separate streams and attaching the appropriate metadata, we
     enable more efficient and precise transmission of each modality.  The
     CAMP protocol leverages multipath transport to distribute these
     streams across different network paths, optimizing the delivery of
     each modality based on its specific transmission needs.

  4.3.  Consistency-Aware Scheduler

     The Consistency-Aware Scheduler is responsible for optimizing the
     transmission of multimodal data streams while ensuring that the
     inter-modal timing remains consistent.  By leveraging multipath
     transport, the scheduler assigns each modality to the most
     appropriate network path based on its specific needs.  It considers
     factors such as bandwidth, latency, and reliability, ensuring that
     each stream (text, audio, or video) is transmitted under the
     conditions that best suit its characteristics.

     The scheduler dynamically adjusts to changing network conditions,
     such as congestion or fluctuating bandwidth, while also ensuring that
     the timing between different streams remains consistent.  It operates
     by coordinating the scheduling of streams in a way that minimizes
     timing skew between modalities, thus improving the overall
     synchronization.  This coordination with the transport layer enables
     a consistent delivery of multimodal data, ensuring that all streams
     are delivered in sync for interactive LLM-based systems.

  4.4.  Client-Side Alignment Mechanism

     The Client-Side Alignment Mechanism is responsible for ensuring that
     multimodal data streams received by the client are synchronized
     correctly before being rendered.  Upon receiving the streams, the
     client uses timestamps and synchronization indices, which were
     embedded during the transmission process, to align the streams (text,
     audio, and video) based on their respective generation times.  This
     ensures that the modalities are presented in a synchronized manner,
     with minimal temporal divergence.

     The alignment process involves using the application layer to adjust
     the playback of each modality based on the synchronization index and
     timestamps.  In the event of minor discrepancies caused by network



  Song, et al.            Expires 3 September 2026               [Page 12]


  Internet-Draft                    camp                        March 2026


     conditions or transmission delays, the client employs techniques such
     as buffering to mitigate these differences and ensure smooth
     playback.  This mechanism relies on continuous feedback from the
     transport layer to adjust the synchronization strategy dynamically,
     ensuring that the multimodal data remains consistent across all
     modalities throughout the interaction.

  5.  Security Considerations

     The CAMP protocol design includes several elements to ensure secure
     transmission of multimodal data across different network paths.
     However, as with any transport protocol, security considerations must
     be addressed to prevent potential security threats.

     1.  *Data Integrity and Confidentiality*
         Given that CAMP uses multipath transport to transmit multimodal
         data across multiple paths, ensuring data integrity and
         confidentiality is critical.  We recommend leveraging the
         existing encryption mechanisms of MPQUIC, which supports
         encryption of data in transit using standard algorithms like
         advanced encryption standard (AES).  This ensures that data is
         not compromised during transmission, preventing eavesdropping and
         unauthorized access.

     2.  *Authentication and Authorization*
         As with any network protocol, ensuring that both sender and
         receiver are authenticated is vital.  CAMP relies on MPQUIC's
         built-in authentication mechanisms, which leverage public key
         infrastructure (PKI) or certificate-based authentication to
         prevent unauthorized parties from participating in the
         communication session.

     3.  *Replay and Man-in-the-Middle Attacks*
         CAMP utilizes time-stamped data and synchronization indices to
         coordinate multimodal streams.  These timestamps should be
         protected from tampering to prevent replay attacks or man-in-the-
         middle (MITM) attacks.  To mitigate this risk, CAMP ensures that
         all communication channels are protected by end-to-end
         encryption, and utilizes integrity protection schemes (e.g.,
         HMAC) to verify the authenticity of the timestamps and
         synchronization indices.










  Song, et al.            Expires 3 September 2026               [Page 13]


  Internet-Draft                    camp                        March 2026


     4.  *DoS and DDoS Attacks*
         The CAMP protocol should also consider protection against denial-
         of-service (DoS) and distributed denial-of-service (DDoS)
         attacks, where malicious users could flood the system with
         excessive data, causing network congestion and delaying or
         disrupting data transmission.  To mitigate such attacks, CAMP
         should implement rate limiting and congestion control mechanisms
         as defined in the MPQUIC specification.

     5.  *Trust and Privacy Considerations*
         Since CAMP is designed to handle multimodal data streams, it is
         important to consider the privacy of the data being transmitted.
         Any sensitive data, such as personally identifiable information
         (PII), should be encrypted and anonymized when possible.
         Additionally, careful access controls should be implemented to
         ensure that only authorized entities can access the data.

     These are just some of the key security considerations related to the
     CAMP protocol.  Future versions of the protocol may introduce
     additional security measures as new threats emerge.

  6.  IANA Considerations

     This document does not define any new protocol numbers, port numbers,
     or other IANA-managed resources.  However, future revisions of the
     CAMP protocol may define new types, identifiers, or registry entries,
     in which case the relevant considerations will be added.

     If any new IANA registries or assignments are required in future
     versions of this document, the authors will request the necessary
     changes from IANA.

  7.  References

  7.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/info/rfc8174>.

  7.2.  Informative References





  Song, et al.            Expires 3 September 2026               [Page 14]


  Internet-Draft                    camp                        March 2026


     [RFC9000]  Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
                Multiplexed and Secure Transport", RFC 9000,
                DOI 10.17487/RFC9000, May 2021,
                <https://www.rfc-editor.org/info/rfc9000>.

     [RFC9221]  Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable
                Datagram Extension to QUIC", RFC 9221,
                DOI 10.17487/RFC9221, March 2022,
                <https://www.rfc-editor.org/info/rfc9221>.

  Authors' Addresses

     Yuhong Song
     Pengcheng Laboratory
     Email: songyh@pcl.ac.cn


     Changlong Li
     East China Normal University
     Email: clli@cs.ecnu.edu.cn


     Qing Li
     Pengcheng Laboratory
     Email: liq@pcl.ac.cn


























  Song, et al.            Expires 3 September 2026               [Page 15]


name: draft-li-iotops-intelligent-security
rev: 00
title: Intelligent Protection Optimization System for IOT
time: 2024-09-01T07:05:09Z
group_uri: /api/v1/group/group/1027/
expires: 2024-09-01T09:24:01Z
states: ["/api/v1/doc/state/2/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.325279+00:00
source: ietf
abstract:
  Communication technology is becoming more and more developed, the
     Internet of Things coverage is becoming more and more
     comprehensive, and a large number of data and devices are
     joining, which also makes more data security and privacy
     issues appear.
     Therefore, this draft proposes a scheme to build an
     information-centered network. By analyzing common network
     attack methods, an intelligent protection
     optimization system is established from three aspects: naming and
     parsing, data exchange, and data caching, so as to achieve
     better content privacy protection without adding additional costs.

full_text:
  IOTOPS                                                       Xinru Li
  Internet-Draft                                               Yuyin Ma
  Intended status: Informational                         Guangshuo Chen
  Expires: 29 August 2024                              29 February 2024                              


          Intelligent Protection Optimization System for IOT
                  draft-li-iotops-intelligent-security-00

  Abstract

     Communication technology is becoming more and more developed, the 
     Internet of Things coverage is becoming more and more 
     comprehensive, and a large number of data and devices are 
     joining, which also makes more data security and privacy 
     issues appear. 
     Therefore, this draft proposes a scheme to build an 
     information-centered network. By analyzing common network 
     attack methods, an intelligent protection
     optimization system is established from three aspects: naming and
     parsing, data exchange, and data caching, so as to achieve 
     better content privacy protection without adding additional costs.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six 
     months and may be updated, replaced, or obsoleted by other 
     documents at any time.  It is inappropriate to use Internet-Drafts
     as reference material or to cite them other than as 
     "work in progress."

     This Internet-Draft will expire on 21 August 2024.

  Copyright Notice

     Copyright (c) 2024 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.


  Li, et al.              Expires 29 August 2024                [Page 1]
  Internet-Draft           Intelligent Protection          February 2024

     Please review these documents carefully, as they describe your
     rights and restrictions with respect to this document.
     Code Components extracted from this document 
     must include Revised BSD License text as 
     described in Section 4.e of the Trust Legal Provisions
     and are provided without warranty as described in
     the Revised BSD License.

  Table of Contents

     1. Introduction
     2. Current Situation and Problems
        2.1. Terminal Layer
        2.2. Transport Layer
        2.3. Processing Layer
     3. Principle
        3.1. Design Principle
        3.2. Typical Characteristics
           3.2.1. Cache Network
           3.2.2. Authenticity of Information
           3.2.3. Seamless Migration
           3.2.4. Space Expansion
           3.2.5. Flexible and Changeable
     4. System Design
        4.1. Naming and Parsing
        4.2. Data Exchange
        4.3. Data Caching
     5. Security Considerations
     6. IANA Considerations
     7. Acknowledgments
     8. References
        8.1. Normative References
        8.2. Informative References
     Authors' Addresses

  1.  Introduction

     With the penetration of new generation information and 
     communication technologies such as artificial intelligence,
     blockchain, and 5G communication into all areas of society,
     various types of intelligent applications and devices have
     also emerged, gradually entering the era of the
     Internet of everything. The application range of Internet
     of Things devices and systems is very wide, and the
     characteristics of diversified equipment, diversified
     environment, and massive real-time information 
     also make security issues the core of Internet of Things
     systems.


  Li, et al.              Expires 29 August 2024                [Page 2]
  Internet-Draft           Intelligent Protection          February 2024

     The original data interaction mode uses location as connection
     to deliver content, but often users are only interested in the
     content itself. Therefore, Inter centre Network (ICN) provides
     a relatively new network working mode. ICN realizes 
     nformation search and transmission through content index,
     which will effectively simplify the network structure and 
     improve the security and reliability of the system.

     To address the issue of content privacy protection in iot systems
     we have designed an intelligent scheme that can perform 
     security assessments on specific users and combine
     the potential of ICN networks to increase the security of content
     privacy without incuring significant overhead.

  2. Current Situation and Problems

     The overall security architecture of the Internet of Things can
     roughly include the physical and information collection
     security layer, transmission security layer and processing
     security layer of the terminal sensor network, which all
     contain security risks.

     2.1. Terminal Layer

        Due to the large number of devices and simple defense
        mechanisms, the devices are vulnerable to attacks, weak
        identity authentication and authorization mechanisms,
        and lack necessary security defense capabilities.

     2.2. Transport Layer

        Massive data transmission is easy to be stolen or
        tampered with, and attackers can also use massive data
        to extract statistical characteristics and analyze users.

     2.3. Processing Layer

        There are many types of equipment data, the calculation
        network is complex and changeable, and the data
        reliability is low. And the Internet of Things applications
        are diverse, may produce malicious program attacks.

  3. Principle

     3.1. Design Principle




  Li, et al.              Expires 29 August 2024                [Page 3]
  Internet-Draft           Intelligent Protection          February 2024

        Add the necessary security mechanisms to separate content and
        location, simplify the addressing process, and turn the network
        into a pure content web.

     3.2. Typical Characteristics

        3.2.1. Cache Network

           All intermediate nodes support caching, and in the case of
           a cache in the network, users do not have to wait for the
           network to forward data from the original node, but can
           obtain information from the nearest node with cached data.

        3.2.2. Authenticity of Information

           The system can encrypt the data that needs to be
           guaranteed by the administrator's permission, so that
           the user can ensure the authenticity of the information.

        3.2.3. Seamless Migration

           Modify different types of cache schemes, plan the data
           exchange of related nodes, and plan the priority, so as to
           avoid service interruption during migration.

        3.2.4. Space Expansion

           Avoid letting the terminal bear the massive data storage
           alone, make full use of rich network equipment, and realize
           the simple terminal to independently receive or send the
           data of interest to the superior.

        3.2.5. Flexible and Changeable

           It avoids the fixation of traditional network location
           and content binding, focuses on the exchange of
           data, and can reduce the possibility of being attacked
           by flexible nodes.

  4. System Design

        The system is built from three aspects, giving full play to the
        potential of ICN and increasing the connection between
        ICN and the Internet of Things.





  Li, et al.              Expires 29 August 2024                [Page 4]
  Internet-Draft           Intelligent Protection          February 2024

     4.1. Naming and Parsing

        The forwarded data needs to include a description of the
        content and the name, and since the information grows
        too fast, the name needs to be short enough to
        accommodate the forwarding capability. The analysis
        is divided into absolute and relative concentration two kinds.

     4.2. Data Exchange

        The system has three basic data exchange modes,
        including center mode, flooding mode and ideal mode.
        The central mode requires the existence of an omniscient
        supernode, the flooding mode can waste a lot of
        bandwidth, and the ideal mode is very complex.

     4.3. Data Caching

        The system will intelligently manage cache data
        and optimize data selection, node selection, time
        selection and mode selection.

  5. Security Considerations

     This document does not contain any security considerations.

  6. IANA Considerations

     This document makes no IANA requests.

  7. Acknowledgements

     The creation of this document has been a collaborative effort, 
     and we extend our gratitude to individuals and organizations 
     whose contributions and insights have enriched the content 
     and quality of this work.

  8. References

     8.1. Normative References

     [IEEE]    M. Cao et al., "Toward On-Device Federated 
               Learning: A Direct Acyclic Graph-Based Blockchain
               Approach", IEEE Trans. Neural Networks and 
               Learning Systems, pp. 1-15.




  Li, et al.              Expires 29 August 2024                [Page 5]
  Internet-Draft           Intelligent Protection          February 2024

     8.2. Informative References

     [IEEE]    F. Song, Y. Ma, Z. Yuan, I. You, G. Pau and
               H. Zhang, "Exploring Reliable Decentralized
               Networks with Smart Collaborative Theory," in
               IEEE Communications Magazine, vol. 61,
               no. 8, pp. 44-50, August 2023,
               doi: 10.1109/MCOM.003.2200443.

  Authors' Addresses

     Xinru Li
     BeiJing JiaoTong University
     Haidian District, Beijing
     Email: 20211011@bjtu.edu.cn
     
     Yuyin Ma
     BeiJing JiaoTong University
     Haidian District, Beijing
     Email: mayuyin@bjtu.edu.cn
     
     Guangshuo Chen
     BeiJing JiaoTong University
     Haidian District, Beijing
     Email: 17733652726@163.com






  Li, et al.              Expires 29 August 2024                [Page 6]


name: draft-yang-spring-sr-policy-intelligent-routing
rev: 03
title: Intelligent Routing Method of SR Policy
time: 2025-06-15T07:09:00Z
group_uri: /api/v1/group/group/1027/
expires: 2025-06-15T12:07:40Z
states: ["/api/v1/doc/state/4/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.328628+00:00
source: ietf
abstract:
  Segment Routing is a source routing paradigm that explicitly
     indicates the forwarding path for packets at the ingress node.  An
     SR Policy is associated with one or more candidate paths, and each
     candidate path is either dynamic, explicit or composite. This
     document describes an intelligent routing method for SR Policy based
     on network quality in MPLS and IPv6 environments.

full_text:
  SPRING Working Group                                            F. Yang
  Internet Draft                                             China Mobile
  Intended status: Standards Track                                 C. Lin
  Expires: June 12, 2025                                           Y. Qiu
                                                     New H3C Technologies
                                                        December 12, 2024



                    Intelligent Routing Method of SR Policy
                  draft-yang-spring-sr-policy-intelligent-routing-03


  Status of this Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF), its areas, and its working groups.  Note that
     other groups may also distribute working documents as Internet-
     Drafts.

     Internet-Drafts are draft documents valid for a maximum of six
     months and may be updated, replaced, or obsoleted by other documents
     at any time.  It is inappropriate to use Internet-Drafts as
     reference material or to cite them other than as "work in progress."

     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt

     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html

     This Internet-Draft will expire on June 12, 2025.

  Copyright Notice

     Copyright (c) 2024 IETF Trust and the persons identified as the
     document authors. All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents
     (http://trustee.ietf.org/license-info) in effect on the date of
     publication of this document. Please review these documents
     carefully, as they describe your rights and restrictions with
     respect to this document. Code Components extracted from this
     document must include Simplified BSD License text as described in



  Yang, et al.           Expire June 12, 2025                  [Page 1]


  Internet-Draft    SR Policy Intelligent Routing Method   December 2024


     Section 4.e of the Trust Legal Provisions and are provided without
     warranty as described in the Simplified BSD License.

  Abstract

     Segment Routing is a source routing paradigm that explicitly
     indicates the forwarding path for packets at the ingress node.  An
     SR Policy is associated with one or more candidate paths, and each
     candidate path is either dynamic, explicit or composite. This
     document describes an intelligent routing method for SR Policy based
     on network quality in MPLS and IPv6 environments.

  Table of Contents


     1. Introduction ................................................ 2
     2. Terminology ................................................. 3
     3. Background and Requriements ................................. 3
     4. Intelligent Routing Method for SR Policy .................... 5
        4.1. Processing Model ....................................... 5
        4.2. Flow Classification .................................... 6
        4.3. Flow Steering .......................................... 6
        4.4. Intelligent Routing .................................... 6
        4.5. Network Quality Measurement ............................ 8
        4.6. Flow Forwarding......................................... 9
     5. Examples of intelligent routing ............................. 9
     6. IANA Considerations ........................................ 11
     7. Security Considerations .................................... 11
     8. References ................................................. 11
        8.1. Normative References .................................. 11
        8.2. Informative References ................................ 11
     9. Acknowledgments ............................................ 12
     Authors' Addresses ............................................ 13

    1. Introduction

     Segment routing (SR) [RFC8402] is a source routing paradigm that
     explicitly indicates the forwarding path for packets at the ingress
     node. The ingress node steers packets into a specific path according
     to the Segment Routing Policy (SR Policy) as defined in [RFC9256].
     In order to distribute SR policies to the headend, [I-D.ietf-idr-
     segment-routing-te-policy] specifies a mechanism by using BGP.

     An SR Policy is associated with one or more candidate paths. A
     composite candidate path acts as a container for grouping SR
     Policies.  As described in section 2.2 in [RFC9256], the composite
     candidate path construct enables combination of SR Policies, each
     with explicit candidate paths and/or dynamic candidate paths with

  Yang, et al.            Expires June 12,2025                [Page 2]


  Internet-Draft    SR Policy Intelligent Routing Method       June 2024


     potentially different optimization objectives and constraints, for
     load-balanced steering of packet flows over its constituent SR
     Policies. For convenience, the composite candidate path formed by
     the combination of SR policies is called parent SR policy in [I-
     D.jiang-spring-parent-sr-policy-use-cases].

     This document describes an intelligent routing method for SR Policy
     based on network quality in MPLS and IPv6 environments.

    2.  Terminology

     The definitions of the basic terms are identical to those found in
     Segment Routing Policy Architecture [RFC9256] and [I-D.jiang-spring-
     parent-sr-policy-use-cases].

    3. Background and Requriements

     In single SR Policy, there are many mechanism provide
     failure/degrade protection, such as TILFA, VPN FRR. However, it is
     not clear how to handle failure or degradation between multiple SR
     policies.

     Take the networking shown in Figure 1 below as an example to
     illustrate the current problems.

     CE1 and CE2 are the two access endpoints of the IP telecom network.
     There are many service flows between CE1 and CE2 that have different
     requirements for forwarding quality. E.g. OA and voice traffic have
     different SLA requirement, and were carried by different SR policies.
     Generally, from CE1 to CE2, voice services with low latency
     requirements are forwarded along the highly reliable path PE1->PE2-
     >CE2. The OA traffic is forwarded along the high bandwidth path PE3-
     >P5->P6->PE2->CE2. When failure or degradation happened in OA
     traffic SR policy, there should be possible to assure basic
     communication for OA traffic by using voice bandwidth.













  Yang, et al.            Expires June 12,2025                [Page 3]


  Internet-Draft    SR Policy Intelligent Routing Method       June 2024


                              +---------------+
                              |   Controller  |
                              +---------------+

                               +------+     +------+
                     +---------+  P1  +-----+  P2  +----------+
                     |         +---+--+     +---+--+          |
                 +---+--+          |   \  /     |             |
         +-------+  PE1 |          |    \/      |             |
         |       +---+--+          |    /\      |             |
      +--+--+        |         +---+--+/  \ +---+--+      +---+--+   +-----+
      | CE1 |        +---------+  P3  +-----+  P4  +------+  PE2 +---+ CE2 |
      +--+--+                  +------+     +------+      +---+--+   +-----+
         |                                                    |
         |       +------+      +------+     +------+          |
         +-------+  PE3 +------+  P5  +-----+  P6  +----------+
                 +------+      +------+     +------+

                            Figure 1

     Based on such scenarios, the following requirements are proposed:

     a) Maximize failure/degradation protection

       In case of failure or degradation detected on one SR policy, it
       should be possible to do inter-policy protection.

     b) Minimal impact after taking repairing action

       Repair action can be done on flow level to minimize the ripple
       effect cause by bandwidth spillover.

     c) Maximize bandwidth efficiency

       For some critical applications, it should be possible to spillover
       from high class to lower class policy in case of degradation.

     In order to better meet these requirements, this document proposes
     an intelligent routing method for SR policy based on network quality
     requirement. The head end node selects the optimal path according to
     the current network quality to improve the path switching speed and
     forwarding performance.

     Refer to [I-D.jiang-spring-sr-policy-group-use-cases], the services
     with different forwarding quality requirements to the same
     destination endpoint can be implemented through parent SR Policy
     group.


  Yang, et al.            Expires June 12,2025                [Page 4]


  Internet-Draft    SR Policy Intelligent Routing Method      March 2024


     Define a parent SR Policy group for the above CE1 to CE2 services.
     Specify the steering policies of services in the parent SR Policy
     group. Different services can select different SR Policy paths in
     the parent SR Policy group according to different quality
     requirements. When the head node perceives that the quality of the
     path of a service is deteriorating (such as bandwidth or delay
     degradation), it searches for other path in the group that is
     suboptimal and also meets its quality requirements.

    4. Intelligent Routing Method for SR Policy

  4.1. Processing Model

     The path priority is assigned to the SR policy forwarding path
     manually by the controller. Each path with quality requirement will
     be assigned with a priority value. The lower the value, the higher
     the priority. That is, when there is a group of qualified paths,
     best path will be selected with higher priority.

     Configure multiple SR policy paths for the service flows with
     specified characteristics in the parent SR Policy group. Assign the
     corresponding path priority to each path according to the priority
     order of the path. If the traffic needs to be shared by multiple SR
     policies, assign the same priority and sharing weight values to
     these SR policies. If there is a backup path for the SR policies,
     lower priority value should be used according to the quality
     requirements.

     After receiving the service packet with the specified
     characteristics, when the network quality is good, the traffic is
     forwarded through the path with high priority. When the network
     quality degradation is happened on the high priority path, such as
     the packet loss rate exceeds the acceptable range, switch to the
     next high priority path of the service.

     If the quality of the high priority forwarding path is restored and
     the specified quality requirements are met, the traffic is switched
     from the low priority forwarding path to the high priority
     forwarding path.

     When there are multiple paths with the same priority, the traffic
     will share the bandwidth on these paths with the same priority
     according to the weight value.

     According to the processing logic, the SR policy intelligent routing
     model can be divided into five units, including Flow Classification,
     Flow Steering, Intelligent Routing, Flow Forwarding, and Network
     Quality Measurement, as shown in Figure 2 below.

  Yang, et al.            Expires June 12,2025                [Page 5]


  Internet-Draft    SR Policy Intelligent Routing Method      March 2024


     +----------------+  +----------+  +-------------+  +------------+
     |                |  |          |  |             |  |            |
     |       Flow     |->|  Flow    |->| Intelligent |->|    Flow    |
     | Classification |  | Steering |  |  Routing    |  | Forwarding |
     |                |  |          |  |             |  |            |
     +----------------+  +----------+  +------+------+  +------------+
                                              ^
                                              |
                                    +-----------------+
                                    |                 |
                                    | Network Quality |
                                    |   Measurement   |
                                    |                 |
                                    +-----------------+

                            Figure 2

     The functions of each unit are described below.

  4.2. Flow Classification

     After receiving the traffic, the head node first needs to label the
     traffic with forwarding class according to classification
     configuration.

     The head node can match flow characteristics in its ingress
     interfaces (upon any field such as Ethernet
     destination/source/VLAN/TOS or IP destination/source/DSCP or
     transport ports or application attribute etc.) and color them with
     an internal per-packet forwarding-class variable.

  4.3. Flow Steering

     According to the forwarding class variables determined by the Flow
     classification, the header node selects the matching forwarding path,
     that is, selects the SR policy or the parent SR policy representing
     a group of policies.

     If multiple SR policy forwarding paths are configured for the
     traffic flow with the specified characteristics, all valid SR
     policies will be retrieved and handed over to the Intelligent Routing
     unit to select the optimal forwarding path.

  4.4. Intelligent Routing

     According to the SR policy(policies) provided by Flow Steering, the
     Intelligent Routing unit obtains the current quality of each SR
     policy path from the Network Quality Measurement unit. Based on the

  Yang, et al.            Expires June 12,2025                [Page 6]


  Internet-Draft    SR Policy Intelligent Routing Method      March 2024


     mapping between the quality and the priority of intelligent routing,
     it selects the forwarding path with the highest priority and the
     best quality requirements for the traffic flow.

                     +------------------------------------------------------+
                     |                            +-----------------------+ |
                     |                            |     SR Policy         | |
                     |     +-----------------+    |   (high priority)     | |
                     |     |   +-----------+ |    |       +-------------+ | |
                     |     |   |  Normal   +----->+       | Active Path | | |
                     |     |   |  Flows    | |    |       +-------------+ | |
                     |     |   +-----------+ |    |       +-------------+ | |
       +--------+    |     |                 |    |       | Standby Path| | |
       |        |    |     |                 |    |       +-------------+ | |
       |Grouping|    |     |                 |    +-------------------+---+ |
       |   SR   +--->+     |     Policy      |                       /      |
       | Policy |    |     |    Decision     +<---Measurement Data -+       |
       |        |    |     |                 |                       \      |
       |        |    |     |                 |    +-------------------+---+ |
       +--------+    |     |                 |    |       +-------------+ | |
                     |     |   +-----------+ |    |       | Active Path | | |
                     |     |   | Spillover +----->+       +-------------+ | |
                     |     |   |   Flows   | |    |       +-------------+ | |
                     |     |   +-----------+ |    |       | Standby Path| | |
                     |     +-----------------+    |       +-------------+ | |
                     |                            |      SR Policy        | |
                     |                            |   (lower priority)    | |
                     |                            +-----------------------+ |
                     +------------------------------------------------------+
                              Figure 3

     When the network quality is good, the traffic is forwarded by the
     policy with high priority. When the network quality of the high
     priority SR policy degrades, such as the remaining bandwidth is
     insufficient, switch to the secondary high priority SR policy path
     with enough remaining bandwidth. Similarly, when the next higher
     priority SR policy forwarding path cannot meet the forwarding
     quality requirements, switch to the lower priority SR policy path.

     If the quality of the high priority forwarding path gets better and
     meets the specified quality requirements, the traffic can be
     recovered from the low priority forwarding path to the high priority
     SR policy forwarding path.

     If multiple SR policies in the parent SR policy have the same
     priority, when the traffic flow selects the path of this priority,
     the traffic flow will share the load on these SR policy paths
     according to the weight value.

  Yang, et al.            Expires June 12,2025                [Page 7]


  Internet-Draft    SR Policy Intelligent Routing Method       June 2024


     In order to better control the switchover process of SR policies
     between different priorities, the quality threshold and switchover
     waiting time can be specified. If the value of network quality
     degradation consistently exceeds the switchover threshold during the
     waiting time, select the SR policy with the next highest priority
     from the highest priority to the lowest. In contrast, if the network
     quality continues to improve, after exceeding the failback waiting
     time, the forwarding path can be switched to a high priority SR
     policy that meets the current network quality requirements.

     To avoid frequent path switching when the network quality is
     unstable, if the current path can meet the forwarding quality
     requirements, the head node can choose not to automatically switch
     back to the higher priority path in case of the quality of the
     higher priority path is restored. The device can provide a
     configuration for automatic failback, and add a restore waiting
     timer. Only after automatic restore is allowed and the restore
     waiting timer is timeout, the forwarding path switch from the
     current path that meets the quality requirements to the path with
     higher priority.

  4.5. Network Quality Measurement

     The Network Quality Measurement unit regularly monitors the quality
     of all effective forwarding paths according to the measurement cycle,
     records the current performance measurement data of the path, and
     reports it to the Intelligent Routing unit, which decides whether to
     switch paths.

     The following network quality parameters of forwarding path can be
     used for path scheduling:

       * Jitter

       * Latency

       * Packet loss

       * Available bandwidth

       * Bandwidth utilization

       * Current traffic statistics

       * Other forwarding performance parameters

     The quality parameters of network forwarding path can be obtained
     through active or passive performance measurement methods, such as

  Yang, et al.            Expires June 12,2025                [Page 8]


  Internet-Draft    SR Policy Intelligent Routing Method      March 2024


     iOAM, STAMP, TWAMP, etc. The network quality parameters can be
     calculated by the controller and distributed to the head end node,
     or calculated by the head end node according to the network
     measurement data. The measurement method and quality parameter
     acquisition method are beyond the scope of this document.

  4.6. Flow Forwarding

     The service flow is forwarded according to the path determined by
     the Intelligent Routing unit.

     When there are multiple paths with the same priority, the traffic
     will share the load among these SR policy paths with the same
     priority according to the weight value.

    5. Examples of intelligent routing

     The application of intelligent routing is described in detail in
     L3VPN over TE scenario. The networking is shown in Figure 4 below.

     CE1 and CE2 belong to the same L3VPN and access the public network
     through PE1, PE2 and PE3 respectively.

     There are two services between CE1 and CE2: voice and OA. The
     traffic from CE1 to CE2 can be forwarded through two paths: Path1
     (PE1->PE2->CE2) and Path2 (PE3->P5->P6->PE2->CE2). Among them, the
     reliability of path 1 is high and the transmission delay is low.
     Path 2 has a large bandwidth.

     The voice service traffic will be forwarded through Path1 first. The
     OA service traffic will be forwarded through Path2 first. When the
     transmission delay of Path1 exceeds the threshold value and Path2
     can meet the delay requirements, switch the voice service to Path2.

     When the remaining bandwidth of Path2 is less than the bandwidth
     guarantee threshold, if Path1 still has enough remaining bandwidth,
     the OA traffic exceeding the bandwidth will be directed to Path1.











  Yang, et al.            Expires June 12,2025                [Page 9]


  Internet-Draft    SR Policy Intelligent Routing Method      March 2024


                               +------+     +------+
                     +---------+  P1  +-----+  P2  +----------+
                     |         +---+--+     +---+--+          |
                 +---+--+          |   \  /     |             |
         +-------+  PE1 |          |    \/      |             |
         |       +---+--+          |    /\      |             |
      +--+--+        |         +---+--+/  \ +---+--+      +---+--+   +-----+
      | CE1 |        +---------+  P3  +-----+  P4  +------+  PE2 +---+ CE2 |
      +--+--+                  +------+     +------+      +---+--+   +-----+
         |                                                    |
         |       +------+      +------+     +------+          |
         +-------+  PE3 +------+  P5  +-----+  P6  +----------+
                 +------+      +------+     +------+
                              Figure 4

     The configuration on the head node CE1 includes the following three
     parts. These configurations can be directly configured on the node
     or distributed through the controller.

     1. Define three intelligent routing policies, and specify the
        threshold of network quality, path priority and the corresponding
        path color value for routing.

         intelligent-routing-policy irp1

           traffic-delay threshold 1000ms

           priority 1 mapping-to color 100

           priority default mapping-to color 200

         intelligent-routing-policy irp2

           remaining-bandwidth threshold 50M

           priority 1 mapping-to color 200

           priority default mapping-to color 100

     2. Configure forwarding paths.

         sr-policy policy-A (color 100, CE2_SID)

           segment-list <SID_PE1, SID_PE2, SID_CE2>

         sr-policy policy-B (color 200, CE2_SID)

           segment-list <SID_PE3, SID_P5, SID_P6, SID_PE2, SID_CE2>

  Yang, et al.           Expires March,  2024               [Page 10]


  Internet-Draft    SR Policy Intelligent Routing Method   June 2024


     3. Configure corresponding intelligent routing policies for services
        with specified characteristics in the parent SR Policy group.

         parent-sr-policy sr-policy-1(color 10, CE2_SID)

          service voice use intelligent-routing-policy irp1

          service oa use intelligent-routing-policy irp2

    6. IANA Considerations

     This document has no IANA actions.

    7. Security Considerations

     To prevent CE from arbitrarily designating intermediate network forwarding
     nodes without restriction, the following method can be adopted to verify
     the legality of the path. Verification of SRv6 path legitimacy is conducted
     on PE/P nodes through a verification table, with each entry including the 
     token value and its corresponding verification information. The token
     serves as the index of the table, and its value could represent a path
     through a path SID. Verification information includes properties that
     determine forwarding actions, such as the legitimate incoming interface for
     the path and the next hop SID. SRv6 packets sent by user-side equipment
     (CE) need to carry the token in the innermost layer. Upon receiving the
     packet, the PE node extracts the token value from the innermost layer of
     the SID List to look up verification table information. If the packet does
     not include a token, the packet is discarded. If a matching result is
     found, the node checks whether the packet's incoming interface matches the
     incoming interface in the verification table entry, and whether the next
     hop address matches the next SID in the packet's SID List. Depending on the
     level of verification, forwarding only occurs when either the incoming
     interface, the next hop, or both match.

  8. References

  8.1. Normative References

     [I-D.ietf-idr-segment-routing-te-policy] Previdi, S., Filsfils, C.,
               Talaulikar, K., Mattes, P., Rosen, E., Jain, D., and S.
               Lin, "Advertising Segment Routing Policies in BGP", draft-
               ietf-idr-segment-routing-te-policy-20 (work in progress),
               July 2022.

     [I-D.jiang-spring-parent-sr-policy-use-cases] Jiang, W., Cheng, W.,
               Lin, C. and Qiu, Y., "Use Cases for Parent SR Policy",
               draft-jiang-spring-parent-sr-policy-use-cases-01 (work in
               progress), January 2023.

     [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
               Decraene, B., Litkowski, S., and R. Shakir, "Segment
               Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
               July 2018, <https://www.rfc-editor.org/info/rfc8402>.

     [RFC9256] Filsfils, C., Talaulikar, K., Voyer, D., Bogdanov, A., and
               P. Mattes, "Segment Routing Policy Architecture", RFC 9256,
               DOI 10.17487/RFC9256, July 2022, <https://www.rfc-
               editor.org/info/rfc9256>.

  8.2. Informative References

     TBD




  Yang, et al.            Expires June 12,2025               [Page 11]


  Internet-Draft    SR Policy Intelligent Routing Method      March 2024


    9. Acknowledgments

     The authors would like to thank the following for their valuable
     contributions of this document:

     TBD










































  Yang, et al.            Expires June 12,2025               [Page 12]


  Internet-Draft    SR Policy Intelligent Routing Method       June 2024


  Authors' Addresses

     Feng Yang
     China Mobile
     Beijing
     China
     Email: yangfeng@chinamobile.com

     Changwang Lin
     New H3C Technologies
     Beijing
     China

     Email: linchangwang.04414@h3c.com


     Yuanxiang Qiu
     New H3C Technologies

     Email: qiuyuanxiang@h3c.com




























  Yang, et al.            Expires June 12,2025               [Page 13]



name: draft-doujiali-cloudnetwork-intelligentoperation
rev: 00
title: An requirement of Cloud Network Intelligent Operation and Maintenance.
time: 2025-04-20T07:05:08Z
group_uri: /api/v1/group/group/1027/
expires: 2025-04-21T02:18:42Z
states: ["/api/v1/doc/state/2/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.333236+00:00
source: ietf
abstract:
  This document intends to serve as a standard for
  Cloud Network Intelligent Operation and Maintenance.

full_text:
  Intelligent Operation                                             J. Dou
  Internet-Draft                                                     Y. Su
  Intended status: Standards Track                                   Z. Li
  Expires: 30 April 2025                                           R. Chen
                                                                     CAICT
                                                           16 October 2024

  INTERNET-DRAFT

  An requirement of Cloud Network Intelligent Operation and Maintenance.

  draft-doujiali-cloudnetwork-intelligentoperation-00



  Abstract
  This document intends to serve as a standard for 
  Cloud Network Intelligent Operation and Maintenance.

  Status of This Memo
  This Internet-Draft is submitted in full conformance with the
  provisions of BCP 78 and BCP 79.
  Internet-Drafts are working documents of the
  Internet Engineering Task Force (IETF). 
  Note that other groups may also distribute working documents
  as Internet-Drafts.  
  The list of current Internet-Drafts is at 
  https://datatracker.ietf.org/drafts/current/.

  Internet-Drafts are draft documents valid for
  a maximum of six months and may be updated, replaced,
  or obsoleted by other documents at any time.  
  It is inappropriate to use Internet-Drafts as reference material
   or to cite them other than as "work in progress."

  This Internet-Draft will expire on 30 April 2025.


  Copyright Notice
  Copyright (c) 2024 IETF Trust and the persons identified 
  as the document authors.  
  All rights reserved.

  This document is subject to BCP 78 
  and the IETF Trust's Legal Provisions Relating to IETF Documents 
  (https://trustee.ietf.org/license-info) in effect on 
  the date of publication of this document.
  Please review these documents carefully, 
  as they describe your rights and restrictions with 
  respect to this document.

  Table of contents



  1. Introduction	 
  2.O&M technical service capabilities
  3.Organizational service capabilities
  4.Security Considerations
  5.Cloud-network planning and deployment capabilities
  6.IANA Considerations
  7.References


   

  1 Introduction
  It includes the following aspects: data collection,
   O&M platform management, business management, 
  organization management, security capabilities, 
  high availability, and cloud network planning 
  and deployment capabilities.

  2 O&M technical service capabilities
  2.1Overview
  This metric is defined as an end-to-end data collection capability, 
  which includes hardware, network, operating system, middleware, 
  application metrics, logs, events, call traces, and other related data.

  2.2Data collection capabilities
  1)Supports the collection capability of underlay& overlay layer data, 
  including but not limited to CPU, memory, network card, I/O, etc.
  2)Supports the collection of network traffic performance indicators, 
  such as throughput, latency, jitter, packet loss rate, etc.;
  3)It supports the data collection, identification 
  and analysis of network layer protocols of network traffic,
  and the collection of protocol performance 
  and abnormal data of each layer.

  2.3O&M management capabilities
  2.3.1Resource management capabilities
  1)It supports unified management and automatic discovery of devices 
  and resources at the underlay and overlay layers.
  2)It supports unified management capabilities of middleware, 
  including but not limited to message middleware, 
  transaction middleware, application service middleware etc.
  3)On-demand quota management of computing resources 
  at the application level is supported. 
  2.3.2Data analysis capabilities
  1)Support the whole process management of data analysis.
  2)Load analysis of computing resources, network devices, 
  virtual network nodes, and network links.
  2.3.3Monitoring and visualization capabilities
  1)It supports real-time monitoring of resources and devices, 
  including but not limited to the real-time running status of 
  computing resources, network resources, 
  storage resources, and security resources.
  2)Supports dynamic real-time perception and update of the relationship 
  between distributed applications, microservice applications, 
  and network resources on the cloud.
  2.3.4Configure management capabilities
  1)Supports change control management of configuration items, 
  such as managing configuration items from 
  the perspective of applications and network resources.
  2)Checksum error alarms can be configured.
  3)Correlation relationships between configuration 
  items can be automatically collected
  2.3.5Ability to manage policies
  1)It supports the design, addition, delivery, 
  and execution of automated policies.
  2)Supports the construction of basic configuration policy models.
  3)It supports the ability to generate configuration scripts, 
  deliver and execute policies in batches.

  2.4Business management capabilities
  2.4.1Intelligent alarm capability
  1)Multiple sources of alarm events are supported.
  2)Alarm rules can be configured and managed by classification.
  3)Alarm correlation detection and recording are supported.
  2.4.2Troubleshooting capabilities
  1)It supports fault classification and hierarchical management, 
  and builds models based on machine learning and deep learning.
  2)It supports the construction of a fault handling rule 
  library and the formation of a fault prediction model.
  3)It supports the ability to quickly deal with faults, 
  based on root cause location and recommended handling methods.
  2.4.3Intelligent optimization capabilities
  Optimize network performance configuration and automatically c
  ollect network performance-related data (including but not limited to 
  device operation information, network traffic logs, link topology 
  logs, and performance indicator logs) in real time.

  3 Organizational service capabilities
  3.1Personnel management capabilities
  1)Support basic information management of personnel 
  organizations, departments, and roles.
  2)Support shift management of whether the staff is 
  on duty and whether they are idle.
  3)Support the formation of character portraits based on the 
  characteristics of the person's history, work orders, problems 
  that he is good at handling.

  3.2Document management capabilities
  Write product description documents, FAQ documents or user help 
  manuals to ensure that the concept definition is accurate, 
  the applicable scenarios are clear, 
  and the operation process is standardized.

  4 Security Considerations
  4.1Infrastructure security
  Support infrastructure, including computing devices protection, 
  storage devices, network devices, and other devices. 

  4.2Data storage is secure
  1)Supports key data transmission and storage encryption.
  2)Supports the desensitization of user privacy sensitive data.
  3)Data backup and disaster recovery, 
  and data backup integrity verification are supported.

  4.3Access and exit security
  1)Data APIs can be used to invoke identity authorization verification.
  2)Data signature verification is supported.
  3)The timeout mechanism is supported.

  4.4Cybersecurity
  1)Supports anti-network attack mechanism 
  and intrusion protection mechanism.
  2)Network isolation is supported.

  5 Cloud-network planning and deployment capabilities
  5.1Cloud-network planning capabilities
  1)Supports intent analysis capabilities.
  2)It supports the configuration and rendering of software resources 
  and hardware resources based on the architecture model, 
  and dynamically adjusts the ability.
  3)Online and offline scaling capabilities are supported.

  5.2Cloud-network delivery capability
  1)Support automated delivery capabilities.
  2)The SAI layer is supported to abstract and unify 
  heterogeneous resources and shield hardware differentiation.
  3)The cloud-network software module supports 
  full-cycle software license management.

  6 IANA Considerations
  This document has no IANA actions.


  7.References

  7.1 Normative references to the reference documents

  The listed documents are essential provisions of this document. 
  Dated references apply only to the corresponding version. 
  Undated references refer to the latest version available.
  GB/T 32400-2015 
  Information Technology Cloud Computing Overview and vocabulary 

  7.2 Terms, definitions, and abbreviations

  GB/T 32400-2015  defined and the following terms, 
  definitions and abbreviation.

  7.2.1  Terms and definitions

  7.2.2 Public cloud : public cloud
  A cloud deployment model can be used by any cloud service customer.
  [ GB/T 32400-2015 ，3.2.33]

  7.2.3 Cloud Networking
  Cloud networking offers connectivity to and between applications 
  and workloads across clouds, cloud services, 
  on-premises data centers, and edge networks.

  7.3 Abbreviations 

  The following abbreviations apply to this document.


  O&M represents Operations &Maintenance.





  Acknowledgments


  Authors' Addresses

      Jiali. Dou 
     CAICT
     52 Huayuan North Road
     Beijing
     China
     Email: doujiali@caict.ac.cn

     Yue Su
     CAICT
     Email: suyue1@caict.ac.cn

     Zihan. Li 
     CAICT
     Email: lizihan1@caict.ac.cn

     Ruihao Chen 
     CAICT 
     Email: chenruihao@caict.ac.cn  

  	


name: draft-yang-srv6ops-intelligent-routing
rev: 00
title: Intelligent Routing Method of SR Policy
time: 2025-06-30T02:39:30Z
group_uri: /api/v1/group/group/1027/
expires: 2026-01-01T02:39:30Z
states: ["/api/v1/doc/state/4/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.336369+00:00
source: ietf
abstract:
  Segment routing (SR) [RFC8402] is a source routing paradigm that
     explicitly indicates the forwarding path for packets at the ingress
     node.  An SR Policy is associated with one or more candidate paths,
     and each candidate path is either dynamic, explicit or composite.
     This document describes an intelligent routing method for SR Policy
     based on network quality in MPLS and IPv6 environments.

full_text:
  srv6ops                                                          F. Yang
  Internet-Draft                                              China Mobile
  Intended status: Informational                                    C. Lin
  Expires: 1 January 2026                             New H3C Technologies
                                                              30 June 2025


                  Intelligent Routing Method of SR Policy
                 draft-yang-srv6ops-intelligent-routing-00

  Abstract

     Segment routing (SR) [RFC8402] is a source routing paradigm that
     explicitly indicates the forwarding path for packets at the ingress
     node.  An SR Policy is associated with one or more candidate paths,
     and each candidate path is either dynamic, explicit or composite.
     This document describes an intelligent routing method for SR Policy
     based on network quality in MPLS and IPv6 environments.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 1 January 2026.

  Copyright Notice

     Copyright (c) 2025 IETF Trust and the persons identified as the
     document authors.  All rights reserved.











  Yang & Lin               Expires 1 January 2026                 [Page 1]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
       1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
     3.  Problem and Requriements  . . . . . . . . . . . . . . . . . .   3
     4.  Intelligent Routing Method for SR Policy  . . . . . . . . . .   5
       4.1.  Processing Model  . . . . . . . . . . . . . . . . . . . .   5
       4.2.  Flow Classification . . . . . . . . . . . . . . . . . . .   6
       4.3.  Flow Steering . . . . . . . . . . . . . . . . . . . . . .   6
       4.4.  Intelligent Routing . . . . . . . . . . . . . . . . . . .   7
       4.5.  Network Quality Measurement . . . . . . . . . . . . . . .   8
       4.6.  Flow Forwarding . . . . . . . . . . . . . . . . . . . . .   9
     5.  Examples of intelligent routing . . . . . . . . . . . . . . .   9
     6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  11
     7.  Security Considerations . . . . . . . . . . . . . . . . . . .  11
     8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  11
       8.1.  Normative References  . . . . . . . . . . . . . . . . . .  11
       8.2.  Informative References  . . . . . . . . . . . . . . . . .  11
     Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  12
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  12

  1.  Introduction

     Segment routing (SR) [RFC8402] is a source routing paradigm that
     explicitly indicates the forwarding path for packets at the ingress
     node.  The ingress node steers packets into a specific path according
     to the Segment Routing Policy (SR Policy) as defined in [RFC9256].
     In order to distribute SR Policies to the headend,
     [I-D.ietf-idr-segment-routing-te-policy] specifies a mechanism by
     using BGP.

     An SR Policy is associated with one or more candidate paths.  A
     composite candidate path acts as a container for grouping SR
     Policies.  As described in section 2.2 in [RFC9256], the composite
     candidate path construct enables combination of SR Policies, each
     with explicit candidate paths and/or dynamic candidate paths with
     potentially different optimization objectives and constraints, for
     load-balanced steering of packet flows over its constituent SR



  Yang & Lin               Expires 1 January 2026                 [Page 2]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


     Policies.  For convenience, the composite candidate path formed by
     the combination of SR Policies is called parent SR Policy in
     [I-D.cheng-spring-sr-policy-group].

     This document describes an intelligent routing method for SR Policy
     based on network quality in MPLS and IPv6 environments.

  1.1.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in
     BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  2.  Terminology

     The definitions of the basic terms are identical to those found in
     Segment Routing Policy Architecture [RFC9256].

  3.  Problem and Requriements

     Take the networking shown in Figure 1 below as an example to
     illustrate the current problems.

     CE1 and CE2 are the two access endpoints of the IP telecom network.
     There are many service flows between CE1 and CE2 that have different
     requirements for forwarding quality.  E.g.  OA and voice traffic have
     different SLA requirement, and were carried by different SR Policies.
     Generally, from CE1 to CE2, voice services with low latency
     requirements are forwarded along the highly reliable path
     PE1->PE2->CE2.  The OA traffic is forwarded along the high bandwidth
     path PE3->P5->P6->PE2->CE2.  When failure or degradation happened in
     OA traffic SR Policy, there should be possible to assure basic
     communication for OA traffic by using voice bandwidth.

     In single SR Policy, there are many mechanism provide failure/degrade
     protection, such as TILFA, VPN FRR.  However, it is not clear how to
     handle failure or degradation between multiple SR Policies.












  Yang & Lin               Expires 1 January 2026                 [Page 3]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


                           +---------------+
                           |   Controller  |
                           +---------------+

                            +------+     +------+
                      +-----+  P1  +-----+  P2  +------+
                      |     +---+--+     +---+--+      |
                  +---+--+      |   \  /     |         |
             +----+  PE1 |      |    \/      |         |
             |    +---+--+      |    /\      |         |
          +--+--+     |     +---+--+/  \ +---+--+  +---+--+  +-----+
          | CE1 |     +-----+  P3  +-----+  P4  +--+  PE2 +--+ CE2 |
          +--+--+           +------+     +------+  +---+--+  +-----+
             |                                         |
             |    +------+  +------+     +------+      |
             +----+  PE3 +--+  P5  +-----+  P6  +------+
                  +------+      +------+     +------+

                                    Figure 1

     Based on such scenarios, the following requirements are proposed:

     1.  Maximize failure/degradation protection

         In case of failure or degradation detected on one SR policy, it
         should be possible to do inter-policy protection.

     2.  Minimal impact after taking repairing action

         Repair action can be done on flow level to minimize the ripple
         effect cause by forwarding path switchover.

     3.  Maximize bandwidth efficiency

         For some critical applications, it should be possible to forward
         the traffic over lower class policy in case of higher class SR
         Policy degradation.

     In order to better meet these requirements, this document proposes an
     intelligent routing method for SR policy based on network quality
     requirement.  The head end node selects the optimal path according to
     the current network quality to improve the path switching speed and
     forwarding performance.

     Refer to [I-D.cheng-spring-sr-policy-group], the services with
     different forwarding quality requirements to the same destination
     endpoint can be implemented through parent SR Policy group.




  Yang & Lin               Expires 1 January 2026                 [Page 4]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


     Define a parent SR Policy group for the above CE1 to CE2 services.
     Specify the steering policies of services in the parent SR Policy
     group.  Different services can select different SR Policy paths in
     the parent SR Policy group according to different quality
     requirements.  When the head node perceives that the quality of the
     path of a service is deteriorating (such as bandwidth or delay
     degradation), it searches for other path in the group that is
     suboptimal and also meets its quality requirements.

  4.  Intelligent Routing Method for SR Policy

  4.1.  Processing Model

     The path priority is assigned to the SR policy forwarding path
     manually by the controller.  Each path with quality requirement will
     be assigned with a priority value.  The lower the value, the higher
     the priority.  That is, when there is a group of qualified paths,
     best path will be selected with higher priority.

     Configure multiple SR policy paths for the service flows with
     specified characteristics in the parent SR Policy group.  Assign the
     corresponding path priority to each path according to the priority
     order of the path.  If there is a backup path for the SR policies,
     lower priority value should be used according to the quality
     requirements.

     After receiving the service packet with the specified
     characteristics, when the network quality is good, the traffic is
     forwarded through the path with high priority.  When the network
     quality degradation is happened on the high priority path, such as
     the packet loss rate exceeds the acceptable range, switch to the next
     high priority path of the service.

     If the quality of the high priority forwarding path is restored and
     the specified quality requirements are met, the traffic is switched
     from the low priority forwarding path to the high priority forwarding
     path after a period of wait-to-restore time.

     According to the processing logic, the SR policy intelligent routing
     model can be divided into five units, including Flow Classification,
     Flow Steering, Intelligent Routing, Flow Forwarding, and Network
     Quality Measurement, as shown in Figure 2 below.









  Yang & Lin               Expires 1 January 2026                 [Page 5]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


       +----------------+  +----------+  +-------------+  +------------+
       |                |  |          |  |             |  |            |
       |       Flow     +->|  Flow    +->| Intelligent +->|    Flow    |
       | Classification |  | Steering |  |  Routing    |  | Forwarding |
       |                |  |          |  |             |  |            |
       +----------------+  +----------+  +-------------+  +------------+
                                                ^
                                                |
                                      +---------+-------+
                                      |                 |
                                      | Network Quality |
                                      |   Measurement   |
                                      |                 |
                                      +-----------------+

                                    Figure 2

     The functions of each unit are described below.

  4.2.  Flow Classification

     After receiving the traffic, the head node first needs to label the
     traffic with forwarding class according to classification
     configuration.

     The head node can match flow characteristics in its ingress
     interfaces (upon any field such as Ethernet destination/source/VLAN/
     TOS or IP destination/source/DSCP or transport ports or application
     attribute etc.) and color them with an internal per-packet
     forwarding-class variable.

  4.3.  Flow Steering

     According to the forwarding class variables determined by the Flow
     classification, the header node selects the matching forwarding path,
     that is, selects the SR policy or the parent SR policy representing a
     group of policies.

     If multiple SR policy forwarding paths are configured for the traffic
     flow with the specified characteristics, all valid SR policies will
     be retrieved and handed over to the Intelligent Routing unit to
     select the optimal forwarding path.









  Yang & Lin               Expires 1 January 2026                 [Page 6]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


  4.4.  Intelligent Routing

     According to the SR policy(policies) provided by Flow Steering, the
     Intelligent Routing unit obtains the current quality of each SR
     policy path from the Network Quality Measurement unit.  Based on the
     mapping between the quality and the priority of intelligent routing,
     it selects the forwarding path with the highest priority and the
     quality measurement of the SR Policy.  Only those qulified SR
     Policies which can reach the threshold are considered as candidate SR
     Policies.

             +---------------------------------------------------+
             |                 Parent SR Policy                  |
             |                               +-----------------+ |
             |                               |    SR Policy    | |
             |                               | (high priority) | |
             |                               | +-------------+ | |
             |                               | | Active Path | | |
             |                               | +-------------+ | |
             |                 +----------+  | +-------------+ | |
             |                 |          |  | | Standby Path| | |
             |                 |          +->| +-------------+ | |
             | +------------+  |          |  +-------------+---+ |
             | | Classified |  |  Policy  |               /      |
             | |            +->| Decision |<-Measurement-+       |
             | |   Traffic  |  |          |               \      |
             | +------------+  |          |  + ------------+---+ |
             |                 |          +->| +-------------+ | |
             |                 |          |  | Active Path | | |
             |                 +----------+  | +-------------+ | |
             |                               | +-------------+ | |
             |                               | | Standby Path| | |
             |                               | +-------------+ | |
             |                               |    SR Policy    | |
             |                               | (lower priority)| |
             |                               + ----------------+ |
             +---------------------------------------------------+

                                    Figure 3

     When the network quality is better than the threshold, the traffic is
     forwarded by the policy with high priority.  When the network quality
     of the high priority SR Policy degrades, such as the loss rate
     increasion, the Policy Decision module will switch the traffic to the
     next high priority one in the candidate SR Policies.  Similarly, when
     the next higher priority SR Policy forwarding path cannot meet the
     forwarding quality requirements, switch to the lower priority SR
     Policy path.



  Yang & Lin               Expires 1 January 2026                 [Page 7]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


     If the quality of the high priority forwarding path gets better and
     meets the specified quality requirements, the traffic can be
     recovered from the low priority forwarding path to the high priority
     SR Policy forwarding path after a period of wait-to-restore time.
     The purpose of wait-to-restore time is specified in order to prevent
     flapping between SR Policies.

     To avoid frequent path switching when the network quality is
     unstable, if the current path can meet the forwarding quality
     requirements, the head node can choose not to automatically switch
     back to the higher priority path in case of the quality of the higher
     priority path is restored.  The device can provide a configuration
     for automatic failback, and add a wait-to-restore timer.  Only after
     automatic restore is allowed and the wait-to-restore timer is
     timeout, the forwarding path switch from the current path that meets
     the quality requirements to the path with higher priority.

  4.5.  Network Quality Measurement

     The Network Quality Measurement unit regularly monitors the quality
     of all effective forwarding paths according to the measurement cycle,
     records the current performance measurement data of the path, and
     reports it to the Intelligent Routing unit, which decides whether to
     switch paths.

     The following network quality parameters of forwarding path can be
     used for path scheduling:

     *  Jitter

     *  Latency

     *  Packet loss

     *  Available bandwidth

     *  Bandwidth utilization

     *  Current traffic statistics

     *  Other forwarding performance parameters










  Yang & Lin               Expires 1 January 2026                 [Page 8]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


     The quality parameters of network forwarding path can be obtained
     through active or passive performance measurement methods, such as
     iOAM, STAMP, TWAMP, etc.  The network quality parameters can be
     calculated by the controller and distributed to the head end node, or
     calculated by the head end node according to the network measurement
     data.  The measurement method and quality parameter acquisition
     method are beyond the scope of this document.

  4.6.  Flow Forwarding

     The service flow is forwarded according to the path determined by the
     Intelligent Routing unit.

     When there are multiple paths with the same priority, the traffic
     will share the load among these SR Policy paths with the same
     priority according to the weight value.

  5.  Examples of intelligent routing

     The application of intelligent routing is described in detail in
     L3VPN over TE scenario.  The networking is shown in Figure 4 below.

     CE1 and CE2 belong to the same L3VPN and access the public network
     through PE1, PE2 and PE3 respectively.

     There are two services between CE1 and CE2: voice and OA.  The
     traffic from CE1 to CE2 can be forwarded through two paths: Path1
     (PE1->PE2->CE2) and Path2 (PE3->P5->P6->PE2->CE2).  Among them, the
     reliability of path 1 is high and the transmission delay is low.
     Path 2 has a large bandwidth.

     The voice service traffic will be forwarded through Path1 first.  The
     OA service traffic will be forwarded through Path2 first.  When the
     transmission delay of Path1 exceeds the threshold value and Path2 can
     meet the delay requirements, switch the voice service to Path2.

     When the remaining bandwidth of Path2 is less than the bandwidth
     guarantee threshold, if Path1 still has enough remaining bandwidth,
     the OA traffic exceeding the bandwidth will be directed to Path1.












  Yang & Lin               Expires 1 January 2026                 [Page 9]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


                            +------+     +------+
                     +------+  P1  +-----+  P2  +-------+
                     |      +---+--+     +---+--+       |
                 +---+--+       |   \  /     |          |
           +-----+  PE1 |       |    \/      |          |
           |     +---+--+       |    /\      |          |
        +--+--+      |      +---+--+/  \ +---+--+   +---+--+   +-----+
        | CE1 |      +------+  P3  +-----+  P4  +---+  PE2 +---+ CE2 |
        +--+--+             +------+     +------+   +---+--+   +-----+
           |                                            |
           |     +------+   +------+     +------+       |
           +-----+  PE3 +---+  P5  +-----+  P6  +-------+
                 +------+   +------+     +------+

                                    Figure 4

     The configuration on the head node CE1 includes the following three
     parts.  These configurations can be directly configured on the node
     or distributed through the controller.

     1.  Define three intelligent routing policies, and specify the
         threshold of network quality, path priority and the corresponding
         path color value for routing.

        intelligent-routing-policy irp1
          traffic-delay threshold 1000ms
          priority 1 mapping-to color 100
          priority default mapping-to color 200
        intelligent-routing-policy irp2
          remaining-bandwidth threshold 50M
          priority 1 mapping-to color 200
          priority default mapping-to color 100

     2.  Configure forwarding paths.

        sr-policy policy-A (color 100, CE2_SID)
          segment-list <SID_PE1, SID_PE2, SID_CE2>
        sr-policy policy-B (color 200, CE2_SID)
          segment-list <SID_PE3, SID_P5, SID_P6, SID_PE2, SID_CE2>

     3.  Configure corresponding intelligent routing policies for services
         with specified characteristics in the parent SR Policy group.

        parent-sr-policy sr-policy-1(color 10, CE2_SID)
          service voice use intelligent-routing-policy irp1
          service oa use intelligent-routing-policy irp2





  Yang & Lin               Expires 1 January 2026                [Page 10]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


  6.  IANA Considerations

     This memo includes no request to IANA.

  7.  Security Considerations

     This document does not introduce any security considerations.

  8.  References

  8.1.  Normative References

     [RFC8402]  Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
                Decraene, B., Litkowski, S., and R. Shakir, "Segment
                Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
                July 2018, <https://www.rfc-editor.org/rfc/rfc8402>.

     [RFC9256]  Filsfils, C., Talaulikar, K., Ed., Voyer, D., Bogdanov,
                A., and P. Mattes, "Segment Routing Policy Architecture",
                RFC 9256, DOI 10.17487/RFC9256, July 2022,
                <https://www.rfc-editor.org/rfc/rfc9256>.

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/rfc/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

  8.2.  Informative References

     [I-D.ietf-idr-segment-routing-te-policy]
                Previdi, S., Filsfils, C., Talaulikar, K., Mattes, P., and
                D. Jain, "Advertising Segment Routing Policies in BGP",
                Work in Progress, Internet-Draft, draft-ietf-idr-segment-
                routing-te-policy-26, 23 October 2023,
                <https://datatracker.ietf.org/doc/html/draft-ietf-idr-
                segment-routing-te-policy-26>.

     [I-D.cheng-spring-sr-policy-group]
                Cheng, W., Wenying, J., Lin, C., Chen, R., Zhang, Y., and
                Y. Liang, "SR Policy Group", Work in Progress, Internet-
                Draft, draft-cheng-spring-sr-policy-group-08, 17 June
                2025, <https://datatracker.ietf.org/doc/html/draft-cheng-
                spring-sr-policy-group-08>.




  Yang & Lin               Expires 1 January 2026                [Page 11]


  Internet-Draft    SR Policy Intelligent Routing Method         June 2025


  Acknowledgements

     The authors would like to thank the following for their valuable
     contributions of this document.

     TBD.

  Authors' Addresses

     Feng Yang
     China Mobile
     Beijing
     China
     Email: yangfeng@chinamobile.com


     Changwang Lin
     New H3C Technologies
     Beijing
     China
     Email: linchangwang.04414@h3c.com






























  Yang & Lin               Expires 1 January 2026                [Page 12]


name: draft-templin-intarea-aero2
rev: 07
title: Automatic Extended Route Optimization (AERO)
time: 2024-04-04T18:58:04Z
group_uri: /api/v1/group/group/1027/
expires: 2024-10-06T18:58:04Z
states: ["/api/v1/doc/state/4/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-04T00:00:06.340310+00:00
source: ietf
abstract:
  This document specifies an Automatic Extended Route Optimization
     (AERO) service for IP internetworking over Overlay Multilink Network
     (OMNI) interfaces.  AERO/OMNI uses IPv6 Neighbor Discovery (IPv6 ND)
     for control plane messaging over the OMNI virtual link.  Router
     discovery and neighbor coordination are employed for network
     admission and to manage the OMNI link forwarding and routing systems.
     Secure multilink path selection, multinet traversal, mobility
     management, multicast forwarding, multihop operation and route
     optimization are naturally supported through dynamic neighbor cache
     updates.  Both Provider-Aggregated (PA) and Provider-Independent (PI)
     addressing services are supported.  AERO is a widely-applicable
     service especially well-suited for air/land/sea/space mobility
     applications including aviation, intelligent transportation systems,
     mobile end user devices, space exploration and many others.

full_text:
  Network Working Group                                 F. L. Templin, Ed.
  Internet-Draft                              Boeing Research & Technology
  Intended status: Standards Track                            4 April 2024
  Expires: 6 October 2024


                Automatic Extended Route Optimization (AERO)
                       draft-templin-intarea-aero2-07

  Abstract

     This document specifies an Automatic Extended Route Optimization
     (AERO) service for IP internetworking over Overlay Multilink Network
     (OMNI) interfaces.  AERO/OMNI uses IPv6 Neighbor Discovery (IPv6 ND)
     for control plane messaging over the OMNI virtual link.  Router
     discovery and neighbor coordination are employed for network
     admission and to manage the OMNI link forwarding and routing systems.
     Secure multilink path selection, multinet traversal, mobility
     management, multicast forwarding, multihop operation and route
     optimization are naturally supported through dynamic neighbor cache
     updates.  Both Provider-Aggregated (PA) and Provider-Independent (PI)
     addressing services are supported.  AERO is a widely-applicable
     service especially well-suited for air/land/sea/space mobility
     applications including aviation, intelligent transportation systems,
     mobile end user devices, space exploration and many others.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 6 October 2024.

  Copyright Notice

     Copyright (c) 2024 IETF Trust and the persons identified as the
     document authors.  All rights reserved.




  Templin                  Expires 6 October 2024                 [Page 1]


  Internet-Draft                    AERO                        April 2024


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .  18
     4.  Automatic Extended Route Optimization (AERO)  . . . . . . . .  18
       4.1.  AERO Node Types . . . . . . . . . . . . . . . . . . . . .  19
       4.2.  The AERO Service over OMNI Links  . . . . . . . . . . . .  20
         4.2.1.  AERO/OMNI Reference Model . . . . . . . . . . . . . .  20
         4.2.2.  Addressing and Node Identification  . . . . . . . . .  24
         4.2.3.  AERO Routing System . . . . . . . . . . . . . . . . .  25
         4.2.4.  Segment Routing Topologies (SRTs) . . . . . . . . . .  27
         4.2.5.  Segment Routing For OMNI Link Selection . . . . . . .  28
       4.3.  OMNI Interface Characteristics  . . . . . . . . . . . . .  28
       4.4.  OMNI Interface Initialization . . . . . . . . . . . . . .  31
         4.4.1.  AERO Proxy/Server and Relay Behavior  . . . . . . . .  31
         4.4.2.  AERO Client Behavior  . . . . . . . . . . . . . . . .  31
         4.4.3.  AERO Host Behavior  . . . . . . . . . . . . . . . . .  32
         4.4.4.  AERO Gateway Behavior . . . . . . . . . . . . . . . .  32
       4.5.  OMNI Interface Neighbor Cache Maintenance . . . . . . . .  32
         4.5.1.  OMNI ND Messages  . . . . . . . . . . . . . . . . . .  35
         4.5.2.  OMNI Neighbor Advertisement Message Flags . . . . . .  38
         4.5.3.  OMNI Neighbor Window Synchronization  . . . . . . . .  39
       4.6.  OMNI Interface Encapsulation and Fragmentation  . . . . .  39
       4.7.  OMNI Interface Decapsulation  . . . . . . . . . . . . . .  42
       4.8.  OMNI Interface Data Origin Authentication . . . . . . . .  42
       4.9.  OMNI Interface MTU  . . . . . . . . . . . . . . . . . . .  43
       4.10. OMNI Interface Forwarding Algorithm . . . . . . . . . . .  44
         4.10.1.  Host Forwarding Algorithm  . . . . . . . . . . . . .  46
         4.10.2.  Client Forwarding Algorithm  . . . . . . . . . . . .  46
         4.10.3.  Proxy/Server and Relay Forwarding Algorithm  . . . .  48
         4.10.4.  Gateway Forwarding Algorithm . . . . . . . . . . . .  51
       4.11. OMNI Interface Error Handling . . . . . . . . . . . . . .  52
       4.12. AERO Mobility Service Coordination  . . . . . . . . . . .  55
         4.12.1.  AERO Service Model . . . . . . . . . . . . . . . . .  55
         4.12.2.  AERO Host and Client Behavior  . . . . . . . . . . .  56
         4.12.3.  AERO Proxy/Server Behavior . . . . . . . . . . . . .  57
       4.13. AERO Address Resolution, Multilink Forwarding and Route
              Optimization . . . . . . . . . . . . . . . . . . . . . .  63



  Templin                  Expires 6 October 2024                 [Page 2]


  Internet-Draft                    AERO                        April 2024


         4.13.1.  Multilink Address Resolution . . . . . . . . . . . .  65
         4.13.2.  Multilink Forwarding . . . . . . . . . . . . . . . .  70
         4.13.3.  Mobile Ad-hoc Network (MANET) Forwarding . . . . . .  79
         4.13.4.  Client/Client Route Optimization . . . . . . . . . .  81
         4.13.5.  Intra-ANET/ENET Route Optimization for AERO Peers  .  83
       4.14. Neighbor Unreachability Detection (NUD) . . . . . . . . .  83
       4.15. Mobility Management and Quality of Service (QoS)  . . . .  85
         4.15.1.  Mobility Update Messaging  . . . . . . . . . . . . .  86
         4.15.2.  Announcing Link-Layer Information Changes  . . . . .  87
         4.15.3.  Bringing New Links Into Service  . . . . . . . . . .  87
         4.15.4.  Deactivating Existing Links  . . . . . . . . . . . .  87
         4.15.5.  Moving Between Proxy/Servers . . . . . . . . . . . .  88
       4.16. Multicast . . . . . . . . . . . . . . . . . . . . . . . .  89
         4.16.1.  Source-Specific Multicast (SSM)  . . . . . . . . . .  90
         4.16.2.  Any-Source Multicast (ASM) . . . . . . . . . . . . .  91
         4.16.3.  Bi-Directional PIM (BIDIR-PIM) . . . . . . . . . . .  92
       4.17. Operation over Multiple OMNI Links  . . . . . . . . . . .  92
       4.18. DNS Considerations  . . . . . . . . . . . . . . . . . . .  93
       4.19. Transition/Coexistence Considerations . . . . . . . . . .  93
       4.20. Proxy/Server-Gateway Bidirectional Forwarding
              Detection  . . . . . . . . . . . . . . . . . . . . . . .  94
       4.21. Time-Varying MNPs . . . . . . . . . . . . . . . . . . . .  94
     5.  Implementation Status . . . . . . . . . . . . . . . . . . . .  94
     6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  95
     7.  Security Considerations . . . . . . . . . . . . . . . . . . .  95
     8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  97
     9.  References  . . . . . . . . . . . . . . . . . . . . . . . . . 100
       9.1.  Normative References  . . . . . . . . . . . . . . . . . . 100
       9.2.  Informative References  . . . . . . . . . . . . . . . . . 102
     Appendix A.  Non-Normative Considerations . . . . . . . . . . . . 108
       A.1.  Implementation Strategies for Route Optimization  . . . . 108
       A.2.  Implicit Mobility Management  . . . . . . . . . . . . . . 109
       A.3.  Direct Underlying Interfaces  . . . . . . . . . . . . . . 109
       A.4.  AERO Critical Infrastructure Considerations . . . . . . . 110
       A.5.  AERO Server Failure Implications  . . . . . . . . . . . . 110
       A.6.  AERO Client / Server Architecture . . . . . . . . . . . . 111
     Appendix B.  Change Log . . . . . . . . . . . . . . . . . . . . . 113
     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . 113

  1.  Introduction

     Automatic Extended Route Optimization (AERO) fulfills the
     requirements of Distributed Mobility Management (DMM) [RFC7333] and
     route optimization [RFC5522] for air/land/sea/space mobility
     applications including aeronautical networking intelligent
     transportation systems, home network users, enterprise mobile device
     users, space exploration and many others.  AERO is a secure
     internetworking and mobility management service that employs the



  Templin                  Expires 6 October 2024                 [Page 3]


  Internet-Draft                    AERO                        April 2024


     Overlay Multilink Network Interface (OMNI)
     [I-D.templin-intarea-omni2] Non-Broadcast, Multiple Access (NBMA)
     virtual link model.

     The OMNI link is an adaptation layer virtual overlay manifested by
     IPv6 encapsulation over a network-of-networks concatenation of
     underlay Internetworks.  Nodes on the link can exchange original IP
     packets or parcels
     [I-D.templin-6man-parcels2][I-D.templin-intarea-parcels2] as single-
     hop neighbors; both IP protocol versions (IPv4 and IPv6) are
     supported.  The OMNI Adaptation Layer (OAL) supports multilink
     operation for increased reliability and path optimization while
     providing fragmentation and reassembly services to support improved
     performance and Maximum Transmission Unit (MTU) diversity.  This
     specification provides a mobility service architecture companion to
     the OMNI specification.

     The AERO service connects Hosts and Clients as OMNI link end systems
     via Proxy/Servers and Relays as intermediate systems as necessary;
     AERO further employs Gateways that interconnect diverse Internetworks
     as OMNI link segments through OAL forwarding at a layer below IP.
     Each node's OMNI interface supports operation of IPv6 Neighbor
     Discovery (IPv6 ND) [RFC4861] as the mobility service control message
     protocol.  A Client's OMNI interface can be configured over multiple
     underlay interfaces, and therefore appears as a single interface with
     multiple link layer addresses.  Each link layer address is subject to
     change due to mobility and/or multilink fluctuations, and link layer
     address changes are signaled by ND messaging the same as for any IPv6
     link.

     AERO provides a secure cloud-based service where mobile node Clients
     use Proxy/Servers acting as proxys and/or designated routers while
     correspondent nodes on foreign networks may use any Relay on the link
     for efficient communications.  Foreign network correspondent nodes
     forward original IP packets/parcels destined to other AERO nodes via
     the nearest Relay, which forwards them through the cloud.  Mobile
     node Clients discover shortest paths to OMNI link neighbors through
     AERO route optimization.  Both unicast and multicast communications
     are supported.

     AERO supports both Provider-Aggregated (PA) and Provider-Independent
     (PI)addressing.  Correspondent nodes on foreign networks configure PA
     addresses from Foreign Network Prefixes (FNPs) advertised by Relays.
     AERO Clients instead obtain stable PA addresses from Stable Network
     Prefixes (SNPs) assigned to and managed by First Hop Segment (FHS)
     Proxy/Servers.  Mobile node Clients can also register Mobile Network
     Prefixes (MNPs) with Mobility Anchor Point (MAP) Proxy/Servers to
     support PI mobile networking.



  Templin                  Expires 6 October 2024                 [Page 4]


  Internet-Draft                    AERO                        April 2024


     AERO Clients receive SNP (PA) addresses and optionally also MNP (PI)
     prefix delegations through control message exchanges with Proxy/
     Servers over their local networks.  Proxy/Servers provide anchor
     points for both local network PA operation and global mobility.  By
     linking mobile PI prefixes with stable PA addresses, the AERO service
     supports the best aspects of PA/PI working together.

     AERO Gateways peer with Proxy/Servers in a secured private BGP
     overlay routing instance to establish a Segment Routing Topology
     (SRT) virtual spanning tree over the underlay Internetworks of one or
     more disjoint administrative domains concatenated as a single unified
     OMNI link.  Each OMNI link instance is characterized by a set of
     Mobility Service Prefixes (MSPs) common to all mobile nodes.  Relays
     provide an optimal route from correspondent nodes on foreign links/
     networks to mobile or fixed nodes on the local OMNI link.  From the
     perspective of underlay Internetworks, each Relay appears as the
     source of a route to the MSP; hence uplink traffic to mobile nodes is
     naturally routed to the nearest Relay.

     AERO can be used with OMNI links that span private-use Internetworks
     and/or public Internetworks such as the global IPv4 and IPv6
     Internets.  In both cases, Clients may be located behind Network
     Address Translators (NATs) on the path to their associated Proxy/
     Servers and/or peers.  A means for robust traversal of NATs while
     avoiding "triangle routing" and critical infrastructure traffic
     concentration through a service known as route optimization is
     therefore provided.

     AERO assumes the use of PIM Sparse Mode in support of multicast
     communication.  In support of Source Specific Multicast (SSM) when a
     Mobile Node is the source, AERO route optimization ensures that a
     shortest-path multicast tree is established with provisions for
     mobility and multilink operation.  In all other multicast scenarios
     there are no AERO dependencies.

















  Templin                  Expires 6 October 2024                 [Page 5]


  Internet-Draft                    AERO                        April 2024


     AERO provides a secure aeronautical internetworking service for both
     manned and unmanned aircraft, where the aircraft is treated as a
     mobile node (MN) that can connect airborne Internet of Things (IoT)
     sub-networks.  AERO is also applicable to a wide variety of other use
     cases.  For example, it can be used to coordinate the links of mobile
     nodes (e.g., cellphones, tablets, laptop computers, etc.) that
     connect into a home enterprise network via public access networks
     with Virtual Private Network (VPN) or open Internetwork services
     enabled according to the appropriate security model.  AERO also
     supports terrestrial vehicular, urban air mobility and mobile
     pedestrian communication services for intelligent transportation
     systems [RFC9365].  Other applicable use cases including home and
     small office networks, enterprise networks and many others represent
     additional large classes of potential AERO/OMNI users.

     Along with OMNI, AERO provides secured optimal routing support for
     the "6 M's of Modern Internetworking", including:

     1.  Multilink - a mobile node's ability to coordinate multiple
         diverse underlay data links as a single logical unit (i.e., the
         OMNI interface) to achieve the required communications
         performance and reliability objectives.

     2.  Multinet - the ability to span the OMNI link over a segment
         routing topology with multiple diverse administrative domain
         network segments while maintaining seamless end-to-end
         communications between mobile Clients and correspondents such as
         air traffic controllers, fleet administrators, other mobile
         Clients, etc.

     3.  Mobility - a mobile node's ability to change network points of
         attachment (e.g., moving between wireless base stations) which
         may result in an underlay interface address change, but without
         disruptions to ongoing communication sessions with peers over the
         OMNI link.

     4.  Multicast - the ability to send a single network transmission
         that reaches multiple nodes belonging to the same interest group,
         but without disturbing other nodes not subscribed to the interest
         group.

     5.  Multihop - a mobile node vehicle-to-vehicle relaying capability
         useful when multiple forwarding hops between vehicles may be
         necessary to "reach back" to an infrastructure access point
         connection to the OMNI link.






  Templin                  Expires 6 October 2024                 [Page 6]


  Internet-Draft                    AERO                        April 2024


     6.  (Performance) Maximization - the ability to exchange large
         packets/parcels between peers without loss due to a link size
         restriction, and to adaptively adjust packet/parcel sizes to
         maintain the best performance profile for each independent
         traffic flow.

     The following numbered sections present the AERO specification.  The
     appendices at the end of the document are non-normative.

  2.  Terminology

     The terminology in the normative references applies; especially, the
     OMNI specification terminology [I-D.templin-intarea-omni2] and the
     IPv6 Neighbor Discovery (IPv6 ND) [RFC4861] node variables, protocol
     constants and message types (including Router Solicitation (RS),
     Router Advertisement (RS), Neighbor Solicitation (NS), Neighbor
     Advertisement (NA), unsolicited NA (uNA) and Redirect) are cited
     extensively throughout.

     OMNI interfaces normally limit the size of their IPv6 ND control
     plane messages to the minimum IPv6 link MTU, but some messages may
     exceed this size if there are sufficient OMNI parameters and/or IP
     packet/parcel attachments.  These larger messages can still travel
     over secured underlying network control plane paths that include
     IPsec tunnels [RFC4301] and/or secured direct point-to-point links
     without loss due to a size restriction by engaging OMNI IPv6
     encapsulation/fragmentation as necessary up to a maximum size of
     65535 octets.

     Throughout the document, the simple terms "Host", "Client", "Proxy/
     Server", "Gateway" and "Relay" refer to "AERO/OMNI Host", "AERO/OMNI
     Client", "AERO/OMNI Proxy/Server", "AERO/OMNI Gateway" and "AERO/OMNI
     Relay", respectively.  Capitalization is used to distinguish these
     terms from other common Internetworking uses in which they appear
     without capitalization, and implies that the node in question both
     configures an OMNI interface and engages the OMNI Adaptation Layer.

     The terms "All-Routers multicast", "All-Nodes multicast", "Solicited-
     Node multicast" and "Subnet-Router anycast" are defined in [RFC4291].

     The term "IP" refers generically to either Internet Protocol version
     (IPv4 [RFC0791] or IPv6 [RFC8200]) for specification elements that
     apply equally to both.

     The terms "application layer (L5 and higher)", "transport layer
     (L4)", "network layer (L3)", "(data) link layer (L2)" and "physical
     layer (L1)" are used consistently with common Internetworking
     terminology, with the understanding that reliable delivery protocol



  Templin                  Expires 6 October 2024                 [Page 7]


  Internet-Draft                    AERO                        April 2024


     users of UDP are considered as transport layer elements.  The OMNI
     specification further defines an "adaptation layer" positioned below
     the network layer but above the link layer, which may include
     physical links and Internet- or higher-layer tunnels.  A (network)
     interface is a node's attachment to a link (via L2), and an OMNI
     interface is therefore a node's attachment to an OMNI link (via the
     adaptation layer).

     The terms "IP jumbogram", "advanced jumbo (AJ)" and "IP parcel" refer
     to special packet formats that enable a new link model for the
     Internet as discussed in [I-D.templin-6man-parcels2]
     [I-D.templin-intarea-parcels2].

     The following terms are defined within the scope of this document:

     IPv6 Neighbor Discovery (IPv6 ND)
        a control message service for coordinating neighbor relationships
        between nodes connected to a common link.  AERO uses the IPv6 ND
        messaging service specified in [RFC4861] in conjunction with the
        OMNI extensions specified in [I-D.templin-intarea-omni2].

     IPv6 Prefix Delegation (IPv6 PD)
        a networking service for delegating IPv6 prefixes to nodes on the
        link.  AERO nodes apply the IPv6 PD service provided by DHCPv6
        [RFC8415] in conjunction with OMNI interface IPv6 ND.

     L3
        The Network layer in the OSI network model.  Also known as "layer
        3", "IP layer", etc.

     L2
        The Data Link layer in the OSI network model.  Also known as
        "layer 2", "link layer", "sub-IP layer", etc.

     Adaptation layer
        An encapsulation mid-layer that adapts L3 to a diverse collection
        of L2 underlay interfaces and their encapsulations.  (No layer
        number is assigned, since numbering was an artifact of the legacy
        reference model that need not carry forward in the modern
        architecture.)  The adaptation layer sees the network layer as
        "L3" and sees all link layer encapsulations as "L2
        encapsulations", which may include UDP, IP and true link layer
        (e.g., Ethernet, etc.) headers.

     Access Network (ANET)
        a connected network region (e.g., an aviation radio access
        network, corporate enterprise network, satellite service provider
        network, cellular operator network, residential WiFi network,



  Templin                  Expires 6 October 2024                 [Page 8]


  Internet-Draft                    AERO                        April 2024


        etc.) that connects Clients to the Mobility Service over the OMNI
        lin.  Physical and/or data link level security is assumed and
        sometimes referred to as "protected spectrum" for wireless
        domains.  Private enterprise networks and ground domain aviation
        service networks may provide multiple secured IP hops between the
        Client's point of connection and the nearest Proxy/Server.

     Mobile Ad-hoc NETwork (MANET)
        a connected network region that shares the same properties as an
        ANET except that links often have undetermined connectivity
        properties, physical and/or data link layer security cannot always
        be assumed and multihop forwarding between Clients acting as MANET
        routers is often necessary.  MANETs use IPv6 Unique Local
        Addressing (ULAs) [RFC4193] internally to support multihop packet
        forwarding between neighboring node as an adaptation layer
        forwarding and addressing service.

     Internetwork (INET)
        a connected network region with a coherent IP addressing plan that
        provides transit forwarding services between (M)ANETs and AERO/
        OMNI nodes that coordinate with the Mobility Service over
        unprotected media.  No physical and/or data link level security is
        assumed, therefore security must be applied by the network and/or
        higher layers.  The global public Internet itself is an example.

     End-user Network (ENET)
        a simple or complex "downstream" network tethered to a Client as a
        single logical unit that travels together.  The ENET could be as
        simple as a single link connecting a single Host, or as complex as
        a large network with many links, routers, bridges and end user
        devices.  The ENET provides an "upstream" link for arbitrarily
        many low-, medium- or high-end devices dependent on the Client for
        their upstream connectivity, i.e., as Internet of Things (IoT)
        entities.  The ENET can also support a recursively-descending
        chain of additional Clients such that the ENET of an upstream
        Client is seen as the ANET of a downstream Client.

     ANET/INET/ENET interface
        a node's attachment to a link in an ANET/INET/ENET.

     underlay network/interface
        an ANET/INET/ENET network/interface over which an OMNI interface
        is configured.  The OMNI interface is seen as a network layer (L3)
        interface by the IP layer, and the OMNI adaptation layer sees the
        underlay interface as a data link layer (L2) interface.  The
        underlay interface either connects directly to the physical or
        virtual communications media or coordinates with another node that
        hosts the media.



  Templin                  Expires 6 October 2024                 [Page 9]


  Internet-Draft                    AERO                        April 2024


     MANET Interface
        a node's underlay interface connection to an ANET with
        indeterminant neighborhood properties over which multihop relaying
        may be necessary.  All MANET interfaces used by AERO/OMNI are IPv6
        interfaces and therefore must configure a Maximum Transmission
        Unit (MTU) at least as large as the IPv6 minimum MTU (1280 octets)
        even if link-layer fragmentation is needed.

     OMNI link
        the same as defined in [I-D.templin-intarea-omni2].  The OMNI link
        employs IPv6 encapsulation to traverse intermediate systems in a
        spanning tree over underlay network segments the same as a bridged
        campus LAN.  AERO nodes on the OMNI link appear as single-hop
        neighbors at the network layer even though they may be separated
        by many underlay network hops; AERO nodes can employ Segment
        Routing [RFC8402] to navigate between different OMNI links, and/or
        to cause packets/parcels to visit selected waypoints within the
        same OMNI link.

     OMNI link segment
        a Proxy/Server and all of its constituent Clients within any
        attached *NETs is considered as a leaf OMNI link segment, with
        each leaf interconnected via links and "bridge" nodes in
        intermediate OMNI link segments.  When the *NETs of multiple leaf
        segments overlap (e.g., due to network mobility), they can combine
        to form larger *NETs with no changes to Client-to-Proxy/Server
        relationships.  The OMNI link consists of the concatenation of all
        OMNI link leaf and intermediate segments as a loop-free spanning
        tree.

     OMNI Adaptation Layer (OAL)
        an OMNI interface sublayer service that encapsulates original IP
        packets/parcels admitted into the interface in an IPv6 header and/
        or subjects them to fragmentation and reassembly.  The OAL is also
        responsible for generating MTU-related control messages as
        necessary, and for providing addressing context for spanning
        multiple segments of an extended OMNI link.

     OMNI Interface
        a node's attachment to an OMNI link.  Since OMNI interface
        addresses are managed for uniqueness, OMNI interfaces do not
        require Duplicate Address Detection (DAD) and therefore set the
        administrative variable 'DupAddrDetectTransmits' to zero
        [RFC4862].

     (network) partition
        frequently, underlay networks such as large corporate enterprise
        networks are sub-divided internally into separate isolated



  Templin                  Expires 6 October 2024                [Page 10]


  Internet-Draft                    AERO                        April 2024


        partitions (a technique also known as "network segmentation").
        Each partition is fully connected internally but disconnected from
        other partitions, and there is no requirement that separate
        partitions maintain consistent Internet Protocol and/or addressing
        plans.  (Each partition is seen as a separate OMNI link segment as
        discussed throughout this document.)

     (OMNI) L2 encapsulation
        the OMNI protocol encapsulation of OAL packets/fragments in an
        outer header or headers to form carrier packets that can be routed
        within the scope of the local (M)ANET/INET/ENET underlay network
        partition.  Common L2 encapsulation combinations include UDP/IP/
        Ethernet, etc. using a port/protocol/type number for OMNI.

     L2 address (L2ADDR)
        an address that appears in the L2 encapsulation for an underlay
        interface and also in IPv6 ND message OMNI options.  L2ADDR can be
        either an IP address for IP encapsulations or an IEEE EUI address
        [EUI] for direct data link encapsulation.  (When UDP/IP
        encapsulation is used, the UDP port number is considered an
        ancillary extension of the IP L2ADDR.)

     original IP packet/parcel
        a whole IP packet/parcel or fragment admitted into the OMNI
        interface by the network layer prior to OAL encapsulation and
        fragmentation, or an IP packet delivered to the network layer by
        the OMNI interface following OAL reassembly/decapsulation.

     OAL packet
        an original IP packet/parcel encapsulated in an OAL IPv6 header
        with an IPv6 Extended Fragment Header extension that includes an
        8-octet (64-bit) OAL Identification value.  Each OAL packet is
        then subject to OAL fragmentation and reassembly.

     OAL fragment
        a portion of an OAL packet following fragmentation but prior to L2
        encapsulation/fragmentation, or following L2 reassembly/
        decapsulation but prior to OAL reassembly.

     (OAL) atomic fragment
        an OAL packet that can be forwarded without fragmentation, but
        still includes an IPv6 Extended Fragment Header with an 8-octet
        (64-bit) OAL Identification value and with Fragment Offset and
        More Fragments both set to 0.

     (L2) carrier packet
        an encapsulated OAL packet/fragment following L2 encapsulation or
        prior to L2 decapsulation.  OAL sources and destinations exchange



  Templin                  Expires 6 October 2024                [Page 11]


  Internet-Draft                    AERO                        April 2024


        carrier packets over underlay interfaces, and may be separated by
        one or more OAL intermediate systems.  OAL intermediate systems
        re-encapsulate OAL packets/fragments during forwarding by removing
        the L2 headers of carrier packets from a previous hop underlay
        network and replacing them with new L2 headers for the next hop
        underlay network.  Carrier packets may themselves be subject to
        fragmentation and reassembly in L2 underlay networks at a layer
        below the OAL.  Carrier packets sent over unsecured paths use OMNI
        protocol L2 encapsulations, while those sent over the secured
        paths use L2 security encapsulations such as IPsec [RFC4301], etc.

     OAL source
        an OMNI interface acts as an OAL source when it encapsulates
        original IP packets/parcels to form OAL packets, then performs OAL
        fragmentation and L2 encapsulation to create carrier packets.
        Every OAL source is also an OAL end system.

     OAL destination
        an OMNI interface acts as an OAL destination when it decapsulates
        carrier packets, then performs OAL reassembly/decapsulation to
        restore the original IP packet/parcel.  Every OAL destination is
        also an OAL end system.

     OAL intermediate system
        an OMNI interface acts as an OAL intermediate system when it
        performs L2 reassembly/decapsulation for carrier packets received
        from a previous hop, then performs L2 encapsulation/fragmentation
        on the enclosed OAL packets/fragments and forwards these new
        carrier packets to the next hop.  OAL intermediate systems
        decrement the OAL Hop Limit during forwarding, and discard the OAL
        packet/fragment if the Hop Limit reaches 0.  OAL intermediate
        systems do not decrement the TTL/Hop Limit of the original IP
        packet/parcel.

     Mobility Service Prefix (MSP)
        an aggregated IP Global Unicast Address (GUA) prefix (e.g.,
        2001:db8::/32, 192.0.2.0/24, etc.) assigned to the OMNI link and
        from which more-specific Mobile and Stable Network Prefixes (MNPs/
        SNPs) are delegated.  OMNI link administrators typically obtain
        MSPs from an Internet address registry, however private-use
        prefixes can alternatively be used subject to certain limitations
        (see: [I-D.templin-intarea-omni2]).  OMNI links that connect to
        the global Internet advertise their MSPs to interdomain routing
        peers.







  Templin                  Expires 6 October 2024                [Page 12]


  Internet-Draft                    AERO                        April 2024


     Mobile Network Prefix (MNP)
        a longer IP prefix derived from an MSP (e.g.,
        2001:db8:1000:2000::/56, 192.0.2.8/30, etc.) and delegated to an
        AERO Client.

     Stable Network Prefix (SNP)
        a global and unique-local IP prefix pair assigned to one or more
        Proxy/Servers that connect local (M)ANETs or INET Client groups to
        the rest of the OMNI link.  Clients request address delegations
        from the SNP that can be used to support PA communications.
        Clients communicate internally within (M)ANETs and INET groups
        using IPv6 Unique Local Addresses (ULAs) [RFC4193] assigned in 1x1
        correspondence to SNP Globally Unique Addresses (GUAs) [RFC4291]
        made visible to external peers through IP network address/prefix
        translation [RFC6145][RFC6146][RFC6147]
        [I-D.bctb-6man-rfc6296-bis].

     Foreign Network Prefix (FNP)
        a global IP prefix not covered by a MSP and assigned to a link or
        network outside of the AERO/OMNI domain.  Relays advertise any of
        their associated FNPs into the AERO/OMNI routing system and
        forward packets between MNP/SNP mobile or fixed nodes on the OMNI
        link and FNP correspondent nodes on other links.

     Interface Identifier (IID)
        the least significant 64 bits of an IPv6 address, as specified in
        the IPv6 addressing architecture [RFC4291].

     Link Local Address (LLA)
        an IPv6 address beginning with fe80::/64 per the IPv6 addressing
        architecture [RFC4291].

     Unique Local Address (ULA)
        an IPv6 address beginning with fd00::/8 followed by a 40-bit
        Global ID followed by a 16-bit Subnet ID per [RFC4193].  (Note
        that [RFC4193] specifies a second form of ULAs based on the prefix
        fc00::/8, which are referred to as "ULA-C" throughout this
        document to distinguish them from the ULAs defined here.)

     Globally Unique Address (GLA)
        a globally unique IPv6 address per the IPv6 addressing
        architecture [RFC4291] or a globally unique IPv4 address that is
        not reserved for a special-purpose per [RFC6890].

     Hierarchical Host Identity Tag (HHIT)
        a 128-bit IPv6 address according to [RFC9374].  Each Client
        assigns a unique HHIT used to bootstrap autoconfiguration in the
        presence of OMNI link infrastructure or for sustained



  Templin                  Expires 6 October 2024                [Page 13]


  Internet-Draft                    AERO                        April 2024


        communications in the absence of infrastructure.  When a Client
        receives a PA SNP GUA/ULA delegation from a Proxy/Server or a PI
        prefix delegation from a MAP, it can begin using PA/PI addresses
        instead of its HHIT for Internetworking communications.

     Provider Independent (PI) Address
        a global unicast IP address allocated from an MNP delegated to a
        Client via a MAP Proxy/Server is considered Provider-Independent
        (Proxy/Server-Independent) or "PI".

     Provider Aggregated (PA) Address
        a global unicast IP address delegated to a Client from a SNP
        assigned to a FHS Proxy/Server is considered Provider-Aggregated
        (Proxy/Server-Aggregated) or "PA".

     AERO node
        a node that is connected to an OMNI link and participates in the
        AERO internetworking and mobility service.

     (AERO) Host
        an AERO node that configures an OMNI interface over an ENET
        underlay interface serviced by an upstream Client.  The Host does
        not assign an LLA or ULA to the OMNI interface, but instead
        assigns the address taken from the ENET underlay interface.  When
        an AERO host forwards an original IP packet/parcel to another AERO
        node on the same ENET, it uses simple IP-in-L2 OMNI encapsulation
        without including an OAL encapsulation header.  The Host is
        therefore an OMNI link termination endpoint.  (Note: as an
        implementation matter, the Host may instead configure the "OMNI
        interface" as a virtual sublayer of the underlay interface
        itself.)

     (AERO) Client
        an AERO node that configures an OMNI interface over one or more
        underlay interfaces and requests SNP address and/or MNP prefix
        delegations from AERO Proxy/Servers.  The Client assigns a HHIT
        (as well as Proxy/Server-specific ULAs) to the OMNI interface for
        use in IPv6 ND exchanges with other AERO nodes and forwards
        original IP packets/parcels to correspondents according to OMNI
        interface neighbor cache state.  The Client coordinates with
        Proxy/Servers and/or other Clients over upstream ANET/INET
        interfaces and may also provide Proxy/Server services for Hosts
        and/or other Clients over downstream ENET interfaces.

     (AERO) Proxy/Server
        an AERO node that provides a proxying service between AERO Clients
        and external peers on its Client-facing (M)ANET interfaces (i.e.,
        in the same fashion as for an enterprise network proxy) as well as



  Templin                  Expires 6 October 2024                [Page 14]


  Internet-Draft                    AERO                        April 2024


        designated router services for coordination with correspondents on
        its INET-facing interfaces.  (Proxy/Servers in the open INET
        instead configure only a single INET interface and no (M)ANET
        interfaces.)  The Proxy/Server configures an OMNI interface and
        maintains BGP peerings with Gateways to provide a local anchor
        point for its stable and/or mobile Clients.  All Proxy/Servers
        configure a Stable Network Prefix (SNP) and manage 1x1 mappings of
        internal Unique Local Addresses (ULAs) and external Globally
        Unique Addresses (GUAs) according to Network Prefix Translation
        for IPv6 (NPTv6) [I-D.bctb-6man-rfc6296-bis].

     (AERO) Relay
        an AERO Proxy/Server that provides forwarding services between
        nodes reached via the OMNI link and correspondents on foreign
        links/networks.  AERO Relays maintain BGP peerings with Gateways
        the same as Proxy/Servers.  Relays also run a dynamic routing
        protocol to discover any Foreign Network Prefix (FNP) routes in
        service on other links/networks, advertise OMNI link MSP(s) to
        other links/networks, and redistribute FNPs discovered on other
        links/networks into the OMNI link BGP routing system.  (Relays
        that connect to major Internetworks such as the global IPv6 or
        IPv4 Internets can also be configured to advertise "default"
        routes into the OMNI link BGP routing system.)

     (AERO) Gateway
        a BGP hub autonomous system node that also provides OAL forwarding
        services for nodes on an OMNI link.  Gateways forward OAL packets/
        fragments between OMNI link segments as OAL intermediate systems
        while decrementing the OAL IPv6 header Hop Limit but without
        decrementing the network layer IP TTL/Hop Limit.  Gateways peer
        with Proxy/Servers and other Gateways to form an IPv6-based OAL
        spanning tree over all OMNI link segments and to discover the set
        of all FNP/MNP/SNP prefixes in service.  Gateways process OAL
        packets/fragments received over the secured spanning tree that are
        addressed to themselves, while forwarding all other OAL packets/
        fragments to the next hop also via the secured spanning tree.
        Gateways forward OAL packets/fragments received over the unsecured
        spanning tree to the next hop either via the unsecured spanning
        tree or via direct encapsulation if the next hop is on the same
        OMNI link segment.  It is important to note that all Gateways are
        also Proxy/Servers, but only those Proxy/Servers configured as
        intermediate nodes in the spanning tree are considered Gateways.









  Templin                  Expires 6 October 2024                [Page 15]


  Internet-Draft                    AERO                        April 2024


     First-Hop Segment (FHS) Client
        a Client that initiates communications with a target peer by
        sending an NS message to establish reverse-path multilink
        forwarding state in OMNI link intermediate systems on the path to
        the target.  Note that in some arrangements the Client's (FHS)
        Proxy/Server (and not the Client itself) initiates the NS.

     Last-Hop Segment (LHS) Client
        a Client that responds to a communications request from a source
        peer's NS by returning an NA response to establish forward-path
        multilink forwarding state in OMNI link intermediate systems on
        the path to the source.  Note that in some arrangements the
        Client's (LHS) Proxy/Server (and not the Client itself) returns
        the NA.

     First-Hop Segment (FHS) Proxy/Server
        a Proxy/Server for an FHS Client's underlay interface that
        forwards the Client's OAL packets into the segment routing
        topology.  FHS Proxy/Servers also act as intermediate forwarding
        systems to facilitate RS/RA exchanges between a Client and its MAP
        Proxy/Server.

     Last-Hop Segment (LHS) Proxy/Server
        a Proxy/Server for an underlay interface of an LHS Client that
        forwards OAL packets received from the segment routing topology to
        the Client over that interface.

     Mobility anchor Point (MAP) Proxy/Server
        a single Proxy/Server selected by a Client that injects the
        Client's MNP into the BGP routing system and provides a designated
        router service for all of the Client's underlay interfaces.
        Clients often select the first FHS Proxy/Server they coordinate
        with to serve in the MAP role (as all FHS Proxy/Servers are
        equally capable candidates to serve as a MAP), however the Client
        can also select any available Proxy/Server for the OMNI link (as
        there is no requirement that the MAP must also be one of the
        Client's FHS Proxy/Servers).  This flexible arrangement supports a
        fully distributed mobility management service.

     Segment Routing Topology (SRT)
        a Multinet OMNI link forwarding region between FHS and LHS Proxy/
        Servers.  FHS/LHS Proxy/Servers and SRT Gateways span the OMNI
        link on behalf of communicating peer nodes.  The SRT maintains a
        spanning tree established through BGP peerings between Gateways
        and Proxy/Servers.  Each SRT leaf segment includes Gateways in a
        "hub" and Proxy/Servers in "spokes", while adjacent segments are
        interconnected by Gateway-Gateway peerings.  The BGP peerings are
        configured over both secured and unsecured underlay network paths



  Templin                  Expires 6 October 2024                [Page 16]


  Internet-Draft                    AERO                        April 2024


        such that a secured spanning tree is available for critical
        control messages while other messages can use the unsecured
        spanning tree.

     Mobile Node (MN)
        an AERO Client and all of its downstream-attached networks that
        move together as a single unit, i.e., an end system and its
        connected IoT sub-networks.

     Mobile Router (MR)
        a MN's on-board router that forwards original IP packets/parcels
        between any downstream-attached networks and the OMNI link.  The
        MR is the MN entity that hosts the AERO Client.

     Address Resolution Source (ARS)
        the node nearest the original source that initiates OMNI link
        address resolution.  The ARS may be a Proxy/Server or Relay for
        the source, or may be the source Client itself.  The ARS is often
        (but not always) also the same node that becomes the FHS source
        during route optimization.

     Address Resolution Target (ART)
        the node toward which address resolution is directed.  The ART may
        be a Relay or the target Client itself.  The ART is often (but not
        always) also the same node that becomes the LHS target during
        route optimization.

     Address Resolution Responder (ARR)
        the node that responds to address resolution requests on behalf of
        the ART.  The ARR may be a Relay, the ART itself, or the ART's
        current MAP Proxy/Server.  Note that a MAP Proxy/Server can assume
        the ARR role even if it is located on a different SRT segment than
        the ART.  The MAP Proxy/Server assumes the ARR role only when it
        receives an RS message from the ART with the 'ARR' flag set (see:
        [I-D.templin-intarea-omni2]).

     Potential Router List (PRL)
        a geographically and/or topologically referenced list of addresses
        of all Proxy/Servers within the same OMNI link.  Each OMNI link
        has its own PRL.

     Distributed Mobility Management (DMM)
        a BGP-based overlay routing service coordinated by Proxy/Servers
        and Gateways that tracks all Proxy/Server-to-Client associations.

     Mobility Service (MS)
        the collective set of all Proxy/Servers, Gateways and Relays that
        provide the AERO Service to Clients.



  Templin                  Expires 6 October 2024                [Page 17]


  Internet-Draft                    AERO                        April 2024


     AERO Forwarding Information Base (AFIB)
        A forwarding table on each OAL source, destination and
        intermediate system that includes AERO Forwarding Vectors (AFV)
        with both multilink forwarding instructions and context for
        reconstructing compressed headers for specific communicating peer
        underlay interface pairs.  The AFIB also supports route
        optimization where one or more OAL intermediate systems in the
        path can be "skipped" to reduce path stretch and decrease load on
        critical infrastructure elements.

     AERO Forwarding Vector (AFV)
        An AFIB entry that includes soft state for each underlay interface
        pairwise communication session between peer neighbors.  AFVs are
        identified by an AFV Index (AFVI) paired with the previous hop L2
        address, with the pair established based on an IPv6 NS/NA or RS/RA
        exchanges.  The AFV also caches underlay interface Identification
        sequence number parameters to support carrier packet filtering.

     AERO Forwarding Vector Index (AFVI)
        A 2-octet or 4-octet integer value supplied by a first hop OAL
        node when it requests a next hop OAL node to create an AFV.  (The
        AFVI is always processed as a 4-octet value, but may be
        transmitted as only the 2 least significant octets when the 2 most
        significant octets are 0.)  The next hop OAL node caches the AFVI
        and L2 address supplied by the previous hop as header compression/
        decompression state for future OAL packets with compressed
        headers.  The first hop OAL node must ensure that the AFVI values
        it assigns to the next hop via a specific underlay interface are
        distinct and reused only after their useful lifetimes expire.

  3.  Requirements

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in BCP
     14 [RFC2119][RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  4.  Automatic Extended Route Optimization (AERO)

     The following sections specify the operation of IP over OMNI links
     using the AERO service:









  Templin                  Expires 6 October 2024                [Page 18]


  Internet-Draft                    AERO                        April 2024


  4.1.  AERO Node Types

     AERO Hosts configure an OMNI interface over an underlay interface
     connected to a Client's ENET and coordinate with both other AERO
     Hosts and Clients over the ENET.  As an implementation matter, the
     Host either assigns the same (MNP-based) IP address from the underlay
     interface to the OMNI interface, or configures the "OMNI interface"
     as a virtual sublayer of the underlay interface itself.  AERO Hosts
     treat the ENET as an ANET, and treat the upstream Client for the ENET
     as a Proxy/Server.  AERO Hosts are seen as OMNI link termination
     endpoints.

     AERO Clients can be deployed as fixed infrastructure nodes close to
     end systems, or as Mobile Nodes (MNs) that can change their network
     attachment points dynamically.  AERO Clients configure OMNI
     interfaces over underlay interfaces with addresses that may change
     due to mobility.  AERO Clients that obtain Mobile Network Prefixes
     (MNPs) register them with the AERO service, and distribute the MNPs
     to ENETs (which may connect AERO Hosts and other Clients).  AERO
     Clients provide Proxy/Server-like services for Hosts and other
     Clients on downstream-attached ENETs.

     AERO Gateways, Proxy/Servers and Relays are critical infrastructure
     elements in fixed (i.e., non-mobile) (M)ANET/INET boundary (or
     standalone INET) deployments and hence have permanent and unchanging
     INET addresses.  Together, they provide access to the AERO service
     OMNI link virtual overlay for connecting AERO Clients and Hosts.
     AERO Gateways (together with Proxy/Servers and Relays) provide the
     secured backbone supporting infrastructure for a Segment Routing
     Topology (SRT) spanning tree for the OMNI link.

     AERO Gateways are Proxy/Servers deployed as OMNI link intermediate
     systems that forward packets both within the same SRT segment and
     between disjoint SRT segments based on an IPv6 encapsulation mid-
     layer known as the OMNI Adaptation Layer (OAL).  The OMNI interface
     and OAL provide an adaptation layer forwarding service that the
     network layer perceives as L2 bridging, since the inner IP TTL/Hop
     Limit is not decremented.  Each Gateway peers with Proxy/Servers,
     Relays and other Gateways in a dynamic routing protocol instance as a
     Distributed Mobility Management (DMM) service for the list of active
     MNPs (see: Section 4.2.3).  Gateways assign one or more Mobility
     Service Prefixes (MSPs) to the OMNI link and configure secured
     tunnels with Proxy/Servers, Relays and other Gateways; they further
     maintain forwarding table entries for each FNP/MNP/SNP prefix in
     service on the OMNI link.






  Templin                  Expires 6 October 2024                [Page 19]


  Internet-Draft                    AERO                        April 2024


     AERO Proxy/Servers distributed across one or more SRT segments
     provide default forwarding and mobility/multilink services for AERO
     Client mobile nodes.  Each Proxy/Server acts as either an OMNI link
     intermediate system or end system according to the service model
     selected by each Client.  Proxy/Servers also peer with Gateways in an
     adaptation layer dynamic routing protocol instance to advertise its
     list of associated MNPs (see Section 4.2.3).  MAP Proxy/Servers
     provide prefix delegation services and track the mobility/multilink
     profiles of each of their associated Clients, where each delegated
     prefix becomes an MNP taken from an MSP.  Proxy/Servers at (M)ANET/
     INET boundaries provide a primary forwarding service for (M)ANET
     Client/Host communications with peers in external INETs, while Proxy/
     Servers in open INETs provide an authentication service for IPv6 ND
     messages but should be used only as a last resort data plane
     forwarding service when a Client cannot forward directly to an INET
     peer or Gateway.  Source Clients securely coordinate with target
     Clients by sending control messages via a First-Hop Segment (FHS)
     Proxy/Server which forwards them over the SRT spanning tree to a
     Last-Hop Segment (LHS) Proxy/Server which finally forwards them to
     the target.

     AERO Relays are Proxy/Servers that provide forwarding services to
     exchange original IP packets/parcels between the OMNI link and fixed
     or mobile nodes on other links/networks.  Relays run a dynamic
     routing protocol to discover any FNP prefixes in service on foreign
     links/networks, and Relays that connect to larger Internetworks (such
     as the Internet) may originate default routes.  The Relay
     redistributes OMNI link MSP(s) into other links/networks, and
     redistributes FNPs via OMNI link Gateway BGP peerings.

  4.2.  The AERO Service over OMNI Links

  4.2.1.  AERO/OMNI Reference Model

     Figure 1 presents the basic OMNI link reference model:
















  Templin                  Expires 6 October 2024                [Page 20]


  Internet-Draft                    AERO                        April 2024


                           +-----------------+
                           | AERO Gateway G1 |
                           | Nbr: S1, S2, P1 |
                           |(X1->S1; X2->S2) |
                           |      MSP M1     |
                           +--------+--------+
         +--------------+           |            +--------------+
         |  AERO P/S S1 |           |            |  AERO P/S S2 |
         |  Nbr: C1, G1 |           |            |  Nbr: C2, G1 |
         |  default->G1 |           |            |  default->G1 |
         |    X1->C1    |           |            |    X2->C2    |
         +-------+------+           |            +------+-------+
                 |       OMNI link  |                   |
         X===+===+==================+===================+===+===X
             |                                              |
       +-----+--------+                            +--------+-----+
       |AERO Client C1|                            |AERO Client C2|
       |    Nbr: S1   |                            |   Nbr: S2    |
       | default->S1  |                            | default->S2  |
       |    MNP X1    |                            |    MNP X2    |
       +------+-------+                            +-----+--------+
              |                                          |
             .-.                                        .-.
          ,-(  _)-.     +-------+     +-------+      ,-(  _)-.
       .-(_  IP   )-.   |  AERO |     |  AERO |    .-(_  IP   )-.
     (__    ENET     )--|Host H1|     |Host H2|--(__    ENET     )
        `-(______)-'    +-------+     +-------+     `-(______)-'

                      Figure 1: AERO/OMNI Reference Model

     In this model:

     *  the OMNI link is an overlay network service configured over one or
        more underlay SRT segments which may be managed by diverse
        administrative domains using incompatible protocols and/or
        addressing plans.

     *  AERO Gateway G1 aggregates Mobility Service Prefix (MSP) M1,
        discovers Mobile Network Prefixes (MNPs) X* and advertises the MSP
        via BGP peerings over secured tunnels to other Gateways in the SRT
        (not shown).  Together, the set of all Gateways provide the
        backbone for an SRT spanning tree for the OMNI link.









  Templin                  Expires 6 October 2024                [Page 21]


  Internet-Draft                    AERO                        April 2024


     *  AERO Proxy/Servers S1 and S2 configure secured tunnels with
        Gateway G1 and also provide mobility, multilink, multicast and
        default router services for the MNPs of their associated Clients
        C1 and C2.  (Proxy/Servers that act as Relays can also advertise
        FNP routes for non-mobile correspondent nodes the same as for MNP
        Clients.)

     *  AERO Clients C1 and C2 associate with Proxy/Servers S1 and S2,
        respectively.  They receive MNP delegations X1 and X2, and also
        act as default routers for their associated physical or internal
        virtual ENETs.

     *  AERO Hosts H1 and H2 attach to the ENETs served by Clients C1 and
        C2, respectively.

     An OMNI link configured over a single underlay network appears as a
     single unified link with a consistent addressing plan; all nodes on
     the link can exchange carrier packets via simple L2 encapsulation
     (i.e., following any necessary NAT traversal) since the underlay is
     connected.  In common practice, however, OMNI links are often
     configured over an SRT spanning tree that bridges multiple distinct
     underlay network segments managed under different administrative
     authorities (e.g., as for worldwide aviation service providers such
     as ARINC, SITA, Inmarsat, etc.).  Individual underlay networks may
     also be partitioned internally, in which case each internal partition
     appears as a separate segment.

     The addressing plan of each SRT segment is consistent internally but
     will often bear no relation to the addressing plans of other
     segments.  Each segment is also likely to be separated from others by
     network security devices (e.g., firewalls, proxys, packet filtering
     gateways, etc.), and disjoint segments often have no common physical
     link connections.  Therefore, nodes can only be assured of exchanging
     carrier packets directly with correspondents in the same segment, and
     not with those in other segments.  The only means for joining the
     segments therefore is through inter-domain peerings between AERO
     Gateways.

     The OMNI link spans multiple SRT segments using the OMNI Adaptation
     Layer (OAL) to provide the network layer with a virtual abstraction
     similar to a bridged campus LAN.  The OAL is an OMNI interface
     sublayer that inserts a mid-layer IPv6 encapsulation header for
     inter-segment forwarding (i.e., bridging) without decrementing the
     network layer TTL/Hop Limit of the original IP packet/parcel.  An
     example OMNI link SRT is shown in Figure 2:






  Templin                  Expires 6 October 2024                [Page 22]


  Internet-Draft                    AERO                        April 2024


      . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
    .                                                                           .
   .                                                                             .
   .     .-(::::::::)                .-(::::::::)               .-(::::::::)     .
   .  .-(::::::::::::)-.   +-+    .-(::::::::::::)-.   +-+   .-(::::::::::::)-.  .
   . (::::    FHS    :::)--|G|--(::: Intermediate ::)--|G|--(::::    LHS    :::) .
   .  `-(::::::::::::)-'   +-+    `-(::Segments::)-'   +-+   `-(::::::::::::)-'  .
   .     `-(::::::)-'                `-(::::::)-'               `-(::::::)-'     .
   .           |                                                      |          .
   .         +---+                                                  +---+        .
   .         |P/S|                                                  |P/S|        .
   .         +---+                                                  +---+        .
   .           |                                                      |          .
   .     .-(::::::::)                                          .-(::::::::)      .
   .  .-(: First Hop :)-.  +-------+             +-------+   .-(: Last Hop :)-.  .
   . (::::  Access  ::::)--| Source|             | Target|--(::::  Access  ::::) .
   .  `-(:: Network ::)-'  | Client|             | Client|     (:: Network ::)-' .
   .     `-(::::::)-'      +-------+             +-------+      `-(::::::)-'     .
   .                                                                             .
   .                                                                             .
   .         <--  Segment Routing Topology (SRT) Spanned by OMNI Link -->        .
     .                                                                          .
       . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

            Figure 2: OMNI Link Segment Routing Topology (SRT)

     In the Segment Routing Topology, a source Client connects via a first
     hop access network served by a First Hop Segment (FHS) Proxy/Server.
     The FHS Proxy/Server then forwards to an FHS Gateway which connects
     to an arbitrarily complex set of Intermediate Segments.  Adjacent
     intermediate Segments are joined by intermediate Gateways (not shown)
     that serve as adaptation layer IPv6 routers, with the final segment
     connected by a Last Hop Segment (LHS) Gateway.  The LHS Gateway then
     forwards to an LHS Proxy/Server which in turn connects to the last
     hop access network where the target Client resides.
















  Templin                  Expires 6 October 2024                [Page 23]


  Internet-Draft                    AERO                        April 2024


     Gateway, Proxy/Server and Relay OMNI interfaces are configured over
     both secured tunnels and open INET underlay interfaces within their
     respective SRT segments.  Within each segment, Gateways configure
     "hub-and-spokes" BGP peerings with Proxy/Servers and Relays as
     "spokes".  Adjacent SRT segments are joined by Gateway-to-Gateway
     peerings to collectively form a spanning tree over the entire SRT.
     The "secured" spanning tree supports authentication and integrity for
     critical control plane messages (and any trailing data plane message
     extensions).  The "unsecured" spanning tree conveys ordinary carrier
     packets without security codes and that must be treated by
     destinations according to data origin authentication procedures.
     AERO nodes can employ route optimization to cause carrier packets to
     take more direct paths between OMNI link neighbors without having to
     follow strict spanning tree paths.

     The AERO Multinet service concatenates SRT segments to form a larger
     network through Gateway-to-Gateway peerings as originally suggested
     in the "Catenet Model for Internetworking" [IEN48]; especially
     Figure 2 follows directly from the illustrations in [IEN48-2].  The
     Catenet concept suggested a "network-of-networks" concatenation of
     independent and diverse Internetwork "segments" to form a much larger
     network supporting end-to-end services.

     The Catenet concept first articulated in the 1970's was distorted
     through the evolution of the Internet in the decades that followed,
     since the adaptation layer was a critical element missing from the
     architecture.  As a result, while the Internet has been successful
     beyond measure it has evolved as a monolithic public routing and
     addressing service interconnecting private domains instead of a true
     network-of-networks which has impeded flexibility and inhibited end-
     to-end services.  The adaptation layer manifested by AERO and OMNI
     now provides the means to address these limitations as well as the
     other "6 Ms of Modern Internetworking" according to the original
     Catenet network-of-networks vision.

  4.2.2.  Addressing and Node Identification

     AERO nodes on OMNI links use the Link-Local Address (LLA) prefix
     fe80::/64 [RFC4291] to assign LLAs to the OMNI interface to satisfy
     the requirements of [RFC4861].  AERO node LLAs only need to be unique
     on the local OMNI link segment, however, since there is no way to
     coordinate duplicate address detection between disjoint OMNI link
     segments.  Therefore, OMNI interface intermediate systems should not
     forward packets with LLA source and/or destination addresses.

     AERO Clients also use the Unique Local Address (ULA) prefix fd00::/8
     followed by a pseudo-random 40-bit Global ID to form the prefix
     {ULA}::/48, then include a 16-bit Subnet ID '*' to form the prefix



  Templin                  Expires 6 October 2024                [Page 24]


  Internet-Draft                    AERO                        April 2024


     {ULA*}::/64 [RFC4291].  AERO Proxy/Servers assign ULAs to Clients as
     *NET internal addresses in 1x1 correspondence with Globally-Unique
     Addresses (GUAs) as *NET external addresses according to NPTv6
     [I-D.bctb-6man-rfc6296-bis].

     AERO MSPs, FNPs, MNPs and SNPs are typically based on Global Unicast
     Addresses (GUAs), but in some cases may be based on IPv4 private
     addresses [RFC1918] or IPv6 ULA-C's [RFC4193].  See
     [I-D.templin-intarea-omni2] for a full specification of LLAs, ULAs,
     GUAs and anycast addresses used by AERO nodes on OMNI links.

     Finally, AERO Clients configure HHITs as specified in
     [I-D.templin-intarea-omni2] to bootstrap the process of receiving ULA
     delegations from Proxy/Servers.

  4.2.3.  AERO Routing System

     The AERO routing system comprises a private Border Gateway Protocol
     (BGP) [RFC4271] service coordinated between Gateways as interior
     nodes and Proxy/Servers and Relays as leaf nodes of a spanning tree.
     The service supports OAL packet/fragment forwarding at a layer below
     IP and does not interact with the public Internet BGP routing system,
     but supports redistribution of information for other links and
     networks connected by Relays.

     In a reference deployment, each Proxy/Server is configured as an
     Autonomous System Border Router (ASBR) for a stub Autonomous System
     (AS) using a 32-bit AS Number (ASN) [RFC4271] that is unique within
     the BGP instance, and each Proxy/Server further uses eBGP to peer
     with one or more Gateways but does not peer with other Proxy/Servers.
     Each SRT segment in the OMNI link must include one or more Gateways
     in a "hub" AS, which peer with the Proxy/Servers within that segment
     as "spoke" ASes.  All Gateways within the same segment are members of
     the same hub AS, and use iBGP to maintain a consistent view of all
     active routes currently in service.  The Gateways of different
     segments peer with one another using eBGP.

     Gateways maintain forwarding table entries for the SNP prefixes
     assigned to Proxy/Serves and the set of all FNP/MNP routes that are
     currently active; Gateways also maintain black-hole routes for the
     OMNI link MSPs so that OAL packets/fragments destined to non-existent
     more-specific routes are flushed from the routing system.  In this
     way, Proxy/Servers and Relays have only partial topology knowledge
     (i.e., they only maintain routing information for their directly
     associated Clients and foreign links) and they forward all other OAL
     packets/fragments to Gateways which have full topology knowledge.





  Templin                  Expires 6 October 2024                [Page 25]


  Internet-Draft                    AERO                        April 2024


     Each OMNI link segment assigns a unique sub-prefix of the MSP known
     as the "SRT prefix".  For example, a first segment could assign
     2001:db8::/48, a second could assign 2001:db8:1::/48, a third could
     assign 2001:db8:2::/48, etc.  Within each segment, each Proxy/Server
     and Gateway configures an SNP within the segment's SRT prefix, e.g.,
     the SNPs 2001:db8::/64, 2001:db8:0:1::/64 2001:db8:0:2::/64 all
     belong to the SRT prefix 2001:db8::/48.

     The administrative authorities for each segment must therefore
     coordinate to assure mutually-exclusive SNP assignments, but internal
     provisioning of SNPs is an independent local consideration for each
     administrative authority.  For each SRT prefix, the Gateway(s) that
     connect that segment assign the all-zero's address of the prefix as a
     Subnet Router Anycast (SRA) address.  For example, the SRA address
     for 2001:db8::/48 is simply 2001:db8::. All Proxy/Servers and
     Gateways also configure assign the SRA address taken from their
     specific SNP, e.g., the SRA address for the SNP 2001:db8:0:1::/64 is
     simply 2001:db8:0:1::.

     SRT prefixes (and their SNP sub-prefixes) are statically represented
     in Gateway forwarding tables.  Gateways join multiple SRT segments
     into a unified OMNI link over multiple diverse network administrative
     domains.  They support a virtual bridging service by first
     establishing forwarding table entries for their SRT prefixes either
     via standard BGP routing or static routes.  For example, if three
     Gateways ('A', 'B' and 'C') from different segments serviced
     2001:db8::/48, 2001:db8:1::/48 and 2001:db8:2::/48 respectively, then
     the forwarding tables in each gateway appear as follows:

     A:  2001:db8::/48->local, 2001:db8:1::/48->B, 2001:db8:2::/48->C

     B:  2001:db8::/48->A, 2001:db8:1::/48->local, 2001:db8:2::/48->C

     C:  2001:db8::/48->A, 2000:db8:1::/48->B, 2001:db8:2::/48->local

     These forwarding table entries rarely change, since they correspond
     to fixed infrastructure elements in their respective segments.

     FNP and MNP routes are instead dynamically advertised in the AERO
     routing system by Proxy/Servers and Relays that provide anchor points
     for their corresponding prefixes.  For example, if three Proxy/
     Servers ('D', 'E' and 'F') service the MNPs 2001:db8:1000:1::64/,
     2001:db8:1000:2::/64 and 2001:db8:1000:2::/48 then the routing system
     would include:

     D:  2001:db8:1000:1::/64

     E:  2001:db8:1000:2::/64



  Templin                  Expires 6 October 2024                [Page 26]


  Internet-Draft                    AERO                        April 2024


     F:  2001:db8:1000:3::/64

     A full discussion of the BGP-based routing system used by AERO is
     found in [I-D.ietf-rtgwg-atn-bgp].

  4.2.4.  Segment Routing Topologies (SRTs)

     The distinct GUA prefixes in an OMNI link domain identify distinct
     Segment Routing Topologies (SRTs).  Each SRT is a mutually-exclusive
     OMNI link overlay instance using a distinct set of GUAs, and emulates
     a bridged campus LAN service for the OMNI link.  In some cases (e.g.,
     when redundant topologies are needed for fault tolerance and
     reliability) it may be beneficial to deploy multiple SRTs that act as
     independent overlay instances.  A communication failure in one
     instance therefore will not affect communications in other instances.

     Each SRT is identified by a distinct GUA prefix and assigns an IPv6
     Subnet Router Anycast (SRA) address used for OMNI interface
     determination in Safety-Based Multilink (SBM) as discussed in
     [I-D.templin-intarea-omni2].  Each OMNI interface further applies
     Performance-Based Multilink (PBM) internally.

     The Gateways and Proxy/Servers of each independent SRT engage in BGP
     peerings to form a spanning tree with the Gateways in non-leaf nodes
     and the Proxy/Servers in leaf nodes.  The spanning tree is configured
     over both secured and unsecured underlay network paths.  The secured
     spanning tree is used to convey secured control messages (and
     sometimes data message extensions) between Proxy/Servers and
     Gateways, while the unsecured spanning tree forwards bulk data
     messages and/or unsecured control messages.

     Each SRT segment is identified by a unique GUA prefix used by all
     Proxy/Servers and Gateways in the segment.  Each AERO node must
     therefore discover an SRT prefix that correspondents can use to
     determine the correct segment, and must publish the SRT prefix in
     IPv6 ND messages.

     Note: The distinct GUA prefixes in an OMNI link domain can be carried
     either in a common BGP routing protocol instance for all OMNI links
     or in distinct BGP routing protocol instances for different OMNI
     links.  In some SBM environments, such separation may be necessary to
     ensure that distinct OMNI links do not include any common
     infrastructure elements as single points of failure.  In other
     environments, carrying the GUAs of multiple OMNI links within a
     common routing system may be acceptable.






  Templin                  Expires 6 October 2024                [Page 27]


  Internet-Draft                    AERO                        April 2024


  4.2.5.  Segment Routing For OMNI Link Selection

     Original IPv6 sources can direct IPv6 packets/parcels to an AERO node
     by including a standard IPv6 Segment Routing Header (SRH) [RFC8754]
     with the IPv6 SRA address for the selected OMNI link as either the
     IPv6 destination or as an intermediate hop within the SRH.  This
     allows the original source to determine the specific OMNI link SRT an
     original IPv6 packet/parcel will traverse when there may be multiple
     alternatives.

     When an AERO node processes the SRH and forwards the original IPv6
     packet/parcel to the correct OMNI interface, the OMNI interface
     writes the next IPv6 address from the SRH into the IPv6 destination
     address and decrements Segments Left.  If decrementing would cause
     Segments Left to become 0, the OMNI interface deletes the SRH before
     forwarding.  This form of Segment Routing supports Safety-Based
     Multilink (SBM).

  4.3.  OMNI Interface Characteristics

     OMNI interfaces are virtual interfaces configured over one or more
     underlay interfaces classified as follows:

     *  (M)ANET interfaces connect to a protected and secured ANET or an
        open MANET that connects to an INETs via Proxy/Servers.  The
        (M)ANET interface may be either on the same L2 link segment as a
        Proxy/Server, or separated from a Proxy/Server by multiple IP
        hops.  (Note that NATs may appear internally within a (M)ANET and
        may require NAT traversal on the path to the Proxy/Server the same
        as for the INET case.)  MANETs are special cases of ANETs in which
        multi-hop forwarding may be necessary and protected and secured
        underlay links cannot always be assumed.

     *  INET interfaces connect to an INET either natively or through one
        or several IPv4 Network Address Translators (NATs).  Native INET
        interfaces have global IP addresses that are reachable from
        correspondent on the same INET.  NATed INET interfaces typically
        have private IP addresses and connect to a private network behind
        one or more NATs with the outermost NAT providing INET access.

     *  ENET interfaces connect a Client's downstream-attached networks,
        where the Client provides forwarding services for ENET Host and
        Client communications to remote peers.  An ENET can be as simple
        as a small IoT sub-network that travels with a mobile Client to as
        complex as a large private enterprise network that the Client
        connects to a larger ANET or INET.





  Templin                  Expires 6 October 2024                [Page 28]


  Internet-Draft                    AERO                        April 2024


     *  VPN interfaces use security encapsulations (e.g.  IPsec tunnels)
        over underlay networks to connect Clients, Proxy/Servers and/or
        Gateways.  VPN interfaces provide security services at lower
        layers of the architecture (L2/L1) the same as for Direct point-
        to-point interfaces.

     *  Direct point-to-point interfaces securely connect Clients, Proxy/
        Servers and/or Gateways over physical or virtual media that does
        not transit any open Internetwork paths.  Examples include a line-
        of-sight link between a remote pilot and an unmanned aircraft, a
        fiberoptic link between Gateways, etc.

     OMNI interfaces use OAL encapsulation and fragmentation as discussed
     in Section 4.6.  OMNI interfaces use L2 encapsulation (see:
     Section 4.6) to exchange carrier packets with OMNI link neighbors
     over INET interfaces and IPsec tunnels as well as over ANET
     interfaces for which the Client and FHS Proxy/Server may be multiple
     IP hops away.  OMNI interfaces use link layer encapsulation only
     (i.e., and no other L2 encapsulations) over Direct underlay
     interfaces or ANET interfaces when the Client and FHS Proxy/Server
     are known to be on the same underlay link.

     OMNI interfaces maintain an adaptation layer neighbor cache for
     tracking per-neighbor state.  OMNI interfaces use IPv6 ND messages
     including Router Solicitation (RS), Router Advertisement (RA),
     Neighbor Solicitation (NS), Neighbor Advertisement (NA), unsolicited
     Neighbor Advertisement (uNA) and Redirect to manage the neighbor
     cache.  In environments where spoofing may be a threat, OMNI
     neighbors should invoke OAL Identification window synchronization in
     their IPv6 ND message exchanges.

     OMNI interfaces send IPv6 ND messages with an OMNI option formatted
     as specified in [I-D.templin-intarea-omni2].  The OMNI option
     includes prefix registration information, Interface Attributes and/or
     Multilink Vectors containing link information parameters for the OMNI
     interface's underlay interfaces (as well as any other per-neighbor
     information).  The presence of the OMNI option identifies each IPv6
     ND message as an adaptation layer (i.e., and not a network layer)
     control message.

     A Host's OMNI interface is configured over an underlay interface
     connected to an ENET provided by an upstream Client.  From the Host's
     perspective, the ENET appears as an ANET and the upstream Client
     appears as a Proxy/Server.  The Host does not provide OMNI
     intermediate system services and is therefore a logical termination
     point for the OMNI link.





  Templin                  Expires 6 October 2024                [Page 29]


  Internet-Draft                    AERO                        April 2024


     A Client's OMNI interface may be configured over multiple (M)ANET/
     INET underlay interfaces.  For example, common mobile handheld
     devices have both wireless local area network ("WLAN") and cellular
     wireless links.  These links are often used "one at a time" with low-
     cost WLAN preferred and highly-available cellular wireless as a
     standby, but a simultaneous-use capability could provide benefits.
     In a more complex example, aircraft frequently have many wireless
     data link types (e.g.  satellite-based, cellular, terrestrial, air-
     to-air directional, etc.) with diverse performance and cost
     properties.

     If a Client's multiple (M)ANET/INET underlay interfaces are used "one
     at a time" (i.e., all other interfaces are in standby mode while one
     interface is active), then successive IPv6 ND messages all include
     OMNI option Interface Attributes, Traffic Selector and/or Multilink
     Vector sub-options with the same underlay interface ifIndex.  In that
     case, the Client would appear to have a single underlay interface but
     with a dynamically changing link layer address.

     If the Client has multiple active (M)ANET/INET underlay interfaces,
     then from the perspective of IPv6 ND it would appear to have multiple
     link layer addresses.  In that case, IPv6 ND message OMNI options MAY
     include sub-options with different underlay interface ifIndexes.

     Proxy/Servers on the open Internet include only a single INET
     underlay interface.  INET Clients therefore discover only the L2ADDR
     information for the Proxy/Server's INET interface.  Proxy/Servers on
     a (M)ANET/INET boundary include both (M)ANET and INET underlay
     interfaces.  (M)ANET Clients therefore must discover both the (M)ANET
     and INET L2ADDR information for their Proxy/Servers.

     Gateway and Proxy/Server OMNI interfaces are configured over underlay
     interfaces that provide both secured tunnels for carrying IPv6 ND and
     BGP protocol control plane messages and open INET access for carrying
     unsecured data plane messages.  The OMNI interface configures a GUA
     and acts as an OAL source to encapsulate original IP packets/parcels,
     then fragments the resulting OAL packets, performs L2 encapsulation/
     fragmentation and sends the resulting carrier packets over the
     secured or unsecured underlay paths.  Note that Gateway and Proxy/
     Server end-to-end transport protocol sessions used by the BGP run
     directly over the OMNI interface and use SNP GUA SRA source and
     destination addresses.  The GUA addresses that appear in the original
     IP packets/parcels of a BGP protocol session may therefore be the
     same as those that appear in the OAL IPv6 encapsulation header.







  Templin                  Expires 6 October 2024                [Page 30]


  Internet-Draft                    AERO                        April 2024


  4.4.  OMNI Interface Initialization

     AERO Proxy/Servers, Clients and Hosts configure OMNI interfaces as
     their point of attachment to the OMNI link.  AERO nodes assign the
     MSPs for the link to their OMNI interfaces (i.e., as a "route-to-
     interface") to ensure that original IP packets/parcels with
     destination addresses covered by an MNP not explicitly associated
     with another interface are directed to an OMNI interface.

     OMNI interface initialization procedures for Proxy/Servers, Clients
     Hosts and Gateways are discussed in the following sections.

  4.4.1.  AERO Proxy/Server and Relay Behavior

     When a Proxy/Server enables an OMNI interface, it assigns an SNP GUA/
     ULA prefix pair.  The Proxy/Server then configures an SRA GUA
     appropriate for the given OMNI link SRT segment externally and
     configures an SRA ULA appropriate for the locally attached *NET
     internally.  The Proxy/Server also configures secured underlay
     interface tunnels and engages in BGP routing protocol sessions over
     the OMNI interface with one or more neighboring Gateways.

     The OMNI interface provides a single interface abstraction to the
     network layer, but internally serves as an NBMA nexus for exchanging
     carrier packets with other OMNI nodes over underlay interfaces and/or
     secured tunnels.  The Proxy/Server further configures a service to
     facilitate IPv6 ND exchanges with AERO Clients and manages per-Client
     neighbor cache entries and IP forwarding table entries based on
     control message exchanges.

     Relays are simply Proxy/Servers that run a dynamic routing protocol
     to redistribute routes between the OMNI interface and foreign
     networks/links (see: Section 4.2.3).  The Relay provisions MNPs and
     advertises the MSP(s) for the OMNI link over its foreign network
     interface attachments.  The Relay further provides an OMNI link
     attachment point for FNP-based topologies.

  4.4.2.  AERO Client Behavior

     When a Client enables an OMNI interface, it assigns a HHIT and sends
     OMNI-encapsulated RS messages over its (M)ANET/INET underlay
     interfaces to an FHS Proxy/Server, which allocates an SNP ULA/GUA
     address pair and optionally coordinates with a MAP Proxy/Server that
     delegates one or more MNPs.  The MAP/FHS Proxy/Servers then return an
     RA message to the Client.  The RS/RA messages may pass through one or
     more NATs in the path between the Client and FHS Proxy/Server.
     (Note: if the Client used a HHIT in its initial RS messages, it will
     discover ULAs in the corresponding RAs that it receives from FHS



  Templin                  Expires 6 October 2024                [Page 31]


  Internet-Draft                    AERO                        April 2024


     Proxy/Servers and begin using these new addresses.  If the Client is
     operating outside the context of AERO infrastructure such as in a
     Mobile Ad-hoc Network (MANET), however, it may continue using HHITs
     for Client-to-Client communications either indefinitely or at least
     until it encounters an infrastructure element that can delegate
     MNPs.)

     A Client can further extend the OMNI link over its (downstream) ENET
     interfaces where it provides a first-hop router for Hosts and other
     AERO Clients connected to the ENET.  A downstream Client that
     connects via the ENET serviced by an upstream Client can in turn
     service further downstream ENETs that connect other Hosts and
     Clients.  This OMNI link extension can be applied recursively over a
     "chain" of ENET Clients.

  4.4.3.  AERO Host Behavior

     When a Host enables an OMNI interface, it assigns an address taken
     from the ENET underlay interface which may itself be a GUA delegated
     by the upstream Client.  The Host does not assign a link-local
     address to the OMNI interface, since no autoconfiguration is
     necessary on that interface.  (As an implementation matter, the Host
     could instead configure the "OMNI interface" as a virtual sublayer of
     the ENET underlay interface itself.)

     The Host sends OMNI-encapsulated RS messages over its ENET underlay
     interface to the upstream Client, which returns encapsulated RAs and
     provides routing services in the same fashion that Proxy/Servers
     provides services for Clients.  Hosts represent the leaf end systems
     in recursively-nested chain of concatenated ENETs, i.e., they
     represent terminating endpoints for the OMNI link.

  4.4.4.  AERO Gateway Behavior

     AERO Gateways configure an OMNI interface and assign a SNP and
     corresponding SRA GUA for their OMNI link SRT segments.  Gateways
     configure underlay interface secured tunnels with Proxy/Servers in
     the same SRT segment and other Gateways in the same (or an adjacent)
     SRT segment.  Gateways then engage in a BGP routing protocol session
     with neighbors over the secured spanning tree (see: Section 4.2.3).

  4.5.  OMNI Interface Neighbor Cache Maintenance

     Each Client, Proxy/Server and Gateway OMNI interface maintains a
     network layer conceptual neighbor cache per [RFC1256] or [RFC4861]
     the same as for any IP interface.  The OMNI interface network layer
     neighbor cache is maintained through static and/or dynamic neighbor
     cache entry configurations.



  Templin                  Expires 6 October 2024                [Page 32]


  Internet-Draft                    AERO                        April 2024


     Each OMNI interface also maintains a separate internal adaptation
     layer conceptual neighbor cache that includes a Neighbor Cache Entry
     (NCE) for each of its active OAL neighbors per [RFC4861].  IPv6 ND
     messages that update the adaptation layer neighbor cache include ULA
     addresses as well as one or more OMNI options.  Throughout this
     document, the terms "neighbor cache" and "NCE" refer to this
     adaptation layer neighbor cache unless otherwise specified.

     Each OMNI interface NCE is indexed by the IPv6 address of a neighbor
     found in the ND message IPv6 header and determines the context for
     Identification verification.  Clients and Proxy/Servers maintain NCEs
     through dynamic RS/RA message exchanges, and also maintain NCEs for
     any active correspondent peers through dynamic NS/NA message
     exchanges.

     Hosts maintain NCEs for Clients and other Hosts through the exchange
     of RS/RA, NS/NA or Redirect messages.  Each NCE is indexed by the IP
     address assigned to the Host ENET interface, which is the same
     address used for L2 encapsulation (i.e., without the insertion of an
     OAL header).  This encapsulation format identifies the NCE as a Host-
     based entry where the Host is a leaf end system in the recursively
     extended OMNI link.

     Clients establish NCEs for their associated FHS and MAP Proxy/Servers
     through the exchange of RS/RA messages.  When a Client and Proxy/
     Server establish NCEs, they set a ReachableTime timer to
     REACHABLE_TIME seconds.  Clients determine the service profiles for
     their FHS and MAP Proxy/Servers by setting the NUD/ARR/RPT flags in
     RS messages and also by setting/clearing the FMT-Forward and FMT-Mode
     flags in the Interface Attributes sub-option.  When the NUD/ARR/RPT
     flags are clear, Proxy/Servers forward all NS/NA messages to the
     Client, while the Client performs mobility update signaling through
     the transmission of uNA messages to all active neighbors following a
     mobility event.  However, in some environments this may result in
     excessive NS/NA control message overhead especially for Clients
     connected to low-end data links.

     Clients can therefore set the NUD/ARR/RPT flags in RS messages they
     send to select their Proxy/Server service profiles.  If the NUD flag
     is set, the FHS Proxy/Server that forwards the RS message assumes the
     role of responding to NS messages and maintains peer NCEs associated
     with the NCE for this Client.  If the ARR flag is set, the MAP Proxy/
     Server that processes the RS message assumes the role of responding
     to NS for Address Resolution (NS(AR)) messages on behalf of this
     Client NCE.  If the RPT flag is set, the MAP Proxy/Server that
     processes the RS message becomes responsible for maintaining a
     "Report List" for each Client NCE for the source addresses of NS(AR)
     messages it forwards or responds to on behalf of this Client.



  Templin                  Expires 6 October 2024                [Page 33]


  Internet-Draft                    AERO                        April 2024


     When a Client sets the RPT flag, the MAP Proxy/Server maintains
     Report List entries based on a ReportTime timer initialized to
     REACHABLE_TIME seconds upon receipt of an NS(AR) and decremented once
     per second while no additional NS(AR)s arrive.  The MAP Proxy/Server
     then sends uNA Mobility Management (MM) messages to each Report List
     entry when it receives a Client mobility update indication (e.g.,
     through receipt of an RS with updated Interface Attributes and/or
     Traffic Selectors).  When a Report List entry ReportTime timer
     expires, the MAP Proxy/Server deletes the entry.  When a Client NCE
     timer expires, the MAP Proxy/Server deletes the NCE along with its
     associated Report List.

     Clients can also set/clear the FMT-Forward and FMT-Mode flags in the
     Interface Attributes sub-option of each RS message to express their
     desired service profile from each FHS Proxy/Server for a specific
     underlay interface.  The FHS Proxy/Server will consider the Client's
     preferences and either accept or override by setting/clearing the
     flags in the corresponding RA message reply.  Implications for these
     bit settings are discussed in [I-D.templin-intarea-omni2].

     Both the Client and its MAP Proxy/Server have full knowledge of the
     Client's current underlay Interface Attributes and Traffic Selectors,
     while FHS Proxy/Servers acting in "proxy" mode have knowledge of only
     the individual Client underlay interfaces they service.  Clients
     determine their FHS and MAP Proxy/Server service models by setting
     the NUD/ARR/RPT flags in the RS messages they send as discussed
     above.

     When an Address Resolution Source (ARS) sends an NS(AR) message
     toward an Address Resolution Target (ART) Client/Relay, the OMNI link
     routing system directs the NS(AR) to a MAP Proxy/Server for the ART.
     The MAP then either acts as an Address Resolution Responder (ARR) on
     behalf of the ART or forwards the NS(AR) to the ART which acts as an
     ARR on its own behalf.  The ARR returns an NA(AR) response to the
     ARS, which creates or updates a NCE for the ART while caching L3 and
     L2 addressing information.  The ARS then (re)sets ReachableTime for
     the NCE to REACHABLE_TIME seconds and performs NS/NA multilink
     forwarding exchanges over specific underlay interface pairs to
     determine paths for sending carrier packets directly to the ART.  The
     ARS otherwise decrements ReachableTime while no further solicited NA
     messages arrive.










  Templin                  Expires 6 October 2024                [Page 34]


  Internet-Draft                    AERO                        April 2024


     Proxy/Servers add an additional state DEPARTED to the list of NCE
     states found in Section 7.3.2 of [RFC4861].  When a Client terminates
     its association, the Proxy/Server OMNI interface sets a DepartTime
     variable for the NCE to DEPART_TIME seconds.  DepartTime is
     decremented unless a new IPv6 ND message causes the state to return
     to REACHABLE.  While a NCE is in the DEPARTED state, the Proxy/Server
     forwards OAL packets/fragments destined to the target Client to the
     Client's new FHS/MAP Proxy/Server instead.

     It is RECOMMENDED that REACHABLE_TIME be set to the default constant
     value 30 seconds as specified in [RFC4861].  It is RECOMMENDED that
     DEPART_TIME be set to the default constant value 10 seconds to accept
     any carrier packets that may be in flight.  When ReachableTime or
     DepartTime decrement to 0, the NCE is deleted.

     AERO nodes also use the value MAX_UNICAST_SOLICIT to limit the number
     of NS messages sent when a correspondent may have gone unreachable,
     the value MAX_RTR_SOLICITATIONS to limit the number of RS messages
     sent without receiving an RA and the value MAX_NEIGHBOR_ADVERTISEMENT
     to limit the number of uNAs that can be sent based on a single event.
     It is RECOMMENDED that MAX_UNICAST_SOLICIT, MAX_RTR_SOLICITATIONS and
     MAX_NEIGHBOR_ADVERTISEMENT be set to 3 the same as specified in
     [RFC4861].

     Different values for the above constants MAY be administratively set;
     however, if different values are chosen, all nodes on the link MUST
     consistently configure the same values.

  4.5.1.  OMNI ND Messages

     OMNI interfaces use IPv6 ND messages as the secured control plane
     messaging service for all adaptation layer neighbor coordination
     exchanges.  OMNI interfaces prepare IPv6 ND messages the same as for
     standard IPv6 ND, but also include a new option type termed the OMNI
     option [I-D.templin-intarea-omni2].  OMNI interfaces use ULAs/GUAs
     instead of LLAs as adaptation layer IPv6 ND message source and
     destination addresses.  This allows multiple different OMNI links/
     segments to be joined into a single link/segment at some future time
     without requiring a global renumbering event.












  Templin                  Expires 6 October 2024                [Page 35]


  Internet-Draft                    AERO                        April 2024


     OMNI interfaces normally limit the size of the IPv6 ND messages they
     send to the IPv6 minimum link MTU, but messages that include a
     substantial amount of OMNI parameters and/or IP packet/parcel
     attachments may occasionally exceed that size.  The OMNI interface
     engages IPv6 encapsulation followed by fragmentation to break IPv6 ND
     messages as large as 65535 octets into fragments no larger than 1280
     octets.  Whenever possible, OMNI interfaces should send multiple
     smaller IPv6 ND messages instead of singleton larger messages to
     minimize fragmentation.

     For each IPv6 ND message, the OMNI interface includes one or more
     OMNI options (and any other ND message options) then completely
     populates all sub-option information.  If the OMNI interface includes
     an authentication sub-option, it first writes the value 0 into the
     authentication signature field then calculates the signature
     beginning with the first IPv6 ND message octet following the header
     Checksum field and continuing over the entire length of the message.
     The OMNI interface next writes the authentication signature value
     into the appropriate OMNI authentication option field, then
     calculates the IPv6 ND message checksum per [RFC4443] beginning with
     a pseudo-header of the IPv6 header and writes the value into the
     Checksum field.  The IPv6 ND message checksum therefore provides
     integrity assurance for the message, while the authentication
     signature covers the entire packet or super-packet.  OMNI interfaces
     verify integrity and authentication of each message received, and
     process the message further only following successful verification.

     OMNI options include per-neighbor information that provides multilink
     forwarding, link layer address and traffic selector information for
     the neighbor's underlay interfaces.  This information is stored in
     both the neighbor cache and AERO Forwarding Information Base (AFIB)
     as basis for the forwarding algorithm specified in Section 4.10.  The
     information is cumulative and reflects the union of the OMNI
     information from the most recent IPv6 ND messages received from the
     neighbor.

     The OMNI option is distinct from any Source/Target Link-Layer Address
     Options (S/TLLAOs) that may appear in an IPv6 ND message according to
     the appropriate IPv6 over specific link layer specification (e.g.,
     [RFC2464]).  If both OMNI options and S/TLLAOs appear, the former
     pertains to the adaptation layer to underlay interface address
     mappings while the latter pertains to the native L2 address format of
     the underlay media.

     OMNI interface IPv6 ND messages may also include other IPv6 ND
     options.  In particular, solicitation messages may include a Nonce
     option if required for verification of advertisement replies.  If an
     OMNI IPv6 ND solicitation message includes a Nonce option, the



  Templin                  Expires 6 October 2024                [Page 36]


  Internet-Draft                    AERO                        April 2024


     advertisement reply must echo the same Nonce.  If an OMNI IPv6 ND
     solicitation message includes a Timestamp option, the recipient must
     also include a Timestamp option in its advertisement reply.  All
     unsolicited advertisement and redirect messages should include a
     Timestamp option.

     AERO Clients send RS messages to the link-scoped All-Routers
     multicast address or the ULA/GUA (SRA) address of a Proxy/Server
     while using unicast or anycast OAL/L2 addresses.  AERO Proxy/Servers
     respond by returning unicast RA messages.  During the RS/RA exchange,
     AERO Clients and Proxy/Servers include state synchronization
     parameters to establish Identification windows and other state.

     AERO Hosts and Clients on ENET underlay networks send RS messages to
     the link-scoped All-Routers multicast address, a GUA (SRA) address of
     a remote MAP Proxy/Server or the MNP (SRA) address of an upstream
     Client while using unicast or anycast OAL/L2 addresses.  The upstream
     AERO Client responds by returning a unicast RA message.

     AERO nodes use NS/NA messages for the following purposes:

     *  NS/NA(AR) messages are used for address resolution and optionally
        to establish sequence number windows.  The ARS sends an NS(AR) to
        the unicast address of the ART, and an ARR with addressing
        information for the ART returns a unicast NA(AR) that contains
        current, consistent and authentic target address resolution
        information.  NS(AR) messages include a solicited-node multicast
        destination address to distinguish them from ordinary NS messages.
        NS/NA(AR) messages must be secured.

     *  Other NS/NA message exchanges are used determine target
        reachability (NS/NA(NUD)), establish/maintain Route Optimization
        state (NS/NA(RO)) or establish/maintain multilink forwarding state
        (NS/NA(MF)).  The source sends an NS to the unicast address of the
        target while optionally including an OMNI Multilink Vector sub-
        option naming a specific underlay interface pair, and the target
        returns a responsive NA.  NS/NA messages that use an in-window
        sequence number and do not update any other state need not include
        an authentication signature but must include an IPv6 ND message
        checksum.  NS/NA messages used to establish or update NCE and/or
        AFIB state must be secured.










  Templin                  Expires 6 October 2024                [Page 37]


  Internet-Draft                    AERO                        April 2024


     *  Unsolicited NA (uNA) messages are used to signal addressing and/or
        other mobility management (uNA(MM)) neighbor state changes (e.g.,
        address changes due to mobility, signal degradation, traffic
        selector updates, etc.). uNA messages can also be also used to
        acknowledge receipt of other IPv6 ND messages (uNA(ACK)) as well
        as to securely convey ICMP error information (uNA(ERR)). uNA
        messages that update state information must be secured.

     *  NS/NA(DAD) messages are not used in AERO, since Duplicate Address
        Detection is not required.

     AERO and OMNI together support an added reliability feature not
     available in ordinary IPv6 ND messaging.  In particular, nodes can
     set the OMNI Neighbor Coordination SNR flag or Window Synchronization
     SYN flag in unicast non-solicitation IPv6 ND messages (including RA,
     NA and Redirect) to request a synchronous (but "unsolicited")
     uNA(ACK) acknowledgement response (see: [I-D.templin-intarea-omni2]).

     The node that processes an SNR/SYN message prepares the response the
     same as for an ordinary uNA as specified in [RFC4861], including the
     setting of the R/S/O flags as discussed below.  The node sets the
     uNA(ACK) Target Address to the unicast destination and uNA(ACK)
     destination address to the unicast source of the original message.

     The node then sets the uNA(ACK) source address to its own address and
     includes any necessary OMNI sub-options but MUST NOT itself set the
     SNR/SYN flags.  If the SNR/SYN message included a Nonce and/or
     Timestamp option, the node includes matching Nonce/Timestamp options
     in the uNA(ACK) response.  The node finally returns the uNA message
     to the source of the SNR/SYN message.

  4.5.2.  OMNI Neighbor Advertisement Message Flags

     As discussed in Section 4.4 of [RFC4861] NA messages include three
     flag bits R, S and O.  OMNI interface NA messages treat the flags as
     follows:

     *  R: The R ("Router") flag is set to 1 in the NA messages sent by
        all AERO forwarding nodes on the OMNI link.  (AERO Hosts are by
        definition the only non-forwarding nodes on the OMNI link and
        therefore set the R flag to 0.)

     *  S: The S ("Solicited") flag is set exactly as specified in
        Section 4.4. of [RFC4861], i.e., it is set to 1 for Solicited NAs
        and set to 0 for uNAs (both unicast and multicast).






  Templin                  Expires 6 October 2024                [Page 38]


  Internet-Draft                    AERO                        April 2024


     *  O: The O ("Override") flag is set to 0 for solicited NAs returned
        by a Proxy/Server ARR and set to 1 for all other solicited and
        unsolicited NAs.

  4.5.3.  OMNI Neighbor Window Synchronization

     In secured environments (e.g., between secured spanning tree
     neighbors, between neighbors on the same secured ANET, etc.), OMNI
     interface neighbors can exchange OAL packets that include randomly-
     initialized and monotonically-increasing (extended) Identification
     values (modulo 2**64) without window synchronization.  In
     environments where spoofing is considered a threat, OMNI interface
     neighbors instead invoke window synchronization by including OMNI
     Multilink Vector sub-options in RS/RA or NS/NA message exchanges to
     maintain send/receive window state in their respective neighbor
     caches as well as in AFIB entries of all OAL intermediate nodes in
     the forward and reverse paths as specified in
     [I-D.templin-intarea-omni2].

     In common arrangements, OAL Identification window synchronization is
     necessary for Client to Client, Client to Proxy/Server or Proxy/
     Server to Proxy/Server message exchanges conducted over unsecured
     Internetwork paths.  Conversely, Proxy/Server to Proxy/Server, Proxy/
     Server to Gateway and Gateway to Gateway message exchanges carried
     over the secured spanning tree do not require window synchronization.

     OAL end system and intermediate nodes verify Identification values of
     OAL packets that traverse the unsecured spanning tree according to
     their populated AFIB state.  This allows each OAL node to exclude
     spurious packets injected into the OMNI link from an off-path
     adversary.

  4.6.  OMNI Interface Encapsulation and Fragmentation

     When the network layer forwards an original IP packet/parcel into an
     OMNI interface, the interface locates or creates a Neighbor Cache
     Entry (NCE) that matches the destination.  The OMNI interface then
     invokes the OMNI Adaptation Layer (OAL) as discussed in
     [I-D.templin-intarea-omni2] which encapsulates the packet/parcel in
     an IPv6 header to produce an OAL packet with ULA/GUA addresses taken
     from a SNP assigned by a Proxy/Server.

     Following encapsulation, the OAL source then fragments the OAL packet
     while including an identical Identification value for each fragment
     that must be within the window for the interface pair selected for
     the neighbor.  The OAL source includes any necessary OAL IPv6
     extension headers including an identical Compressed Routing Header
     (CRH) [I-D.ietf-6man-comp-rtg-hdr] with each fragment containing AERO



  Templin                  Expires 6 October 2024                [Page 39]


  Internet-Draft                    AERO                        April 2024


     Forwarding Vector Indexes (AFVIs) as discussed in Section 4.13.  (The
     OAL source includes AFVIs no larger than 65535 as 2-octet values and
     includes larger AFVIs as 4-octet values when it prepares the CRH.)
     The OAL source can instead invoke OAL header compression by replacing
     the full OAL IPv6 header (OFH), CRH and Extended Fragment Header with
     an OAL Compressed Header (OCH) (see: [I-D.templin-intarea-omni2]).

     For messages that will traverse unsecured paths, the OAL source
     finally performs L2 encapsulation/fragmentation on each resulting OAL
     fragment to form a carrier packet, with source address set to its own
     L2 address (e.g., 192.0.2.100) and destination set to the L2 address
     of the next hop OAL intermediate system or destination (e.g.,
     192.0.2.1).  The carrier packet encapsulation format in the above
     example is shown in Figure 3:

          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |           L2 Headers          |
          ~       src = 192.0.2.100       ~
          |        dst = 192.0.2.1        |
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          ~   L2 IPv6 Extension Headers   ~
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |        OAL IPv6 Header        |
          ~        src={ULA,GUA}-1        ~
          |        dst={ULA,GUA}-2        |
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          ~   OAL IPv6 Extension Headers  ~
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |       Original IP Header      |
          ~     (first-fragment only)     ~
          ~        src={ULA,GUA}-3        ~
          |        dst={ULA,GUA}-4        |
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |                               |
          ~                               ~
          ~ Original Packet Body/Fragment ~
          ~                               ~
          |                               |
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                        Figure 3: Carrier Packet Format

     (Note that carrier packets exchanged by Hosts on ENETs do not include
     the OAL IPv6 or CRH headers, i.e., the OAL encapsulation is NULL and
     only the L2 encapsulations including any L2 IPv6 extension headers
     are included.)





  Templin                  Expires 6 October 2024                [Page 40]


  Internet-Draft                    AERO                        April 2024


     In this format, the OAL source encapsulates the original IP header
     and packet/parcel body/fragment in an OAL IPv6 header, the CRH is a
     Routing Header extension of the OAL header, the Extended Fragment
     Header identifies each fragment, and the L2 headers are prepared as
     discussed in [I-D.templin-intarea-omni2].  The OAL source sends each
     such carrier packet into the SRT unsecured spanning tree, where they
     may be forwarded over multiple OAL intermediate systems until they
     arrive at the OAL destination.  These carrier packets may themselves
     be subject to L2 fragmentation and reassembly along the path.

     The OMNI link control plane service distributes Client MNP prefix
     information that may change occasionally due to regional node
     mobility, as well as more static information for Relay FNPs and per-
     segment SNPs that rarely change.  OMNI link Gateways and Proxy/
     Servers use the information to establish and maintain a forwarding
     plane spanning tree that connects all nodes on the link.  The
     spanning tree supports a virtual bridging service according to link
     layer (instead of network layer) information, but may often include
     longer paths than necessary.

     Each OMNI interface therefore also includes an AERO Forwarding
     Information Base (AFIB) that caches AERO Forwarding Vectors (AFVs)
     which can provide both carrier packet Identification context and more
     direct forwarding "shortcuts" that avoid strict spanning tree paths.
     As a result, the spanning tree is always available but OMNI
     interfaces can often use the AFIB entries established through route
     optimization to greatly improve performance and reduce load on
     critical infrastructure elements.

     For OAL packets/fragments undergoing L2 re-encapsulation at an OAL
     intermediate system, the OMNI interface performs L2 reassembly/
     decapsulation followed by Identification verification and OAL
     reassembly only if the OAL packet/fragment is addressed to itself.
     The OMNI interface then decrements the OAL IPv6 header Hop Limit and
     discards the packet/fragment if the Hop Limit reaches 0.  Otherwise,
     the OMNI interface updates the OAL addresses if necessary, includes
     an appropriate Identification, performs OAL fragmentation then for
     each OAL fragment performs L2 encapsulation/fragmentation to produce
     carrier packets appropriate for next segment forwarding.

     When an FHS Gateway forwards an OAL packet/fragment to an LHS Gateway
     over the unsecured spanning tree, it reconstructs the OAL header
     based on AFV state, inserts a CRH immediately following the OAL
     header and adjusts the OAL payload length and destination address
     field.  The FHS Gateway includes a single AFVI in the CRH that each
     intermediate Gateway on the path to the LHS Gateway can use to search
     its AFIB, then forwards the OAL packet/fragment over the unsecured
     spanning tree.  Each intermediate Gateway in succession searches for



  Templin                  Expires 6 October 2024                [Page 41]


  Internet-Draft                    AERO                        April 2024


     an AFV based on the included AFVI, then replaces the AFVI with a
     value that will be recognized by the next Gateway.  When the LHS
     Gateway receives the OAL packet/fragment, it locates the AFV for the
     next hop based on the CRH AFVI then forwards the OAL packet/fragment
     to the LHS Proxy/Server or the Client itself.

     OAL packets/fragments that travel over secured spanning tree hops do
     not include OMNI L2 encapsulations.  They are instead admitted into
     secured links such as IPsec tunnels or direct links where they may be
     subject to L2 security encapsulations as secured carrier packets.
     (Note that OMNI protocol L2 encapsulations could be used above the L2
     security services, but this could result in excessive encapsulation
     in some instances.)

  4.7.  OMNI Interface Decapsulation

     When an OAL node receives OAL packets/fragments addressed to another
     node, it discards the L2 headers and includes new L2 headers
     appropriate for the next hop in the forwarding path to the OAL
     destination (after first performing any necessary L2 fragmentation or
     reassembly).  The node then sends these new carrier packets into the
     next hop underlay interface.

     When an OAL node receives OAL packets/fragments addressed to itself,
     it performs L2 reassembly/decapsulation, verifies the Identification,
     then performs OAL reassembly/decapsulation to obtain the original OAL
     packet or super-packet (see: [I-D.templin-intarea-omni2]).  Next, if
     the enclosed original IP packet(s)/parcel(s) are destined either to
     itself or to a destination reached via an interface other than the
     OMNI interface, the OAL node discards the OAL encapsulation and
     forwards the original IP packet(s)/parcel(s) to the network layer.

     If the original IP packet(s)/parcel(s) are destined to another node
     reached by the OMNI interface, the OAL node instead changes the OAL
     source to its own address, changes the OAL destination to the address
     of the next-hop node over the OMNI interface, decrements the Hop
     Limit, then performs L2 encapsulation/fragmentation and forwards
     these new carrier packets into the next hop underlay interface.

     Further OMNI link decapsulation details are specified in
     [I-D.templin-intarea-omni2].  Further OMNI link forwarding procedures
     are specified in Section 4.10.

  4.8.  OMNI Interface Data Origin Authentication

     AERO nodes employ simple data origin authentication procedures.  In
     particular:




  Templin                  Expires 6 October 2024                [Page 42]


  Internet-Draft                    AERO                        April 2024


     *  AERO Gateways and Proxy/Servers accept carrier packets received
        from the secured spanning tree.

     *  AERO Proxy/Servers and Clients accept carrier packets and original
        IP packets/parcels that originate from within the same secured
        ANET.

     *  AERO Clients and Relays accept original IP packets/parcels from
        downstream network correspondents based on ingress filtering.

     *  AERO Hosts, Clients, Relays, Proxy/Servers and Gateways verify
        carrier packet L2 encapsulation addresses according to
        [I-D.templin-intarea-omni2].

     *  OAL end systems and intermediate systems forward/accept OAL
        packets/fragments with Identification values within the current
        window for the OAL source neighbor for a specific underlay
        interface pair and drop any packets with out-of-window
        Identification values.

     AERO nodes silently drop any packets/parcels that do not satisfy the
     above data origin authentication procedures.  Further security
     considerations are discussed in Section 7.

  4.9.  OMNI Interface MTU

     The OMNI interface observes the link nature of tunnels, including the
     Maximum Transmission Unit (MTU), Effective MTU to Receive (EMTU_R)
     and the role of fragmentation and reassembly
     [I-D.ietf-intarea-tunnels].  The OMNI interface employs an OMNI
     Adaptation Layer (OAL) that accommodates multiple underlay links with
     diverse MTUs.  OMNI interface packet sizing considerations are
     specified in [I-D.templin-intarea-omni2], where the OMNI interface
     MTU can essentially be considered "unlimited".

     When the network layer presents an original IP packet/parcel to the
     OMNI interface, the OAL source encapsulates and fragments the packet/
     parcel if necessary.  When the network layer presents the OMNI
     interface with multiple original IP packets/parcels bound to the same
     OAL destination, the OAL source can concatenate them as a single OAL
     super-packet as discussed in [I-D.templin-intarea-omni2] before
     applying fragmentation.  The OAL source then submits each OAL
     fragment for L2 encapsulation/fragmentation for transmission as
     carrier packets over an underlay interface connected to either a
     physical link (e.g., Ethernet, WiFi, Cellular, etc.) or a virtual
     link such as an Internet or higher-layer tunnel.





  Templin                  Expires 6 October 2024                [Page 43]


  Internet-Draft                    AERO                        April 2024


     Note: Although a CRH may be inserted or removed by a Gateway in the
     path (see: Section 4.10.4), this does not interfere with the
     destination's ability to reassemble since the CRH is not included in
     the fragmentable part and its removal/transformation does not
     invalidate fragment header information.

  4.10.  OMNI Interface Forwarding Algorithm

     Original IP packets/parcels enter a node's OMNI interface either from
     the network layer (i.e., from a local application or the IP
     forwarding system) while carrier packets enter from the link layer
     (i.e., from an OMNI interface neighbor).  All original IP packets/
     parcels and carrier packets entering a node's OMNI interface first
     undergo data origin authentication as discussed in Section 4.8.
     Those that satisfy data origin authentication are processed further,
     while all others are dropped silently.

     Original IP packets/parcels that enter the OMNI interface from the
     network layer are forwarded to an OMNI interface neighbor using OAL
     encapsulation and fragmentation to produce carrier packets for
     transmission over underlay interfaces.  (If forwarding state
     indicates that the original IP packet/parcel should instead be
     forwarded back to the network layer, the packet/parcel is dropped to
     avoid looping).  Carrier packets that enter the OMNI interface from
     the link layer are either re-encapsulated and re-admitted into the
     link layer, or reassembled and forwarded to the network layer where
     they are subject to either local delivery or IP forwarding.

     When the network layer of a router forwards an original IP packet/
     parcel into the OMNI interface, it decrements the TTL/Hop Limit
     following standard IP router conventions.  Once inside the OMNI
     interface, however, the OAL does not further decrement the original
     IP packet/parcel TTL/Hop Limit since its adaptation layer forwarding
     actions occur below the network layer.  The original IP packet/
     parcel's TTL/Hop Limit will therefore be the same when it exits the
     destination OMNI interface as when it first entered the source OMNI
     interface.

     When an OAL intermediate system receives a carrier packet, it
     performs L2 reassembly/decapsulation to obtain the enclosed OAL
     packet/fragment.  When the intermediate system forwards an OAL
     packet/fragment not addressed to itself, it decrements the OAL Hop
     Limit without decrementing the network layer IP TTL/Hop Limit.  If
     decrementing would cause the OAL Hop Limit to become 0, the OAL
     intermediate system drops the OAL packet/fragment.  This ensures that
     original IP packet(s)/parcel(s) cannot enter an endless loop.





  Templin                  Expires 6 October 2024                [Page 44]


  Internet-Draft                    AERO                        April 2024


     OMNI interfaces may have multiple underlay interfaces and/or neighbor
     cache entries for neighbors with multiple underlay interfaces (see
     Section 4.3).  The OAL uses Interface Attributes and/or Traffic
     Selectors to select an outbound underlay interface for each OAL
     packet and also to select segment routing and/or link layer
     destination addresses based on the neighbor's target underlay
     interfaces.  AERO implementations SHOULD permit network management to
     dynamically adjust Traffic Selector values at runtime.

     If an OAL packet/fragment matches the Interface Attributes and/or
     Traffic Selectors of multiple outgoing interfaces and/or neighbor
     interfaces, the OMNI interface replicates the packet and sends a
     separate copy via each of the (outgoing / neighbor) interface pairs;
     otherwise, it sends a single copy via an interface with the best
     matching attributes/selectors.  (While not strictly required, the
     likelihood of successful reassembly may improve when the OMNI
     interface sends all fragments of the same fragmented OAL packet/
     fragment consecutively over the same underlay interface pair to avoid
     complicating factors such as delay variance and reordering.)  AERO
     nodes keep track of which underlay interfaces are currently
     "reachable" or "unreachable", and only use "reachable" interfaces for
     forwarding purposes.

     In addition to standard forwarding based on Interface Attributes and/
     or Traffic Selectors, nodes may employ a policy engine that would
     provide further guidance to the forwarding algorithm.  For example
     the policy engine may suggest a load balancing profile over multiple
     underlay interface pairs, with portions of a traffic flow spread
     between multiple paths according to Equal Cost MultiPath or Link
     Aggregation Groups (LAGs) [RFC6438] (note that Interface Attributes
     include an underlay interface group identifier).  Other policies may
     suggest the use of paths with the least cost, best performance, etc.
     This document therefore specifies mechanisms without mandating any
     particular policies.

     All Clients, Proxy/Servers and Gateways serve as OAL intermediate
     nodes for the purpose of forwarding OAL packets/fragments over the
     unsecured spanning tree based on AFIB entries.  When an OAL
     intermediate node forwards an OAL packet/fragment with an L2 source
     address and AFVI that matches an AFV, the node first verifies that
     the Identification is in sequence.  The OAL intermediate node then
     rewrites the packet's AFVI with a value that will be recognized by
     the next OAL hop and forwards the packet.  The chain of OAL source,
     intermediate and destination nodes may therefore traverse many
     Clients, Proxy/Servers and Gateways on the path.






  Templin                  Expires 6 October 2024                [Page 45]


  Internet-Draft                    AERO                        April 2024


     The following sections discuss the OMNI interface-specific forwarding
     algorithms for Hosts, Clients, Proxy/Servers and Gateways.  In the
     following discussion, an original IP packet/parcel's destination
     address is said to "match" if it is the same as a cached address, or
     if it is covered by a cached FNP/SNP/MNP.

  4.10.1.  Host Forwarding Algorithm

     When an original IP packet/parcel enters a Host's OMNI interface from
     the network layer the Host searches for a NCE that matches the
     destination.  If there is a matching NCE, the Host performs OMNI L2
     encapsulation/fragmentation as discussed in
     [I-D.templin-intarea-omni2] then forwards the resulting carrier
     packets into the ENET addressed to the L2 address of the neighbor.
     If there is no match, the host instead sends the carrier packets to
     its upstream Client.

     After sending carrier packets, the Host may receive an OAL Redirect
     message from its upstream Client to inform it of another AERO node on
     the same ENET that would provide a better first hop.  The Host
     authenticates the Redirect message, then updates its neighbor cache
     accordingly.

  4.10.2.  Client Forwarding Algorithm

     When an original IP packet/parcel enters a Client's OMNI interface
     from the network layer the Client searches for a NCE that matches the
     destination.  If there is a matching NCE for a neighbor reached via
     an (M)ANET/INET interface (i.e., an upstream interface), the Client
     selects one or more "reachable" neighbor interfaces in the entry for
     forwarding purposes.  Otherwise, the Client performs OAL
     encapsulation and fragmentation if necessary, forwards the resulting
     OAL packet/fragments to an FHS Proxy/Server, then either invokes
     address resolution and multilink forwarding procedures per
     Section 4.13 or allows the FHS Proxy/Server to invoke these
     procedures on its behalf.  If there is a matching NCE for a neighbor
     reached via an ENET interface (i.e., a downstream interface), the
     Client instead forwards the original IP packet/parcel to the
     downstream Host or Client using L2 encapsulation and fragmentation if
     necessary.

     When a carrier packet enters a Client's OMNI interface from the link
     layer, the Client performs L2 reassembly/decapsulation if necessary
     to obtain the OAL packet/fragment then examines the OAL destination.
     If the OAL destination matches one of the Client's ULAs the Client
     (acting as an OAL destination) verifies that the Identification is
     in-window for the matching AFV, then reassembles/decapsulates as
     necessary and delivers the original IP packet/parcel to the network



  Templin                  Expires 6 October 2024                [Page 46]


  Internet-Draft                    AERO                        April 2024


     layer.  If the OAL destination matches a NCE for a dependent peer
     Client on an ENET interface, the Client instead forwards the OAL
     packet/fragment to the peer while decrementing the OAL Hop Limit.  If
     the OAL destination matches a NCE for a Host on an ENET interface,
     the Client instead reassembles then forwards the original IP packet/
     parcel to the Host while using L2 encapsulation/fragmentation (i.e.,
     without invoking the OAL) if necessary.  If the OAL destination does
     not match, the Client drops the original IP packet/parcel and MAY
     return a network layer ICMP Destination Unreachable message subject
     to rate limiting (see: Section 4.11).

     When a Client forwards an OAL packet/fragment from an ENET Host to a
     neighbor connected to the same ENET, it also returns a Redirect
     message to inform the Host that it can reach the neighbor directly as
     an ENET peer.

     Note: Clients and their FHS Proxy/Server (and other Client) peers can
     exchange original IP packets/parcels over ANET underlay interfaces
     using OMNI L2 encapsulation/fragmentation without invoking the OAL,
     since the ANET is secured at the link and physical layers.  By
     forwarding original IP packets/parcels without invoking the OAL, the
     ANET peers use the same L2 encapsulation/fragmentation procedures as
     specified for Hosts above.

     Note: Clients and their FHS Proxy/Server (and other Client) peers can
     exchange original IPv6 packets/parcels with SNP ULA addresses over
     MANET underlay interfaces and without invoking OAL or L2
     encapsulation; the IPv6 header of the packet itself includes
     addresses specific to the OAL layer and can employ OAL-style
     fragmentation and header compression as necessary.  ULA-based
     communications are sufficient for Client-to-Client communications
     within the MANET, while packets that enter or exit the MANET via a
     FHS Proxy/Server may be subject to NPTv6 [I-D.bctb-6man-rfc6296-bis].

     Note: The forwarding table entries established in peer Clients of a
     MANET multihop forwarding region are based on ULAs and/or HHITs used
     to seed the multihop routing protocols.  When ULAs are used, the
     subnet ID in the ULA /64 prefix provides topological relevance for
     the multihop forwarding region, while the 64-bit Interface Identifier
     encodes the 1x1 mapping of the MANET-internal ULA to the MANET-
     external GUA maintained by the Proxy/Server that configures the GUA/
     ULA SNP.

     Note: Clients within MANETs support Client-to-Client multihop
     forwarding when necessary to reach destinations or FHS Proxy/Servers
     that may be multiple OAL hops away.  In this way, forwarding Clients
     act as OAL intermediate nodes and forward based on AFV state that is
     indexed by the AFVIs included in each OAL packet/fragment.



  Templin                  Expires 6 October 2024                [Page 47]


  Internet-Draft                    AERO                        April 2024


  4.10.3.  Proxy/Server and Relay Forwarding Algorithm

     When the network layer admits an original IP packet/parcel into a
     Proxy/Server's OMNI interface, the OAL drops the packet/parcel to
     avoid looping if forwarding state indicates that it should be
     forwarded back to the network layer.  Otherwise, the OAL examines the
     IP destination address to determine if it matches the SNP SRA GUA of
     a neighboring Gateway found in the OMNI interface's network layer
     neighbor cache.  If so, the Proxy/Server performs OAL encapsulation
     and fragmentation then performs L2 encapsulation/fragmentation and
     forwards the resulting carrier packets to the Gateway over a secured
     link (e.g., an IPsec tunnel, Direct link, etc.) to support control
     plane functions such as the operation of the BGP routing protocol.
     If the destination matches an FNP/MNP associated with a (foreign)
     Proxy/Server or Client, the (local) Proxy/Server instead assumes the
     Relay role and forwards the original IP packet/parcel in a similar
     manner as for Clients.  Specifically, if there is a matching NCE the
     Proxy/Server selects one or more "reachable" neighbor interfaces in
     the entry for forwarding purposes; otherwise, the Proxy/Server
     performs OAL encapsulation/fragmentation followed by L2
     encapsulation/fragmentation and forwards the resulting carrier
     packets while invoking address resolution and multilink forwarding
     procedures per Section 4.13.

     When the Proxy/Server receives/reassembles carrier packets on
     underlay interfaces that contain OAL packets/fragments with both a
     source and destination OAL address that correspond to the same
     Client's delegated MNP or SNP GUA, the Proxy/Server drops the carrier
     packets regardless of their OMNI link point of origin.  The Proxy/
     Server also drops original IP packets/parcels received on underlay
     interfaces either directly from a (M)ANET Client or following
     reassembly of carrier packets received from a (M)ANET/INET Client if
     the original IP destination corresponds to the same Client's
     delegated MNP or SNP GUA.  Proxy/Servers also drop carrier packets
     that contain OAL packets/fragments with foreign OAL destinations that
     do not match the SNP/MNP GUA associated with one of their local
     (M)ANET/INET Clients.  These checks are essential to prevent
     forwarding inconsistencies from accidentally or intentionally
     establishing endless loops that could congest nodes and/or (M)ANET/
     INET links.

     Proxy/Servers process carrier packets that contain OAL packets/
     fragments with OCH headers or with destinations that match their SNP
     SRA ULA/GUA and also include a CRH header that encodes AFVI
     information.  The Proxy/Server examines the L2 source address and
     AFVI to locate the corresponding AFV entry in the AFIB.  If the
     carrier packets were not received from the secured spanning tree, the
     Proxy/Server must then verify that the L2 address is "trusted"



  Templin                  Expires 6 October 2024                [Page 48]


  Internet-Draft                    AERO                        April 2024


     according to the AFV.  If the carrier packets were trusted, the
     Proxy/Server then forwards them according to the AFV state while
     decrementing the OAL packet/fragment Hop Limit.

     For OAL packets/fragments with destinations that match their SNP SRA
     ULA/GUA but do not include a CRH/OCH, the Proxy/Server instead
     performs L2 reassembly/decapsulation, verifies the Identification and
     performs OAL reassembly to obtain the original IP packet/parcel.  For
     data packets/parcels addressed to its own GUA that arrived via the
     secured spanning tree, the Proxy/Server delivers the original IP
     packet/parcel to the network layer to support secured BGP routing
     protocol control messaging.  For data packets/parcels originating
     from one of its dependent Clients, the Proxy/Server instead performs
     OAL encapsulation/fragmentation followed by L2 encapsulation/
     fragmentation and sends the resulting carrier packets while invoking
     address resolution and multilink forwarding procedures per
     Section 4.13.  For IPv6 ND control messages, the Proxy/Server instead
     authenticates the message and processes it as specified in later
     sections of this document while updating neighbor cache and/or AFIB
     state accordingly.

     When the Proxy/Server receives a carrier packet that contains an OAL
     packet/fragment with OAL destination set to an SNP ULA or MNP GUA of
     one of its Client neighbors established through RS/RA exchanges, it
     accepts the carrier packet only if data origin authentication
     succeeds.  If the NCE state is DEPARTED, the Proxy/Server changes the
     OAL destination address to the SNP SRA GUA of the new Proxy/Server,
     decrements the OAL Hop Limit, then performs L2 encapsulation/
     fragmentation and forwards the resulting carrier packets into the
     spanning tree which will eventually deliver them to the new Proxy/
     Server.  If the neighbor cache state for the Client is REACHABLE and
     the Proxy/Server is a MAP responsible for serving as the Client's
     address resolution responder and/or default router, it verifies the
     Identification then submits the OAL packet/fragment for reassembly
     then decapsulates and processes the resulting IPv6 ND message or
     original IP packet/parcel accordingly.  Otherwise, the Proxy/Server
     decrements the OAL Hop Limit, performs L2 encapsulation/fragmentation
     and sends the carrier packets to the Client which must then perform
     data origin verification and reassembly.  (In the latter case, the
     Client may receive fragments of the same original IP packet/parcel
     from different Proxy/Servers but this will not interfere with
     reassembly.)

     When the Proxy/Server receives a carrier packet that contains an OAL
     packet/fragment with OAL destination set to a FNP address that does
     not match the MSP, it accepts the carrier packet only if data origin
     authentication succeeds and if there is a network layer forwarding
     table entry for the FNP.  The Proxy/Server then performs L2



  Templin                  Expires 6 October 2024                [Page 49]


  Internet-Draft                    AERO                        April 2024


     reassembly/decapsulation, verifies the Identification, performs OAL
     reassembly/decapsulation to obtain the original IP packet/parcel,
     then presents it to the network layer (as a Relay) where it will be
     delivered according to standard IP forwarding.

     Clients and their FHS Proxy/Server peers can exchange original IP
     packets/parcels over ANET underlay interfaces using L2 encapsulation
     IPv6 Extended Fragment Headers only and no OAL addressing
     information, since the ANET is secured at the link and physical
     layers.  (For packets that do not require fragmentation, the peers
     can even omit the Extended Fragment Header.)  FHS Proxy/Servers will
     then supply an OAL Full (OFH) or Compressed (OCH) header when they
     forward ANET Client original IP packets/parcels toward final
     destinations located in other networks.

     Clients and their FHS Proxy/Server peers can exchange original IPv6
     packets/parcels that use SNP ULAs over MANET underlay interfaces
     using OAL forwarding only over the multihop MANET and without
     including an additional OAL encapsulation header.  The FHS Proxy/
     Server will apply NPTv6 (followed by OAL encapsulation and
     fragmentation if necessary) to forward packets leaving the MANET and
     addressed to an FNP/MNP/SNP GUA correspondent in a remote network.
     The FHS Proxy/Server will similarly apply OAL reassembly and
     decapsulation followed by NPTv6 to forward packets entering the MANET
     and addressed to an SNP ULA/GUA Client in the local network.

     Proxy/Servers forward OAL packets/fragments received in secure
     control plane carrier packets via the SRT secured spanning tree and
     forward other OAL packets/fragments via the unsecured spanning tree.
     When a Proxy/Server receives a carrier packet from the secured
     spanning tree, it considers the message as authentic without having
     to verify network or higher layer authentication signatures.  When a
     Proxy/Server receives a carrier packet from the unsecured spanning
     tree, it applies data origin authentication itself and/or forwards
     the enclosed unsecured OAL contents toward the destination which must
     apply data origin authentication on its own behalf.

     If the Proxy/Server has multiple original IP packets/parcels to send
     to the same neighbor, it can concatenate them as a single OAL super-
     packet [I-D.templin-intarea-omni2].











  Templin                  Expires 6 October 2024                [Page 50]


  Internet-Draft                    AERO                        April 2024


  4.10.4.  Gateway Forwarding Algorithm

     When the network layer admits an original IP packet/parcel into the
     Gateway's OMNI interface, the OAL drops the packet if routing
     indicates that it should be forwarded back to the network layer to
     avoid looping.  Otherwise, the Gateway examines the IP destination
     address to determine if it matches the SNP SRA GUA of a neighboring
     Gateway or Proxy/Server by examining the OMNI interface's network
     layer neighbor cache.  If so, the Gateway performs OAL encapsulation/
     fragmentation followed by L2 encapsulation/fragmentation and forwards
     the resulting carrier packets to the neighboring Gateway or Proxy/
     Server over a secured link (e.g., an IPsec tunnel, etc.) to support
     the operation of control plane functions (including the BGP routing
     protocol) between OAL neighbors.

     Gateways forward OAL packets/fragments reassembled from spanning tree
     carrier packets while decrementing the OAL Hop Limit but not the
     original IP header TTL/Hop Limit.  Gateways send carrier packets that
     contain OAL packets/fragments with critical IPv6 ND control messages
     or BGP routing protocol control messages via the SRT secured spanning
     tree, and may send other carrier packets via the secured/unsecured
     spanning tree or via more direct paths according to AFIB information.
     When the Gateway receives a carrier packet, it reassembles/
     decapsulates to obtain the OAL packet/fragment then searches for an
     AFIB entry that matches the OAL header AFVI or an IP forwarding table
     entry that matches the OAL destination address.

     Gateways process carrier packets that contain OAL packets/fragments
     with OAL destinations that do not match their SNP/SRT SRA GUA in the
     same manner as for traditional IP forwarding within the OAL, i.e.,
     they forward packets not explicitly addressed to themselves.
     Gateways locally process OAL packets/fragments with OCH headers or
     full OAL headers with their SNP/SRT SRA GUA as the OAL destination.
     If the OAL packet/fragment contains an OCH or a full OAL header with
     a CRH extension, the Gateway examines the AFVI to locate the AFV
     entry in the AFIB for next hop forwarding.  If an AFV is found, the
     Gateway uses the next hop AFVI to forward the OAL packet/fragment to
     the next hop while decrementing the OAL Hop Limit but without
     reassembling.  When the Gateway forwards the OAL packet/fragment, it
     rewrites the OCH/CRH AFVI with the value it will represent to the
     next OAL hop.

     If the OAL packet/fragment includes a full OAL header addressed to
     itself but does not include an AFVI, the Gateway instead reassembles
     if necessary and processes the OAL packet further.  The Gateway first
     determines whether the OAL packet includes an NS/NA message then
     processes the message according to the multilink forwarding
     procedures discussed in Section 4.13.  If the carrier packets arrived



  Templin                  Expires 6 October 2024                [Page 51]


  Internet-Draft                    AERO                        April 2024


     over the secured spanning tree and the enclosed OAL packets/fragments
     are addressed to its SNP/SRT SRA GUA, the Gateway instead reassembles
     then discards the OAL header and forwards the original IP packet/
     parcel to the network layer to support secured BGP routing protocol
     control messaging.  The Gateway instead drops all other OAL packets.

     Gateways forward OAL packets/fragments received in carrier packets
     that arrived from a first segment via the secured spanning tree to
     the next segment also via the secured spanning tree.  Gateways
     forward OAL packets/fragments received in carrier packets that
     arrived from a first segment via the unsecured spanning tree to the
     next segment also via the unsecured spanning tree.  Gateways
     configure a single IPv6 routing table that determines the next hop
     for a given OAL destination, where the secured/unsecured spanning
     tree is determined through the selection of the underlay interface to
     be used for transmission (e.g., an IPsec tunnel or an open INET
     interface).

     As for Proxy/Servers, Gateways must verify that the L2 source
     addresses of carrier packets not received from the secured spanning
     tree are "trusted" before forwarding according to an AFV (otherwise,
     the carrier packet must be dropped).

  4.11.  OMNI Interface Error Handling

     When an AERO node admits an original IP packet/parcel into the OMNI
     interface, it may receive link and/or network layer error
     indications.  The AERO node may also receive OMNI link error
     indications in OAL-encapsulated uNA(ERR) messages that include
     authentication signatures.

     A link layer error indication is an ICMP error message generated by a
     router in an underlay network on the path to the neighbor or by the
     neighbor itself.  The message includes an IP header with the address
     of the node that generated the error as the source address and with
     the link layer address of the AERO node as the destination address.

     The IP header is followed by an ICMP header that includes an error
     Type, Code and Checksum.  Valid type values include "Destination
     Unreachable", "Time Exceeded", "Parameter Problem" etc.
     [RFC0792][RFC4443].

     The ICMP header is followed by the leading portion of the carrier
     packet that generated the error, also known as the "packet-in-error".
     For ICMPv6, [RFC4443] specifies that the packet-in-error includes:
     "As much of invoking packet as possible without the ICMPv6 packet
     exceeding the minimum IPv6 MTU" (i.e., no more than 1280 bytes).  For
     ICMPv4, [RFC0792] specifies that the packet-in-error includes:



  Templin                  Expires 6 October 2024                [Page 52]


  Internet-Draft                    AERO                        April 2024


     "Internet Header + 64 bits of Original Data Datagram", however
     [RFC1812] Section 4.3.2.3 updates this specification by stating: "the
     ICMP datagram SHOULD contain as much of the original datagram as
     possible without the length of the ICMP datagram exceeding 576
     bytes".

     The link layer error message format is shown in Figure 4:

          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |                               |
          ~    IP Header of link layer    ~
          ~         error message         ~
          |                               |
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          ~          ICMP Header          ~
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ---
          |                               |   P
          ~   carrier packet L2 and OAL   ~   a
          ~     encapsulation headers     ~   c
          |                               |   k
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   e
          |                               |   t
          ~original IP packet/parcel hdrs ~
          ~    (first-fragment only)      ~   i
          |                               |   n
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |                               |   e
          ~    Portion of the body of     ~   r
          ~ the original IP packet/parcel ~   r
          ~       (all fragments)         ~   o
          |                               |   r
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ---

            Figure 4: OMNI Interface Link-Layer Error Message Format

     The AERO node rules for processing these link layer error messages
     are as follows:

     *  When an AERO node receives a link layer Parameter Problem message,
        it processes the message the same as described as for ordinary
        ICMP errors in the normative references [RFC0792][RFC4443].

     *  When an AERO node receives persistent link layer Time Exceeded
        messages, the IP ID field may be wrapping before earlier fragments
        awaiting reassembly have been processed.  In that case, the node
        should begin including integrity checks and/or institute rate
        limits for subsequent carrier packets.




  Templin                  Expires 6 October 2024                [Page 53]


  Internet-Draft                    AERO                        April 2024


     *  When an AERO node receives persistent link layer Destination
        Unreachable messages in response to carrier packets that it sends
        to one of its neighbor correspondents, the node should process the
        message as an indication that a path may be failing, and
        optionally initiate NUD over that path.  If it receives
        Destination Unreachable messages over multiple paths, the node
        should allow future carrier packets destined to the correspondent
        to flow through a default route and re-initiate route
        optimization.

     *  When an AERO Client receives persistent link layer Destination
        Unreachable messages in response to carrier packets that it sends
        to one of its neighbor Proxy/Servers, the Client should mark the
        path as unusable and use another path.  If it receives Destination
        Unreachable messages on many or all paths, the Client should
        associate with a new Proxy/Server and release its association with
        the old Proxy/Server as specified in Section 4.15.5.

     *  When an AERO Proxy/Server receives persistent link layer
        Destination Unreachable messages in response to carrier packets
        that it sends to one of its neighbor Clients, the Proxy/Server
        should mark the underlay path as unusable and use another underlay
        path.

     *  When an AERO Proxy/Server receives link layer Destination
        Unreachable messages in response to a carrier packet that it sends
        to one of its permanent neighbors, it treats the messages as an
        indication that the path to the neighbor may be failing.  However,
        the dynamic routing protocol should soon re-converge and correct
        the temporary outage.

     When an AERO Gateway receives a carrier packet for which the network
     layer destination address is covered by an MSP assigned to a black-
     hole route, the Gateway drops the carrier packet if there is no more-
     specific routing information for the destination and returns an OMNI
     interface Destination Unreachable message subject to rate limiting.

     When an AERO node receives a carrier packet for which OAL reassembly
     is currently congested, it returns an OMNI interface Packet Too Big
     (PTB) message as discussed in [I-D.templin-intarea-omni2] (note that
     the PTB messages could indicate either "hard" or "soft" errors).

     AERO nodes include ICMPv6 error messages intended for an OAL source
     as sub-options in the OMNI option of secured uNA(ERR) messages.  When
     the OAL source receives the uNA(ERR) message, it can extract the
     ICMPv6 error message enclosed in the OMNI option and either process
     it locally or translate it into a network layer error to return to
     the original source.



  Templin                  Expires 6 October 2024                [Page 54]


  Internet-Draft                    AERO                        April 2024


  4.12.  AERO Mobility Service Coordination

     AERO nodes observes the Router Discovery and Prefix Registration
     specifications found in [I-D.templin-intarea-omni2].  AERO nodes
     further coordinate their autoconfiguration actions with the mobility
     service as discussed in the following sections.

  4.12.1.  AERO Service Model

     Each AERO Proxy/Server on the OMNI link is configured to respond to
     Client prefix delegation/registration requests.  Each Proxy/Server is
     provisioned with a database of MNP-to-Client ID mappings for all
     Clients enrolled in the AERO service, as well as any information
     necessary to authenticate each Client.  The Client database is
     maintained by a central administrative authority for the OMNI link
     and securely distributed to all Proxy/Servers, e.g., via the
     Lightweight Directory Access Protocol (LDAP) [RFC4511], via static
     configuration, etc.  Clients receive the same service regardless of
     the Proxy/Servers they select.

     Clients associate each of their (M)ANET/INET underlay interfaces with
     FHS Proxy/Servers.  Each FHS Proxy/Server locally services one or
     more of the Client's underlay interfaces, and the Client typically
     selects one among them to serve as the MAP Proxy/Server (the Client
     may instead select a "third-party" MAP Proxy/Server that does not
     directly service any of its underlay interfaces).  All of the
     Client's other FHS Proxy/Servers forward proxyed copies of RS/RA
     messages between the MAP Proxy/Server and Client without assuming the
     MAP role functions themselves.

     Each Client typically associates with a single MAP Proxy/Server,
     while all other Proxy/Servers are candidates for providing the MAP
     role for other Clients.  An FHS Proxy/Server assumes the MAP role
     when it receives an RS message with its own SNP SRA GUA/ULA or link-
     scoped All-Routers multicast as the destination.  An FHS Proxy/Server
     assumes the proxy role when it receives an RS message with the SNP
     SRA GUA of another Proxy/Server as the destination.  (An FHS Proxy/
     Server can also assume the proxy role when it receives an RS message
     addressed to link-scoped All-Routers multicast if it can determine
     the SNP SRA GUA of a better candidate Proxy/Server to serve as a
     MAP.)










  Templin                  Expires 6 October 2024                [Page 55]


  Internet-Draft                    AERO                        April 2024


     Hosts and Clients on ENET interfaces associate with an upstream
     Client on the ENET the same as a Client would associate with an ANET
     Proxy/Server.  Specifically, the Host/Client sends an RS message via
     the ENET which directs the message to the upstream Client.  The
     upstream Client then responds to the RS message by returning an RA.
     In this way, the downstream nodes see the ENET as an ANET and see the
     upstream Client as a Proxy/Server for that ANET.

     AERO Hosts, Clients and Proxy/Servers use IPv6 ND messages to
     maintain adaptation layer NCEs.  AERO Proxy/Servers configure their
     OMNI interfaces as advertising NBMA interfaces, and therefore send
     unicast RA messages with a short Router Lifetime value (e.g.,
     ReachableTime seconds) in response to a Client's RS message.
     Thereafter, Clients send additional RS messages to keep Proxy/Server
     state alive.

     AERO Clients and FHS/MAP Proxy/Servers include SNP ULA/GUA address
     delegation (and optionally also MNP prefix delegation) DHCPv6
     parameters in RS/RA messages.  The IPv6 ND messages are exchanged
     between the Client and any FHS Proxy/Servers acting as proxys for the
     MAP Proxy/Server as specified in [I-D.templin-intarea-omni2]
     according to the address/prefix management schedule required by the
     service.  If the Client knows its MNP in advance, it can include the
     MNP in its DHCPv6 prefix delegation request.  If the MAP Proxy/Server
     accepts the Client's MNP assertion (or if it delegates a new MNP for
     the Client), it injects the MNP into the routing system and
     establishes the necessary neighbor cache state.

     All Host, Client and Proxy/Server behaviors for the exchange of RS/RA
     messages are conducted according to the Router Discovery and Prefix
     Registration specifications found in Section 15 of
     [I-D.templin-intarea-omni2].  The following sections observe all of
     the OMNI specifications, and include additional specifications of the
     interactions of Client-Proxy/Server RS/RA exchanges with the AERO
     mobility service.

  4.12.2.  AERO Host and Client Behavior

     AERO Hosts and Clients discover the addresses of candidate FHS Proxy/
     Servers as specified in Section 15 of [I-D.templin-intarea-omni2].
     The Host/Client then performs RS/RA exchanges over each of its
     underlay interfaces to associate with (possibly multiple) FHS Proxy/
     Serves and a single MAP Proxy/Server if necessary.  The Host/Client
     sends each RS (either directly via Direct interfaces, via an IPsec
     tunnel for VPN interfaces, via an access router for (M)ANET
     interfaces or via INET encapsulation for INET interfaces) and waits
     up to RetransTimer milliseconds for an RA message reply (see
     Section 4.12.3) while retrying up to MAX_RTR_SOLICITATIONS if



  Templin                  Expires 6 October 2024                [Page 56]


  Internet-Draft                    AERO                        April 2024


     necessary.  If the Host/Client receives no RAs, or if it receives an
     RA with Router Lifetime set to 0, the Client SHOULD abandon attempts
     through the first candidate Proxy/Server and try another Proxy/
     Server.

     After the Host/Client registers its underlay interfaces, it may wish
     to change one or more registrations, e.g., if an interface changes
     address or becomes unavailable, if traffic selectors change, etc.  To
     do so, the Host/Client prepares an RS message to send over any
     available underlay interface as above.  The RS includes an OMNI
     option with prefix registration/delegation information and with an
     Interface Attributes sub-option specific to the selected underlay
     interface.  When the Host/Client receives the MAP Proxy/Server's RA
     response, it has assurance that both the MAP and FHS Proxy/Servers
     have been updated with the new information.

     If the Host/Client wishes to discontinue use of a MAP Proxy/Server it
     issues an RS message over any underlay interface with an OMNI Proxy/
     Server Departure sub-option that encodes the (old) MAP Proxy/Server's
     SNP SRA GUA.  When the MAP Proxy/Server processes the message, it
     releases any MNPs, sets the NCE state for the Host/Client to DEPARTED
     and returns an RA reply with Router Lifetime set to 0.  After a short
     delay (e.g., 2 seconds), the MAP Proxy/Server withdraws the MNP from
     the routing system.  (Alternatively, when the Host/Client associates
     with a new FHS/MAP Proxy/Server it can include an OMNI "Proxy/Server
     Departure" sub-option in RS messages with the SNA SRA GUAs of the Old
     FHS/MAP Proxy/Servers.)

  4.12.3.  AERO Proxy/Server Behavior

     AERO Proxy/Servers act as both IP routers and IPv6 ND proxys, and
     support address and prefix delegation services for Clients.  When a
     FHS/MAP Proxy/Server receives a prospective Client's secured RS
     message, it SHOULD return an immediate RA reply with Router Lifetime
     set to 0 if it is currently too busy or otherwise unable to service
     the Client; otherwise, it processes the RS and performs DHCPv6
     address delegation for SNP ULA/GUA pairs while returning the ULA/GUA
     prefixes per [RFC8028] as specified in Section 15 of
     [I-D.templin-intarea-omni2].  If the RS message also contains DHCPv6
     prefix delegation parameters the FHS Proxy/Server processes the
     prefix delegations locally as a MAP or forwards a proxyed version of
     the RS to another candidate MAP Proxy/Server.









  Templin                  Expires 6 October 2024                [Page 57]


  Internet-Draft                    AERO                        April 2024


     When the MAP Proxy/Server processes the RS, it determines the correct
     MNPs for the Client by processing OMNI DHCPv6 sub-option(s).  When
     the MAP Proxy/Server returns the MNPs, it also creates forwarding
     table entries for the MNP resulting in BGP updates (see:
     Section 4.2.3).  The MAP Proxy/Server then returns an RA to the
     Client via the FHS Proxy/server as specified in Section 15 of
     [I-D.templin-intarea-omni2].

     After the initial RS/RA exchange, the MAP Proxy/Server maintains a
     ReachableTime timer for each of the Client's underlay interfaces
     individually (and for the Client's NCE collectively) set to expire
     after ReachableTime seconds.  If the Client (or an FHS Proxy/Server)
     issues additional RS messages, the MAP Proxy/Server sends an RA
     response and resets ReachableTime.  If the MAP Proxy/Server receives
     an IPv6 ND message with a prefix release indication it sets the
     Client's NCE to the DEPARTED state and withdraws the MNP route from
     the routing system after a short delay (e.g., 2 seconds).  If
     ReachableTime expires before a new RS is received on an individual
     underlay interface, the MAP Proxy/Server marks the interface as DOWN.
     If ReachableTime expires before any new RS is received on any
     individual underlay interface, the MAP Proxy/Server sets the NCE
     state to STALE and sets a 10 second timer.  If the MAP Proxy/Server
     has not received a new RS or uNA(MM) message with a prefix release
     indication before the 10 second timer expires, it deletes the NCE and
     withdraws the MNP from the routing system.

     The MAP Proxy/Server processes any IPv6 ND messages pertaining to the
     Client while forwarding to the Client or responding on the Client's
     behalf as necessary.  The MAP Proxy/Server may also issue unsolicited
     RA messages, e.g., with reconfigure parameters to cause the Client to
     renegotiate its prefix delegation/registrations, with Router Lifetime
     set to 0 if it can no longer service this Client, etc.  The MAP
     Proxy/Server may also receive carrier packets via the secured
     spanning tree that contain initial data sent while route optimization
     is in progress.  The MAP Proxy/Server reassembles the enclosed OAL
     packets/fragments, then re-encapsulates/re-fragments and sends the
     carrier packets to the target Client via an FHS Proxy/Server if
     necessary.  Finally, If the NCE is in the DEPARTED state, the old MAP
     Proxy/Server forwards any OAL packets/fragments it receives from the
     secured spanning tree and destined to the Client to the new MAP
     Proxy/Server, then deletes the entry after DepartTime expires.

     Note: Clients SHOULD arrange to notify former MAP Proxy/Servers of
     their departures, but MAP Proxy/Servers are responsible for expiring
     neighbor cache entries and withdrawing MNP routes even if no
     departure notification is received (e.g., if the Client leaves the
     network unexpectedly).  MAP Proxy/Servers SHOULD therefore set Router
     Lifetime to ReachableTime seconds in solicited RA messages to



  Templin                  Expires 6 October 2024                [Page 58]


  Internet-Draft                    AERO                        April 2024


     minimize persistent stale cache information in the absence of Client
     departure notifications.  A short Router Lifetime also ensures that
     proactive RS/RA messaging between Clients and FHS Proxy/Servers will
     keep any NAT state alive (see above).

     Note: All Proxy/Servers on an OMNI link MUST advertise consistent
     values in the RA Cur Hop Limit, M and O flags, Reachable Time and
     Retrans Timer fields the same as for any link, since unpredictable
     behavior could result if different Proxy/Servers on the same link
     advertised different values.

  4.12.3.1.  Additional Proxy/Server Considerations

     AERO Clients register with FHS Proxy/Servers for each underlay
     interface.  Each of the Client's FHS Proxy/Servers in turn inform a
     single MAP Proxy/Server of the Client's underlay interface(s) that it
     services.  For Clients on Direct and VPN/IPsec underlay interfaces,
     the FHS Proxy/Server for each interface is directly connected, for
     Clients on (M)ANET underlay interfaces the FHS Proxy/Server is
     located on the (M)ANET/INET boundary, and for Clients on INET
     underlay interfaces the FHS Proxy/Server is located somewhere in the
     connected Internetwork.  When FHS Proxy/Server "B" processes a Client
     registration, it must either assume the MAP role or forward a proxyed
     registration to another Proxy/Server "A" acting as the MAP.  Proxy/
     Servers satisfy these requirements as follows:

     *  when FHS Proxy/Server "B" receives a Client RS message, it first
        verifies that the OAL Identification is within the window for the
        AFV associated with the NCE for this Client neighbor and
        authenticates the message.  If no NCE was found, Proxy/Server "B"
        instead creates one in the STALE state and caches the Client-
        supplied Interface Attributes, Origin Indication and Multilink
        Vector sub-option parameters as well as the Client's observed L2
        address (noting that it may differ from the Origin address if
        there were NATs on the path).  Proxy/Server "B" then examines the
        RS destination address.  If the destination address is the SNP SRA
        GUA of a different Proxy/Server "A", Proxy/Server "B" prepares a
        separate proxyed version of the RS message with an OAL header with
        source set to its own SNP SRA GUA and destination set to Proxy/
        Server A's SNP SRA GUA.  Proxy/Server "B" also writes its own L2
        address information over the Interface Attributes sub-option L2
        information supplied by the Client, omits or zeros the Origin
        Indication sub-option, omits the Multilink Vector sub-option then
        forwards the message into the OMNI link secured spanning tree.

     *  when MAP Proxy/Server "A" receives the RS, it assumes the MAP
        role, delegates MNPs for the Client if necessary, and creates/
        updates a NCE indexed by the Client's MNP SRA GUA(s) with FHS



  Templin                  Expires 6 October 2024                [Page 59]


  Internet-Draft                    AERO                        April 2024


        Proxy/Server "B"'s Interface Attributes as the link layer address
        information for this FHS ifIndex.  MAP Proxy/Server "A" then
        prepares an RA message with source set to its own SNP SRA GUA,
        destination set to the Client's SNP GUA, and with OMNI option
        DHCPv6 sub-options with the prefix delegation results.  MAP Proxy/
        Server "A" then encapsulates the RA in an OAL header with source
        set to its own SNP SRA GUA and destination set to the SNP SRA GUA
        of FHS Proxy/Server "B", then finally performs fragmentation if
        necessary and sends the resulting carrier packets into the secured
        spanning tree.

     *  when FHS Proxy/Server "B" reassembles the RA, it locates the
        Client NCE based on the RA destination.  If the RA message
        includes an OMNI "Proxy/Server Departure" sub-option with non zero
        old FHS/MAP Proxy/Server SNP GUAs that do not match its own GUA,
        FHS Proxy/Server "B" first sends a uNA(MM) to the old FHS/MAP
        Proxy/Servers named in the sub-option.  Proxy/Server "B" then re-
        inserts the cached Multilink Vector sub-option for this Client
        while updating the window synchronization parameters.  If the RA
        message delegates a new SNP ULA/GUA pair, Proxy/Server "B" then
        resets the RA destination to the corresponding Client SNP ULA for
        this interface.

     *  Proxy/Server "B" then re-encapsulates the message with OAL source
        set to its own ULA and OAL destination set to the address that
        appeared in the Client's RS message OAL source, with an
        appropriate Identification value, with an authentication signature
        if necessary, with the Client's Interface Attributes sub-option
        echoed and with the cached observed L2 addresses written into an
        Origin Indication sub-option.  Proxy/Server "B" sets the P flag in
        the RA flags field to indicate that the message has passed through
        a proxy [RFC4389] then fragments the RA if necessary and returns
        the fragments to the Client.

     *  The Client repeats this process over each of its additional
        underlay interfaces while treating each additional FHS Proxy/
        Server "C", "D", "E", etc. as a proxy to facilitate RS/RA
        exchanges between MAP "A" and the Client.  The Client creates/
        updates NCEs for each such FHS Proxy/Server as well as the MAP
        Proxy/Server in the process.











  Templin                  Expires 6 October 2024                [Page 60]


  Internet-Draft                    AERO                        April 2024


     After the initial RS/RA exchanges each FHS Proxy/Server forwards any
     of the Client's carrier packets that contain OAL packets/fragments
     with destinations for which there is no matching NCE to a Gateway
     using OAL encapsulation with its own SNP SRA GUA as the source and
     with destination determined by the Client.  The Proxy/Server instead
     forwards any OAL packets/fragments destined to a neighbor cache
     target directly to the target according to the OAL or link layer
     information - the process of establishing neighbor cache entries is
     specified in Section 4.13.

     While the Client is still associated with FHS Proxy/Servers "B", "C",
     "D", "E", etc., each FHS Proxy/Server can send NS, RS and/or uNA
     messages to update the NCEs of other AERO nodes on behalf of the
     Client based on changes in Interface Attributes, Traffic Selectors,
     Multilink Vectors, etc.  This allows for higher-frequency Proxy-
     initiated RS/RA messaging over well-connected INET infrastructure
     supplemented by lower-frequency Client-initiated RS/RA messaging over
     constrained (M)ANET data links.

     If the MAP Proxy/Server "A" ceases to send solicited RAs, FHS Proxy/
     Servers "B", "C", "D", "E", etc. can send unsolicited RAs over the
     Client's underlay interface with destination set to (link-local) All-
     Nodes multicast and with Router Lifetime set to zero to inform
     Clients that the MAP Proxy/Server has failed.  Although Proxy/Servers
     "B", "C", "D", "E", etc. can engage in IPv6 ND exchanges on behalf of
     the Client, the Client can also send IPv6 ND messages on its own
     behalf, e.g., if it is in a better position to convey state changes.
     The IPv6 ND messages sent by the Client include the Client's MNP SRA
     GUA as the source in order to differentiate them from the IPv6 ND
     messages sent by a FHS Proxy/Server.

     If the Client becomes unreachable over all underlay interfaces it
     serves, the MAP Proxy/Server sets the NCE state to DEPARTED and
     retains the entry for DepartTime seconds.  While the state is
     DEPARTED, the MAP Proxy/Server forwards any OAL packets/fragments
     destined to the Client to a new MAP Proxy/Server if known; otherwise,
     it discards the OAL packets/fragments.  When DepartTime expires, the
     MAP Proxy/Server deletes the NCE, withdraws the MNP route and
     discards any further carrier packets that contain OAL packets/
     fragments destined to the former Client.

     In (M)ANETs that employ a Proxy/Server, the Client can inject its
     MNPs into the (M)ANET routing system.  For IPv6, the MNPs are simply
     IPv6 GUA prefixes per the IPv6 addressing architecture; for IPv4, the
     MNPs are 6to4 address prefixes per [RFC3056].  In both cases, the
     Client can forward original IP packets/parcels into the (M)ANET
     without invoking OAL encapsulation such that the (M)ANET routing
     system transports them to the Proxy/Server or a peer (M)ANET Client.



  Templin                  Expires 6 October 2024                [Page 61]


  Internet-Draft                    AERO                        April 2024


     This encapsulation avoidance can be beneficial, e.g., if the Client
     connects to the (M)ANET via low-end data links such as some aviation
     wireless links.  (Note: for original IPv4 packets/parcels, the Client
     injects them as IPv6 packets/parcels with 6to4 addresses.)

     If the (M)ANET first-hop access router is on the same underlay link
     as the Client and recognizes the AERO/OMNI protocol, the Client can
     avoid OAL encapsulation for both its control and data messages.  When
     the Client connects to the link, it can send an unencapsulated RS
     message with source address set to its own MNP SRA GUA or SNP ULA,
     and with destination address set to the SNP SRA ULA of the Client's
     selected Proxy/Server or to link-scoped All-Routers multicast.  The
     Client includes an OMNI option formatted as specified in
     [I-D.templin-intarea-omni2].  The Client then sends the
     unencapsulated RS message, which will be intercepted by the AERO-
     aware ANET access router.

     The (M)ANET access router then performs OAL encapsulation on the RS
     message and forwards it to a Proxy/Server at the (M)ANET/INET
     boundary.  When the access router and Proxy/Server are one and the
     same node, the Proxy/Server would share an underlay link with the
     Client but its message exchanges with outside correspondents would
     need to pass through a security gateway at the (M)ANET/INET border.
     The method for deploying access routers and Proxys (i.e. as a single
     node or multiple nodes) is a (M)ANET-local administrative
     consideration.

     Note: When a Proxy/Server alters the IPv6 ND message contents before
     forwarding (e.g., such as altering the OMNI option contents), the
     original IPv6 ND message checksum and authentication signature values
     are invalidated and must be re-calculated.

     Note: When a Proxy/Server receives a secured Client NS message, it
     performs the same proxying procedures as for described for RS
     messages above.  The proxying procedures for NS/NA message exchanges
     is specified in Section 4.13.

  4.12.3.2.  Detecting and Responding to Proxy/Server Failures

     In environments where fast recovery from Proxy/Server failure is
     required, FHS Proxy/Servers SHOULD use proactive Neighbor
     Unreachability Detection (NUD) to track MAP Proxy/Server reachability
     in a fashion that parallels Bidirectional Forwarding Detection (BFD)
     [RFC5880].  Each FHS Proxy/Server can then quickly detect and react
     to failures so that cached information is re-established through
     alternate paths.  The NS/NA control messaging is carried only over
     well-connected ground domain networks (i.e., and not low-end
     aeronautical radio links) and can therefore be tuned for rapid



  Templin                  Expires 6 October 2024                [Page 62]


  Internet-Draft                    AERO                        April 2024


     response.

     FHS Proxy/Servers can perform continuous NS/NA exchanges with the MAP
     Proxy/Server, e.g., one exchange per N seconds.  The FHS Proxy/Server
     sends the NS message via the spanning tree with its own SNP SRA GUA
     as the source and the SNP SRA GUA of the MAP Proxy/Server as the
     destination, and the MAP Proxy/Server responds with an NA.  When the
     FHS Proxy/Server also sends RS messages to a MAP Proxy/Server on
     behalf of Clients, the resulting RA responses can be considered as
     equivalent hints of forward progress.  This means that the FHS Proxy/
     Server need not also send a periodic NS if it has already sent an RS
     within the same period.  If the MAP Proxy/Server fails (i.e., if the
     FHS Proxy/Server ceases to receive advertisements), the FHS Proxy/
     Server can quickly inform Clients by sending unsolicited RA messages

     The FHS Proxy/Server sends unsolicited RA messages with source
     address set to the MAP Proxy/Server's address, destination address
     set to (link-local) All-Nodes multicast, and Router Lifetime set to
     0.  The FHS Proxy/Server SHOULD send MAX_FINAL_RTR_ADVERTISEMENTS RA
     messages separated by small delays [RFC4861].  Any Clients that had
     been using the failed MAP Proxy/Server will receive the RA messages
     and select a different Proxy/Server to assume the MAP role (i.e., by
     sending an RS with destination set to the SNP SRA GUA of the new
     MAP).

  4.13.  AERO Address Resolution, Multilink Forwarding and Route
         Optimization

     AERO nodes invoke address resolution, multilink forwarding and route
     optimization when they need to forward initial original IP packets/
     parcels to new neighbors over (M)ANET/INET interfaces and for ongoing
     multilink forwarding coordination with existing neighbors.

     Possible source and destination addresses for original IP packets
     that traverse a local (M)MANET/INET and/or the rest of the OMNI link
     include addresses taken from an FNP or MNP, or the SNP ULA/GUA
     assigned to a Client.  (No other IP address types should appear on
     the OMNI link.)

     Address resolution is based on an IPv6 ND NS/NA(AR) messaging
     exchange between an Address Resolution Source (ARS) as the NS(AR)
     source and the target neighbor as the Address Resolution Target
     (ART).  Either the ART itself or the ART's current FHS/MAP Proxy/
     Server (or Relay) serves as the Address Resolution Responder (ARR),
     i.e., the NA(AR) source.






  Templin                  Expires 6 October 2024                [Page 63]


  Internet-Draft                    AERO                        April 2024


     If the original IP packet uses an FNP/MNP address as the source, the
     NS(AR) source and NA(AR) destination use the corresponding FNP/MNP
     Subnet Router Anycast (SRA) address as the source.  If the original
     IP packet uses an SNP GUA as the source, the NS(AR) source and NA(AR)
     destination use the SNP GUA as the source.  The original IP packet
     destination address appears in the Target Address of each NS/NA(AR).

     Address resolution is initiated by the first eligible ARS closest to
     the original source as follows:

     *  For Clients on VPN/IPsec and Direct interfaces, the Client's FHS
        Proxy/Server is the ARS.

     *  For Clients on (M)ANET interfaces, either the FHS Proxy/Server or
        the Client itself may be the ARS.

     *  For Clients on INET interfaces, the Client itself is the ARS.

     *  For FNP correspondent nodes on INET/ENET interfaces serviced by a
        Relay, the Relay is the ARS.

     *  For Clients that engage the MAP Proxy/Server in "mobility anchor"
        mode, the MAP Proxy/Server is the ARS.

     *  For peers within the same (M)ANET/ENET, address resolution and
        route optimization is through receipt of Redirect messages.

     The AERO routing system directs an address resolution request sent by
     the ARS to the ARR.  The ARR then returns an address resolution reply
     which must include information that is complete, current, consistent
     and authentic.  Both the ARS and ARR are then jointly responsible for
     periodically refreshing the address resolution, and for quickly
     informing each other of any changes.  Following address resolution,
     the ARS and ART perform subsequent multilink forwarding and route
     optimization exchanges to maintain optimal forwarding profiles.
















  Templin                  Expires 6 October 2024                [Page 64]


  Internet-Draft                    AERO                        April 2024


     During address resolution, multilink forwarding and/or route
     optimization an NS/NA message source may attach a small number of
     original IP packets/parcels associated with the message exchange as
     super-packet extensions per [I-D.templin-intarea-omni2].  The
     authentication signatures and/or lower-layer security features
     employed at the OAL source and each OAL intermediate system will
     provide authorization and integrity services for both the NS/NA
     messages and their IP packet/parcel attachments.  When an OAL source
     or intermediate system forwards a secured NS/NA super-packet, it
     should perform OAL encapsulation followed by fragmentation using a
     fragment size no larger than 1280 octets to ensure that the fragments
     will traverse any possible secured spanning tree paths.  The final
     OAL intermediate system in the path will then securely forward the
     NS/NA message IP packet/parcel attachments to the OAL target.

     The address resolution, multilink forwarding and route optimization
     procedures are specified in the following sections.

  4.13.1.  Multilink Address Resolution

     When one or more original IP packets/parcels from a source node
     destined to a target node arrive, the ARS checks for a NCE with an
     FNP/MNP SRA prefix or SNP GUA that matches the target destination.
     If there is a NCE in the REACHABLE state, the ARS invokes the OAL and
     forwards the resulting carrier packets according to the cached state
     then returns from processing.

     Otherwise, if there is no NCE the ARS creates one in the INCOMPLETE
     state.  The ARS then prepares an NS message for Address Resolution
     (NS(AR)) to send toward an ART while attaching the original IP
     packet(s)/parcel(s) to the end of the NS(AR) as an OAL super-packet
     (see above).  The resulting NS(AR) message must be sent securely, and
     includes source, destination and target addresses as discussed above.
     If the source address is an MNP SRA address, the NS(AR) message also
     includes Route Information Options (RIOs) [RFC4191] for any of the
     source Client's MNPs.

     The ARS then includes an OMNI option with an authentication sub-
     option (if necessary), Interface Attributes and/or Traffic Selectors
     for all of the source Client's underlay interfaces.  The ARS then
     calculates and includes an authentication signature (if necessary)
     followed by the checksum, then submits the NS(AR) message for OAL
     encapsulation.

     The ARS sets the OAL source to its own SNP ULA/GUA and sets the OAL
     destination according to the Client's RS message "RPT" flag (see:
     [I-D.templin-intarea-omni2]).  If the "RPT" flag was set, the ARS
     sets the OAL destination to the SNP SRA GUA of its MAP Proxy/Server



  Templin                  Expires 6 October 2024                [Page 65]


  Internet-Draft                    AERO                        April 2024


     which maintains a Report List; otherwise, the ARS sets the OAL
     destination to the FNP/MNP SRA address or SNP GUA corresponding to
     the ART.  The ARS then includes an appropriate Identification value,
     performs OAL fragmentation and L2 encapsulation/fragmentation, then
     sends the resulting carrier packets into the SRT secured spanning
     tree without decrementing the network layer TTL/Hop Limit field.

     When the ARS is a Client, it must instead use the SNP SRA ULA of the
     interface-specific FHS Proxy/Server as the OAL destination.  The ARS
     Client then performs OAL fragmentation followed by L2 encapsulation/
     fragmentation then forwards the carrier packets to the FHS Proxy/
     Server.  The FHS Proxy/Server then performs L2 reassembly/
     decapsulation, verifies the Identification, performs OAL reassembly,
     verifies the NS(AR) checksum/authentication signature and confirms
     that the Client's claimed MNP RIOs and SNP ULA source address are
     correct.  The FHS Proxy/Server then changes the OAL source to its own
     SNP SRA GUA and changes the OAL destination to the SNP SRA GUA of the
     MAP Proxy/Server or FNP/MNP SRA address or SNP GUA corresponding to
     the ART as specified above.  The FHS Proxy/Server next includes an
     appropriate Identification, performs OAL fragmentation, performs L2
     encapsulation/fragmentation and sends the resulting carrier packets
     into the secured spanning tree on behalf of the Client.

     Note: both the source and target Client/Relay and their MAP Proxy/
     Servers include current and accurate information for their multilink
     Interface Attributes profile.  The MAP Proxy/Servers can be trusted
     to provide an authoritative ARR response and/or mobility update
     message on behalf of the source/target should the need arise.  While
     the source or target itself has no such trust basis, any attempt to
     mount an attack by providing false Interface Attributes information
     would only result in black-holing of return traffic, i.e., the
     "attack" could only result in denial of service to the source/target
     itself.  The source/target's asserted Interface Attributes therefore
     do not need to be validated by the MAP Proxy/Server.

  4.13.1.1.  ARS MAP Proxy/Server NS(AR) Processing

     If the ARS Client's MAP Proxy/Server maintains a Report List, the
     carrier packets containing the NS(AR) will first arrive at the MAP
     due to the OAL destination address supplied by the ARS (see above).
     This source MAP then performs L2 reassembly/decapsulation, verifies
     the Identification, performs OAL reassembly and records the NS(AR)
     Target Address in the Report List for this source Client.  The MAP
     then leaves the OAL source address unchanged, but changes the OAL
     destination address to the FNP/MNP SRA GUA or SNP GUA corresponding
     to the ART.  The MAP then decrements the OAL header Hop Limit,
     includes an appropriate Identification, performs OAL fragmentation
     followed by L2 encapsulation/fragmentation and sends the resulting



  Templin                  Expires 6 October 2024                [Page 66]


  Internet-Draft                    AERO                        April 2024


     carrier packets into the secured spanning tree.

  4.13.1.2.  Relaying the NS(AR)

     When a Gateway receives carrier packets containing the NS(AR), it
     performs L2 reassembly/decapsulation and determines the next hop by
     consulting its standard IPv6 forwarding table for the OAL header
     destination address.  The Gateway next decrements the OAL header Hop
     Limit, performs L2 encapsulation/fragmentation and sends the carrier
     packet(s) via the secured spanning tree the same as for any IPv6
     router where they may traverse multiple intermediate OMNI link
     segments interconnected by Gateways.  The final Gateway will deliver
     the carrier packets via the secured spanning tree to the LHS/MAP
     Proxy/Server (or Relay) that services the ART.

  4.13.1.3.  NS(AR) Processing at the ARR/ART

     When the LHS/MAP Proxy/Server (or Relay) of the ART receives the
     NS(AR) secured carrier packets with the FNP/MNP SRA GUA or SNP GUA of
     the ART as the OAL destination, it performs L2 reassembly/
     decapsulation followed by OAL reassembly then either forwards the
     NS(AR) to the ART or processes it locally if it is acting as the
     ART's designated ARR.  The LHS/MAP Proxy/Server (or Relay) processes
     the message as follows:

     *  if the NS(AR) target matches a Client NCE in the DEPARTED state,
        the (old) MAP Proxy/Server resets the OAL destination address to
        the SNP SRA address of the Client's new MAP Proxy/Server.  The old
        MAP Proxy/Server then decrements the OAL header Hop Limit,
        performs OAL fragmentation followed by L2 encapsulation/
        fragmentation and forwards the resulting carrier packets over the
        secured spanning tree.

     *  If the NS(AR) target matches a Client NCE in the REACHABLE state,
        the LHS/MAP Proxy/Server (or Relay) notes whether the NS(AR)
        arrived from the secured spanning tree.  If the message arrived
        via the secured spanning tree the LHS/MAP Proxy/Server (or Relay)
        verifies the NS(AR) checksum only; otherwise, it must also verify
        the message authentication signature.

     *  If the LHS/MAP Proxy/Server maintains a Report List for the ART,
        it next records the NS(AR) source address in the Report List for
        this ART.  If the MAP Proxy/Server is the ART's designated ARR, it
        forwards any original IP packet(s)/parcel(s) attached to the
        NS(AR) super-packet to the ART and prepares to return an NA(AR) as
        discussed below; otherwise, the LHS/MAP Proxy/Server determines
        the underlay interface for the ART and proceeds as follows:




  Templin                  Expires 6 October 2024                [Page 67]


  Internet-Draft                    AERO                        April 2024


        -  If the LHS/MAP Proxy/Server is also the ART's FHS Proxy/Server
           on the underlay interface used to convey the NS(AR) to the ART,
           it includes an authentication signature if necessary then
           recalculates the NS(AR) checksum.  The Proxy/Server then
           changes the OAL source to its own SNP SRA ULA and OAL
           destination to the ULA of the ART, decrements the OAL Hop
           Limit, includes an appropriate Identification value, performs
           OAL fragmentation followed by L2 encapsulation/fragmentation
           and forwards the resulting carrier packets over the underlay
           interface to the ART.

        -  If the MAP Proxy/Server is not the LHS Proxy/Server on the
           underlay interface used to convey the NS(AR) to the ART, it
           instead recalculates the NS(AR) checksum, changes the OAL
           source to its own SNP SRA GUA and changes the OAL destination
           to the SNP SRA GUA of the LHS Proxy/Server for this ART
           interface.  The MAP Proxy/Server next decrements the OAL Hop
           Limit, includes an appropriate Identification value, performs
           OAL fragmentation followed by L2 encapsulation/fragmentation
           and forwards the resulting carrier packets over the secured
           spanning tree.

        -  When the LHS Proxy/Server receives the carrier packets, it
           performs L2 reassembly/decapsulation, reassembles the NS(AR)
           and verifies the checksum, then forwards to the ART while
           changing the OAL addresses to ULAs the same as described above.

     *  If the NS(AR) target matches one of its FNP routes, the MAP Proxy/
        Server serves as both a Relay and an ARR, since the Relay forwards
        original IP packets/parcels toward foreign target nodes at the
        network layer.

     If the ARR is a Relay or the ART itself, it first creates or updates
     a NCE for the NS(AR) source address while caching all RIOs, Interface
     Attributes and Traffic Selector information.  Next, the ARR prepares
     a solicited NA(AR) message to return to the ARS with the IPv6 source,
     destination and target addresses set as described above.

     The ARR then includes RIOs for all of the ART's MNPs plus Interface
     Attributes and Traffic Selector sub-options for all of the ART's
     underlay interfaces with current information for each interface.  The
     ARR next sets the NA(AR) message R flag to 1 (as a router) and S flag
     to 1 (as a response to a solicitation) and sets the O flag to 1 (as
     an authoritative responder).

     The ARR finally includes an authentication signature if necessary,
     calculates the NA(AR) message checksum, then submits the NA(AR) for
     encapsulation with OAL source set to its own ULA and destination set



  Templin                  Expires 6 October 2024                [Page 68]


  Internet-Draft                    AERO                        April 2024


     to the ULA that appeared in the NS(AR) OAL source while including an
     appropriate Identification.  The ARR then performs OAL fragmentation
     followed by L2 encapsulation/fragmentation, and forwards the
     resulting carrier packets.

     When the ART's FHS Proxy/Server receives carrier packets sent by an
     ART acting as an ARR on its own behalf, it performs L2 reassembly and
     decapsulation, verifies the Identification, performs OAL reassembly,
     then verifies the checksum/authentication signature.  The Proxy/
     Server then verifies that the RIO information is acceptable, changes
     the OAL source address to its own SNP SRA GUA and changes the OAL
     destination to the FNP/MNP SRA GUA or SNP GUA corresponding to the
     NA(AR) destination.  The Proxy/Server next decrements the OAL Hop
     Limit, includes an appropriate Identification then recalculates the
     NA(AR) checksum.  The Proxy/Server finally performs OAL fragmentation
     followed by L2 encapsulation/fragmentation and forwards the resulting
     carrier packets into the secured spanning tree.

  4.13.1.4.  Relaying the NA(AR)

     When a Gateway receives NA(AR) carrier packets, it performs L2
     reassembly/decapsulation and determines the next hop by consulting
     its standard IPv6 forwarding table for the OAL header destination
     address.  The Gateway then decrements the OAL header Hop Limit,
     performs L2 encapsulation/fragmentation and forwards the resulting
     carrier packets via the SRT secured spanning tree where they may
     traverse multiple intermediate OMNI link segments interconnected by
     other Gateways.  The final-hop Gateway will deliver the carrier
     packets via the secured spanning tree to a Proxy/Server for the ARS.

  4.13.1.5.  Processing the NA(AR) at the ARS

     When the ARS receives NA(AR) carrier packets, it performs L2
     reassembly/decapsulation, verifies the Identification, performs OAL
     reassembly, then searches for a NCE that matches the NA(AR) target
     address.  The ARS then processes the message the same as for standard
     IPv6 Address Resolution [RFC4861].  In the process, it caches all RIO
     and OMNI option information in the NCE for the ART (including
     Interface Attributes, Traffic Selectors, etc.), and caches the NA(AR)
     source address plus any RIO/MNP SRA GUAs as ART addresses.

     When the ARS is a Client, the SRT secured spanning tree will first
     deliver the solicited NA(AR) message to the Client's FHS Proxy/
     Server, which re-adjusts the OAL header and forwards the message to
     the Client.  If the Client is on a well-managed ANET, physical
     security and protected spectrum ensures security for the NA(AR)
     without needing an additional authentication signature; if the Client
     is in a MANET or on the open INET the Proxy/Server must instead



  Templin                  Expires 6 October 2024                [Page 69]


  Internet-Draft                    AERO                        April 2024


     include an authentication signature (while adjusting the OMNI option
     size, if necessary).  The Proxy/Server uses its own SNP SRA ULA as
     the OAL source and the SNP ULA of the Client as the OAL destination
     when it forwards the NA(AR).  The Proxy/Server then decrements the
     OAL Hop Limit, includes an appropriate Identification, performs OAL
     fragmentation followed by L2 encapsulation/fragmentation and forwards
     the resulting carrier packets over the underlay interface to the
     Client.

  4.13.1.6.  Reliability

     After the ARS transmits the first NS(AR), it should wait up to
     RETRANS_TIMER seconds to receive a responsive NA(AR).  The ARS can
     then retransmit the NS(AR) up to MAX_UNICAST_SOLICIT times before
     giving up.

  4.13.2.  Multilink Forwarding

     Following address resolution, the ARS and ART (i.e., the end system
     Clients or their respective Proxy/Servers) can assert multilink
     forwarding paths through underlay interface pairs serviced by the
     same source/destination ULAs by sending NS/NA messages for Multilink
     Forwarding (MF) with OMNI Multilink Vector sub-options.  The NS/
     NA(MF) messages establish multilink forwarding and header compression
     state in OAL intermediate systems in the path between the ARS and
     ART.  Note that either the ARS or ART can independently initiate
     multilink forwarding by sending NS(MF) messages on behalf of specific
     underlay interface pairs.  (Underlay interface FHS/LHS directionality
     must also be factored into the paths established for multilink
     forwarding.)

     If the original IP packet uses an FNP/MNP GUA as the source, the
     source Client or Proxy/Server uses the corresponding FNP/MNP Subnet
     Router Anycast (SRA) GUA as the NS(MF) source.  If the original IP
     packet uses an SNP GUA as the source, the source Client or Proxy/
     Server uses the SNP GUA as the NS(MF) source.  In both cases, the
     source Client or Proxy/Server uses the original IP packet destination
     address for both the Target Address and destination address of the
     NS(MF).  When the target Client or Proxy/Server returns an NA(MF), it
     sets the destination to the same address that appeared in the NS(MF)
     source, sets the Target Address to the same address that appeared in
     the NS(MF) target and sets the source address to the SNP SRA GUA in
     the Proxy/Server case or to the SNP GUA in the Client case.

     When an OAL source asserts a multilink forwarding path through the
     transmission of an NS(MF) message, it includes an IPv6 Minimum Path
     MTU Hop-by-Hop Option for the (adaptation layer) IPv6 header per
     [RFC9268].  Each OAL intermediate node and OAL IPv6 router along the



  Templin                  Expires 6 October 2024                [Page 70]


  Internet-Draft                    AERO                        April 2024


     path then updates the minimum MTU per the specification.  When the
     OAL destination responds with an NA(MF) message, it returns an IPv6
     Minimum Path MTU Option based on the one it received in the NS(MF)
     message per [RFC9268].  This allows the OAL source to discover any
     OAL Fragment Size (OFS) limitations for this OAL destination (see:
     [I-D.templin-intarea-omni2]).  The OAL source then returns a uNA(MF)
     with IPv6 Minimum Path MTU option to the OAL destination allowing the
     destination to discover the OAL path MTU along the reverse path.  For
     this reason, OAL nodes that connect SRT segments MUST implement
     [RFC9268].

     The multilink forwarding profile provides support for redundant paths
     that each OAL node can harness to its best advantage.  For example,
     OAL nodes can use traffic selectors to guide the dispersal of
     different traffic types over available multilink paths, while other
     factors such as metrics, cost, provider, etc. can also provide useful
     decision points.  OAL nodes can also employ multilink forwarding for
     fault tolerance by sending redundant data over multiple paths
     simultaneously, or for load balancing where the individual packets of
     a single traffic flow are spread across multiple independent paths.
     OAL nodes that engage in multilink forwarding therefore must
     incorporate a policy engine that selects both inbound and outbound
     multilink paths for a given traffic profile at a given point in time.
     This specification therefore provides multilink forwarding mechanisms
     without mandating any specific multilink policy.

     All Client, Proxy/Server and Gateway nodes that configure OMNI
     interfaces and engage in multilink coordination include an additional
     forwarding table termed the AERO Forwarding Information Base (AFIB)
     that supports OAL packet/fragment forwarding based on OMNI neighbor
     underlay interface pairs.  The AFIB contains per-interface-pair AERO
     Forwarding Vectors (AFVs) identified by the L2 address of the
     previous OAL hop plus a value known as the AFV Index (AFVI).  The
     AFVs cache uncompressed OAL header information to support forwarding
     of packets with compressed headers as well as previous/next-hop
     addressing and AFVI information.  The AFVs also cache window
     synchronization state (i.e., the starting sequence number and window
     size) for the specific underlay interface pair.  Using the window
     synchronization state, simple Identification-based data origin
     authentication is enabled at each OAL source, intermediate system and
     target node.

     Client and Proxy/Server OMNI interfaces manage end system AFIB
     entries in conjunction with their internal Neighbor Cache, where the
     NCEs link to (possibly) multiple AFVs with one per FHS/LHS interface
     ifIndex pair.  When OMNI interface peers need to coordinate, they
     locate a NCE for the peer (established through address resolution)
     then use the NCE as a nexus that aggregates potentially many AVFs



  Templin                  Expires 6 October 2024                [Page 71]


  Internet-Draft                    AERO                        April 2024


     which cache AFVIs to be used to for multilink forwarding.  Gateway
     OMNI interfaces and the OMNI interfaces of Clients or Proxy/Servers
     acting as OAL intermediate nodes manage transit AFIB entries
     independently of their internal Neighbor Caches.  These transit AFVs
     are indexed by the L2 address and AFVI supplied by the previous hop.

     OAL source, intermediate system and target nodes create AFVs/AFVIs
     when they process an NS/NA(MF) initiator message or a responsive
     NA(MF) message with OMNI Multilink Vector sub-options (see:
     [I-D.templin-intarea-omni2]).  The OAL source of the initiating NS/
     NA(MF) (which is also the OAL destination of the responsive NA(MF))
     is considered to reside in the "First Hop Segment (FHS)", while the
     OAL destination of the NS/NA(MF) (which is also the OAL source of the
     responsive NA(MF)) is considered to reside in the "Last Hop Segment
     (LHS)".

     The FHS and LHS roles are determined on a per-interface-pair basis.
     After address resolution, either peer is equally capable of
     initiating multilink forwarding on behalf of a specific FHS/LHS
     underlay interface pair.  The peer that sends the initiating NS/
     NA(MF) message with Multilink Vector for a specific pair becomes the
     FHS peer while the one that returns the responsive NA(MF) becomes the
     LHS peer for that pair only.  It is therefore commonplace that peers
     may assume the FHS role for some pairs while assuming the LHS role
     for other pairs, i.e., even though each peer maintains only a single
     NCE.

     When an OAL node sends/forwards an initiating NS/NA(MF) or responsive
     NA(MF) with a Multilink Vector sub-option, it creates an AFV, caches
     the NS/NA(MF) OAL IPv6 header, records the previous hop L2 address
     and AFVI, then generates a new next hop AFVI.  The next hop AFVI
     should be selected within the range [0 - (2**16-1)] unless all values
     within that range are already in active use for this next hop;
     otherwise, the AFVI must be selected within the range [2**16 -
     (2**32-1)].  When the OAL node forwards future OAL packets/fragments
     that include the previous hop L2 address and AFVI, it can
     unambiguously locate the correct AFV and use the cached forwarding
     information to forward to the next OAL hop.

     OAL nodes cache AFVs for up to ReachableTime seconds following their
     initial creation.  If the node processes another NS/NA(MF) message
     specific to an AFV, it resets ReachableTime to REACHABLE_TIME
     seconds, i.e., the same as for NCEs.  If ReachableTime expires, the
     node deletes the AFV.

     The following sections provide the detailed specifications of these
     NS/NA(MF) exchanges for all nodes along the forward and reverse
     paths.



  Templin                  Expires 6 October 2024                [Page 72]


  Internet-Draft                    AERO                        April 2024


  4.13.2.1.  FHS Client-Proxy/Server NS(MF) Forwarding

     When an FHS OAL source has an original IP packet/parcel to send
     toward an LHS OAL target, it first performs address resolution
     resulting in the creation of a NCE for the FNP/MNP SRA GUA and/or SNP
     GUA(s) of the target then selects a source and target underlay
     interface pair.  The FHS source then uses its cached information for
     the target interface as LHS information then prepares an NS(MF)
     message with a Multilink Vector sub-option then sets the NS(MF)
     source, target and destination addresses as specified above.

     The FHS source next creates an AFV then generates and assigns an AFVI
     that is unique for its communications to the next OAL hop.  The FHS
     source then includes the AFVI and window synchronization parameters
     in the Multilink Vector sub-option and prepares the NS(MF) message
     for transmission while also caching the window synchronization in the
     local AFV.

     If the FHS source is the FHS Proxy/Server, it performs OAL
     encapsulation while setting the OAL source to its own SNP SRA GUA and
     setting the OAL destination to the SNP SRA GUA of one of its FHS
     Gateways.  The FHS Proxy/Server then includes an appropriate
     Identification value, performs OAL fragmentation followed by L2
     encapsulation/fragmentation then forwards the resulting carrier
     packets into the secured spanning tree which will deliver them to an
     FHS Gateway.

     If the FHS source is the FHS Client, it instead includes an
     authentication signature if necessary.  The FHS Client then
     calculates the NS(MF) message checksum, performs OAL encapsulation,
     sets the OAL source to its own SNP ULA and sets the OAL destination
     to the SNP SRA ULA of the FHS Proxy/Server.  The FHS Client finally
     includes an appropriate Identification value for the FHS Proxy/
     Server, performs OAL fragmentation followed by L2 encapsulation/
     fragmentation and forwards the resulting carrier packets to the FHS
     Proxy/Server.

     When the FHS Proxy/Server receives the carrier packets, it performs
     L2 reassembly/decapsulation, verifies the Identification, performs
     OAL reassembly if necessary and verifies the NS(MF) checksum or
     authentication signature.  The FHS Proxy/Server then creates an AFV
     (i.e., the same as the FHS Client had done) while caching the FHS
     Client addressing, AFVI and window synchronization information as
     previous hop information for this AFV.  The FHS Proxy/Server next
     generates a new unique AFVI to forward to the next OAL hop, then both
     caches the AFVI in the AFV and writes it into the Multilink Vector
     sub-option, i.e., while over-writing the value supplied by the FHS
     Client.  The FHS Proxy/Server then calculates the NS(MF) checksum and



  Templin                  Expires 6 October 2024                [Page 73]


  Internet-Draft                    AERO                        April 2024


     sets the OAL source address to its own SNP SRA GUA and destination
     address to the SNP GUA of an FHS Gateway.  The FHS Proxy/Server
     finally decrements the OAL Hop Limit and includes an Identification
     appropriate for the secured spanning tree.  The FHS Proxy/Server
     finally performs OAL fragmentation followed by L2 encapsulation/
     fragmentation and forwards the resulting carrier packets into the
     secured spanning tree.

  4.13.2.2.  FHS/intermediate/LHS Gateway NS(MF) Forwarding

     Gateways in the spanning tree forward OAL packets/fragments not
     explicitly addressed to themselves, while forwarding those that
     arrived via the secured spanning tree to the next hop also via the
     secured spanning tree and forwarding all others via the unsecured
     spanning tree.  When an FHS Gateway receives an OAL packet/fragment
     over the secured spanning tree addressed to its SNP GUA or the FHS
     SNP SRA GUA, it instead performs L2 reassembly/decapsulation,
     verifies the Identification, then finally performs OAL reassembly to
     obtain the NS(MF) then verifies the checksum.  The FHS Gateway next
     creates an AFV and caches the previous hop Multilink Vector
     information, i.e. the same as the FHS Proxy/Server had done.  The FHS
     Gateway then generates a locally-unique AFVI for next hop and both
     caches the value in the AFV and copies it into the Multilink Vector
     sub-option.

     The FHS Gateway then examines the SRT prefixes corresponding to both
     the FHS and LHS.  If the FHS Gateway has a local interface connection
     to both the FHS and LHS (whether they are the same or different
     segments), the FHS/LHS Gateway caches the NS(MF) Multilink Vector
     information in the AFV, and writes a new locally-unique AFVI for the
     next hop into the AFV and Multilink Vector.  The FHS gateway then
     sets its LHS SNP SRA GUA as the OAL source and the SNP SRA GUA of the
     LHS Proxy/Server as the OAL destination.  If the FHS and LHS prefixes
     are different, the FHS Gateway instead sets the LHS SNP SRA GUA of
     the next OAL Gateway hop found in the IPv6 routing table as the OAL
     destination.  The FHS Gateway then decrements the OAL Hop Limit,
     includes an appropriate Identification, recalculates the NS(MF)
     checksum, performs OAL fragmentation followed by L2 encapsulation/
     fragmentation and forwards the resulting carrier packets into the
     secured spanning tree.

     When the FHS and LHS Gateways are different, the LHS Gateway will
     receive carrier packets over the secured spanning tree from the FHS
     Gateway, noting there may be many intermediate Gateways in the path
     between FHS and LHS which will update their transit AFVs in the same
     fashion while selecting new locally-unique AFVIs for the next hop and
     caching and updating Multilink Vector information.  The LHS Gateway
     then performs L2 reassembly/decapsulation, verifies the



  Templin                  Expires 6 October 2024                [Page 74]


  Internet-Draft                    AERO                        April 2024


     Identification, performs OAL reassembly to obtain the NS(MF),
     verifies the checksum then creates an AFV (i.e., the same as all
     previous hop Gateways had done) while caching the Multilink Vector
     information from the previous hop and creating a new AFVI for the
     next hop.  The LHS Gateway then sets its own SNP SRA GUA as the OAL
     source and the SNP SRA GUA of the LHS Proxy/Server as the OAL
     destination, decrements the OAL Hop Limit, includes an appropriate
     Identification, recalculates the NS(MF) checksum, performs OAL
     fragmentation followed by L2 encapsulation/fragmentation and forwards
     the resulting carrier packets into the secured spanning tree.

  4.13.2.3.  LHS Proxy/Server-Client NS/NA(MF) Receipt/Forwarding

     When the LHS Proxy/Server receives the carrier packets from the
     secured spanning tree, it performs L2 reassembly/decapsulation,
     verifies the Identification, performs OAL reassembly, verifies the
     NS(MF) checksum then creates an AFV and caches the previous hop
     Multilink Vector and addressing information.

     If the NS(MF) destination matches an FNP/MNP SRA GUA or SNP GUA
     associated with the target and the LHS Proxy/Server is configured to
     respond on the target's behalf, it next prepares to return a
     responsive NA(MF).  The LHS Proxy/Server next creates or updates an
     NCE for the NS(MF) source address (if necessary) with state set to
     STALE.

     The LHS Proxy/Server then creates an NA(MF) while copying the NS(MF)
     Multilink Vector sub-option into the NA(MF) and including responsive
     window synchronization information.  The LHS Proxy/Server then
     generates a locally-unique AFVI and both assigns it to the AFV and
     writes it into the Multilink Vector.  The LHS Proxy/Server then
     encapsulates the NA(MF) with OAL source set to its own SNP SRA GUA
     and OAL destination set to the SNP SRA GUA of the (previously
     visited) LHS Gateway.  The LHS Proxy/Server then includes an
     appropriate Identification value, calculates the NA(MF) checksum,
     performs OAL fragmentation followed by L2 encapsulation/fragmentation
     and forwards the resulting carrier packets into the secured spanning
     tree.













  Templin                  Expires 6 October 2024                [Page 75]


  Internet-Draft                    AERO                        April 2024


     If the LHS Client is configured to respond on its own behalf, the LHS
     Proxy/Server instead creates a locally-unique AFVI for the Client and
     both caches it in the newly-created AFV and writes it into the
     Multilink Vector.  The LHS Client then includes an authentication
     signature in the NS(MF) if necessary, then recalculates the NS(MF)
     checksum, changes the OAL source to its own SNP SRA ULA and changes
     the OAL destination to the SNP ULA of the LHS Client.  The LHS Proxy/
     Server then decrements the OAL Hop Limit, includes an appropriate
     Identification value, performs OAL fragmentation followed by L2
     encapsulation/fragmentation and forwards the resulting carrier
     packets to the LHS Client.

     When the LHS Client receives the carrier packets, it performs L2
     reassembly/decapsulation, verifies the Identification, performs OAL
     reassembly then verifies the NS(MF) checksum/authentication
     signature.  The LHS Client then creates a NCE for the NS(MF) source
     address (if necessary) in the STALE state the caches the NS(MF)
     Multilink Vector information in a new AFV associated with the NCE
     corresponding to the NS(MF) source.  The LHS Client finally generates
     and assigns a locally-unique AFVI the same as for previous hops,
     which it caches in the new AFV.

     The LHS Client then prepares an NA(MF) using the same procedures as
     for the LHS Proxy/Server above (while including responsive window
     synchronization information and the new AFVI in the Multilink Vector
     sub-option).  The LHS Client includes an authentication signature if
     necessary, calculates the NA(MF) message checksum, then encapsulates
     the NA(MF) with OAL source set to its own SNP ULA and OAL destination
     set to the SNP SRA ULA of the LHS Proxy/Server.  The LHS Client
     finally includes an appropriate Identification, performs OAL
     fragmentation followed by L2 encapsulation/fragmentation and forwards
     the resulting carrier packets to the LHS Proxy/Server.

  4.13.2.4.  Reverse Path State Establishment

     When the LHS Proxy/Server receives the carrier packets, it performs
     L2 reassembly/decapsulation, verifies the Identification, performs
     OAL reassembly, verifies the NA(MF) checksum/authentication
     signature, then creates an AFV that caches the Client's AFVI and
     other Multilink Vector information in the same manner as the FHS
     Proxy/Server had done.  The LHS Proxy/Server then forwards the NA(MF)
     over the reverse path toward the initiating FHS node, where it may
     traverse many intermediate Gateways.








  Templin                  Expires 6 October 2024                [Page 76]


  Internet-Draft                    AERO                        April 2024


     Each Gateway along the reverse path processes the Multilink Vector
     information in the NA(MF) message in the same way that the Gateways
     in the forward path had processed the NS(MF).  In the end, the NA(MF)
     will arrive at the initiating FHS node and AFV state will be
     established in all end and intermediate systems in both the forward
     and reverse paths.

     The forward and reverse paths between the FHS initiator and LHS
     responder may be different and may therefore have different OAL path
     MTUs.  The LHS responder will also require a responsive ACK to its
     SYN/ACK window synchronization directives.  For this reason, the FHS
     initiator returns a secured uNA(MF) that includes an IPv6 Minimum
     Path MTU Option based on the one it received in the NA(MF) message
     per [RFC9268].  This provides the LHS responder with both a
     responsive ACK and a measurement of the LHS->FHS OAL path MTU.

  4.13.2.5.  OAL End System Exchanges Following Synchronization

     Following the initial NS/NA/uNA(MF) exchange OAL end systems can
     begin exchanging ordinary carrier packets that include AFVIs and with
     Identification values within their respective send/receive windows
     without requiring security signatures and/or secured spanning tree
     traversal.  OAL end and intermediate systems can also consult their
     AFIBs when they receive carrier packets that contain OAL packets/
     fragments with AFVIs to unambiguously locate the correct AFV and can
     use the AFV state to forward OAL packets/fragments to the next hop
     OAL end systems must then perform continuous NS/NA(MF) exchanges to
     update window state, register new interface pairs for optimized
     multilink forwarding, confirm reachability and/or refresh AFIB cache
     state in the path before ReachableTime expires.

     While the OAL end systems continue to actively exchange OAL packets,
     they are jointly responsible for updating cache state and per-
     interface reachability before expiration.  Window synchronization
     state is performed on a per-interface-pair basis and tracked in the
     AFVs which are also linked to the appropriate NCE.  However, the
     window synchronization exchange only confirms target Client
     reachability over the specific underlay interface pair.  Reachability
     for other underlay interfaces that share the must be determined
     individually using additional NS/NA(MF) messages that include
     Multilink Vectors.










  Templin                  Expires 6 October 2024                [Page 77]


  Internet-Draft                    AERO                        April 2024


     OAL sources can the begin including CRHs in OAL packets/fragments
     with an AFVI that OAL intermediate systems can use for shortest-path
     forwarding based on AFVIs instead of spanning tree OAL IPv6
     addresses.  OAL sources and intermediate systems can instead forward
     OAL packets/fragments with OCH/OFH headers that include an AFVI since
     all OAL nodes in the path up to (and sometimes including) the OAL
     destination have already established AFVs.

     When a Proxy/Server receives OAL packets/fragments destined to a
     local SRT segment Client or forwards OAL packets/fragments received
     from a local segment Client, it first locates the correct AFV.  If
     the OAL packet/fragment includes a secured IPv6 ND message, the
     Proxy/Server uses the Client's NCE established through RS/RA
     exchanges to re-encapsulate/re-fragment while sending outbound
     secured carrier packets via the secured spanning tree and sending
     inbound secured carrier packets while including an authentication
     signature/checksum.  For ordinary OAL packets/fragments, the Proxy/
     Server uses the same AFV if directed by AFVI and/or OAL addressing.
     Otherwise it locates an AFV established through an NS/NA(MF) exchange
     between the Client and the remote SRT segment peer, and forwards the
     OAL packet/fragments without first reassembling/decapsulating.

     When a source Client forwards OAL packets/fragments it can employ
     header compression according to the AFVs established through an NS/
     NA(MF) exchange with a remote or local peer.  When a target Client
     receives carrier packets that contain OAL packets/fragments that
     match a local AFV, the Client first verifies the Identification then
     decompresses the headers if necessary, reassembles to obtain the OAL
     packet then decapsulates and delivers the original IP packet/parcel
     to the network layer.

     When synchronized peer Clients in the same SRT segment with FMT-
     Forward and FMT-Mode set discover each other's NATed L2ADDR
     addresses, they can exchange carrier packets that contain OAL
     packets/fragments directly with header compression using AFVIs
     discovered as above (see: Section 4.13.4).

     When the FHS Client or FHS Proxy/Server sends an NS(MF) for the
     purpose of establishing multilink forwarding state, it should wait up
     to RETRANS_TIMER seconds to receive a responsive NA(MF).  The FHS
     node can then retransmit the NS(MF) up to MAX_UNICAST_SOLICIT times
     before giving up.  Note that each successive attempt establishes new
     AFV state in the OAL intermediate systems, but that any abandoned
     stale AFV state will be quickly reclaimed.







  Templin                  Expires 6 October 2024                [Page 78]


  Internet-Draft                    AERO                        April 2024


  4.13.2.6.  Rapid Commit Multilink Forwarding

     Multilink forwarding can often be invoked in conjunction with Address
     Resolution in order to reduce control message overhead and round-trip
     delays.  When an ART acting as an ARR receives an NS(AR) with a set
     of Interface Attributes for the ARS source Client, it can perform
     "rapid commit" by immediately invoking multilink forwarding as above
     at the same time as returning the NA(AR).

     In order to perform rapid commit, the ARR includes a Multilink Vector
     sub-option in the NA(AR) as though it were initiating a multilink
     coordination NS/NA(MF) exchange as specified above.  The ARR then
     includes any Interface Attributes and/or Traffic Selector sub-options
     as necessary to satisfy the address resolution request.  The ARR then
     returns the NA(AR) to the ARS using the same hop-by-hop OAL
     addressing disciplines as specified above for an ordinary multilink
     NS/NA(MF) exchange.  This will cause the NA(AR) to visit all FHS/LHS
     OAL intermediate systems on the path towards the ARS.

     When the NA(AR) traverses the return path to the ARS, OAL
     intermediate systems in the path process the Multilink Vector
     information the same as for an ordinary multilink forwarding exchange
     as specified above, i.e., without examining the remaining NA(AR)
     message contents.  This results in the ARR node now assuming the FHS
     role and the ARS assuming the LHS role from the perspective of
     multilink forwarding coordination.  When the NA(AR) arrives, the ARS
     processes the Multilink Vector and window synchronization parameters
     while also processing all other NA(AR) OMNI option information,
     thereby eliminating an extraneous message transmission and associated
     delay.  The ARS (now acting as an LHS peer) then returns a responsive
     NA(MF); if no NA(MF) response is received within RETRANS_TIMER
     seconds, the ARR can retransmit the NA(AR) up to
     MAX_NEIGHBOR_ADVERTISEMENT times before giving up.

  4.13.3.  Mobile Ad-hoc Network (MANET) Forwarding

     Clients with OMNI interfaces configured over underlay interfaces with
     indeterminant neighborhood properties may be connected to ANETs
     coordinated as Mobile Ad-hoc NETworks (MANETs).  Each MANET may be
     either completely outside of the range of any OMNI link Proxy/Servers
     or may require multihop traversal between Clients acting as MANET
     routers to reach Proxy/Servers that connect to the rest of the OMNI
     link.  The former class of MANETs must operate in isolation solely
     based on the HHIT unique IPv6 addresses they configure locally.  The
     latter class allows MANET routers to extend infrastructure-based
     addressing information including MNPs over multiple OMNI link hops as
     discussed in the OMNI specification.




  Templin                  Expires 6 October 2024                [Page 79]


  Internet-Draft                    AERO                        April 2024


     MANET Clients configure their OMNI interfaces over one or more MANET
     interfaces where multihop forwarding may be necessary.  Routing
     protocols suitable for use over MANET interfaces include OSPFv3
     [RFC5340] with MANET Designated Router (OSPF-MDR) extensions
     [RFC5614], OLSR [RFC7181], AODV [I-D.perkins-manet-aodvv2] and
     others.  Other services specific to MANET link-local and/or site-
     local operations (including SMF [RFC6621], DLEP [RFC8175] and others)
     are also considered in-scope.  These services strive for optimal use
     of available radio bandwidth and power consumption in their control
     message transmissions, but efficient data plane operation is also
     essential.

     Clients must therefore reduce overhead through minimal encapsulation
     and effective header compression whenever possible.  For this reason,
     when the MANET routing protocol discovers a new route the Client
     configures a lesser-preferred forwarding table entry over the
     corresponding MANET interface and a more-preferred forwarding table
     entry over the OMNI interface.  This will cause the network layer to
     direct outbound packets to the OMNI interface, which can apply header
     compression and underlay MANET interface selection.

     When two Clients within the same MANET communicate using IPv6
     addresses that are advertised in the MANET routing protocol, their
     OMNI interfaces can avoid OAL encapsulation and treat the IPv6 header
     supplied by the network layer as if it were an OAL encapsulation
     header.  This includes the application of OAL fragmentation and
     header compression as discussed in the OMNI specification.  When a
     MANET node needs to forward IPv4 packets, it converts the IPv4
     headers into IPv6 headers that use 6to4 addresses that encapsulate
     the IPv4 source and destination addresses [RFC3056].

     Proxy/Servers that connect a MANET to the rest of the OMNI link act
     as regular Proxy/Servers for exchanges with external INETs, but act
     as Clients over their MANET interfaces.  Each such Proxy/Server
     therefore has at least two underlay interfaces, including an INET
     interface and a MANET interface.  The Proxy/Server therefore services
     the MANET as if it were an ordinary Client but presents itself as a
     Proxy/Server to external facing INETs.

     The process for a multihop Client to establish multilink forwarding
     and header compression AFV state in the MANET is conducted in the
     same fashion as described above and using the same NS/NA(MF) message
     exchanges.  Each intermediate MANET node in the path configures AFV
     state and assigns AFVIs in the same fashion as for intermediate
     Gateways in the secured spanning tree except that the NS/NA(MF)
     messages requires an authentication signature since neighboring MANET
     nodes do not configure IPsec tunnels.  The NS/NA(MF) messages extend
     from the initiating FHS MANET Client, then across any intermediate



  Templin                  Expires 6 October 2024                [Page 80]


  Internet-Draft                    AERO                        April 2024


     FHS MANET Clients to the FHS Proxy/Server, then across the secured
     SRT spanning tree, then to the LHS Proxy/Server, then finally across
     any intermediate LHS MANET Clients to the responding LHS Client.  The
     LHS responder then returns an NA(MF) along the reverse path which
     establishes reverse path multilink forwarding and header compression
     state all the way to the FHS initiator.  In all other ways, the NS/
     NA(MF) exchanges are the same as for non-MANET networks.

     Following the NS/NA(MF) exchanges, each MANET router in the path in
     both the FHS and LHS MANETs will have established AFVs containing
     multilink forwarding and header compression state.  The AFVs
     determine AFVI-based forwarding based on the OCH header contents, and
     each MANET router only forwards packet with in-window Identification
     values.  MANET routers maintain AFVs for up to ReachableTime seconds
     unless they are refreshed by either a new NS/NA(MF) exchange or the
     transmission of any data packet with a full OAL header with an in-
     window Identification value and a CRH extension.  New window
     synchronization exchanges must also be performed periodically to
     avoid window exhaustion and/or spoofing based on predictable
     Identifications.

     Note: while the MANET routing protocol runs directly over the node's
     MANET interfaces to discover routing information, the node configures
     lesser-preferred forwarding table entries over the MANET interface
     and corresponding more-preferred forwarding table entries over the
     OMNI interface.  This causes the network layer to forward outbound
     packets via the OMNI interface which applies encapsulation,
     fragmentation and/or header compression as necessary before
     forwarding over the underlying MANET interface.  The OMNI protocol
     designator in the UDP port, IP protocol or Ethernet EtherType field
     will then cause the packets to visit the OMNI interface of each
     successive next-hop MANET node.

  4.13.4.  Client/Client Route Optimization

     When the FHS/LHS Clients are both located on the same SRT segment,
     Client-to-Client route optimization is possible following the
     establishment of any necessary state in NATs in the path.  Both
     Clients will have already established state via their respective
     shared segment Proxy/Servers (and possibly also the shared segment
     Gateway) and can begin sending carrier packets directly via NAT
     traversal while avoiding any Proxy/Server and/or Gateway hops.

     When the FHS/LHS Clients on the same SRT segment perform the initial
     NS/NA(MF) exchange to establish AFIB state, they first examine the
     FMT-Forward and FMT-Mode settings to determine whether direct-path
     forwarding is even possible for one or both Clients (direct-path
     forwarding is only possible for one or both when FMT-Forward and FMT-



  Templin                  Expires 6 October 2024                [Page 81]


  Internet-Draft                    AERO                        April 2024


     Mode are both 1).  The NS/NA(MF) messages then include an Origin
     Indication (i.e., in addition to a Multilink Vector sub-option) with
     the mapped addresses discovered during the RS/RA exchanges with their
     respective Proxy/Servers.  After the AFV paths have been established,
     both Clients can begin sending carrier packets via strict AFV paths
     while establishing a direct path for Client-to-Client route
     optimization.

     To establish the direct path, either Client (acting as the source)
     transmits a bubble to the mapped L2 address for the target Client
     which primes its local chain of NATs for reception of future carrier
     packets from that L2 address (see: [RFC4380] and
     [I-D.templin-intarea-omni2]).  The source Client then prepares an
     NS(RO) message with its own MNP SRA GUA or SNP GUA as the source,
     with the MNP SRA GUA or SNP GUA of the target as the destination and
     with an OMNI option with an Interface Attributes sub-option.  The
     source Client then encapsulates the NS(RO) in an OAL header with its
     own SNP ULA as the source, with the SNP ULA of the target Client as
     the destination and with an in-window Identification for the target.
     The source Client then performs OAL fragmentation followed by L2
     encapsulation/fragmentation with L2 headers addressed to its Proxy/
     Server then sends the resulting carrier packets to the Proxy/Server.

     When the Proxy/Server receives the carrier packets, it re-
     encapsulates and sends them as unsecured carrier packets according to
     AFIB state where they will eventually arrive at the target Client
     which can perform L2 reassembly/decapsulation, verify the
     Identification and perform OAL reassembly.  Following reassembly, the
     target Client prepares an NA(RO) message with its own MNP SRA GUA or
     SNP GUA as the source, with the MNP SRA GUA or SNP GUA of the source
     Client as the destination and with an OMNI option with an Interface
     Attributes sub-option.  The target Client then encapsulates the
     NA(RO) in an OAL header with its own SNP ULA as the source, with the
     SNP ULA of the source Client as the destination and with an in-window
     Identification for the source Client.  The target Client then
     performs OAL fragmentation followed by L2 encapsulation/fragmentation
     then forwards the resulting carrier packets directly to the source
     Client.

     Following the initial NS/NA(RO) exchange, both Clients mark their
     respective (source, target) underlay interface pairs as "trusted" for
     no more than ReachableTime seconds.  The Clients can then begin
     exchanging ordinary data packets as OCH encapsulated carrier packets.
     While the Clients continue to exchange packets via the direct path
     avoiding all Proxy/Servers and Gateways, they should perform
     additional NS/NA(RO) exchanges via their local Proxy/Servers to
     refresh NCE state as well as send additional bubbles to the peer's
     Origin address information if necessary to refresh NAT state.



  Templin                  Expires 6 October 2024                [Page 82]


  Internet-Draft                    AERO                        April 2024


     Note: these procedures are suitable for a widely-deployed but basic
     class of NATs.  Procedures for advanced NAT classes are outlined in
     [RFC6081], which provides mechanisms that can be employed equally for
     AERO using the corresponding sub-options specified by OMNI.

     Note: each communicating pair of Clients may need to maintain NAT
     state for peer to peer communications via multiple underlay interface
     pairs.  It is therefore important that Origin Indications are
     maintained with the correct peer interface and that the NCE may cache
     information for multiple peer interfaces.

     Note: the source and target Client exchange Origin information during
     the secured NS/NA(RO) multilink route optimization exchange.  This
     allows for subsequent NS/NA(RO) exchanges to proceed using only the
     Identification value as a data origin confirmation.  However, Client-
     to-Client peerings that require stronger security may also include
     authentication signatures for mutual authentication.

  4.13.5.  Intra-ANET/ENET Route Optimization for AERO Peers

     When a Client forwards an OAL packet (or an original IP packet/
     parcel) from a Host or another Client connected to one of its
     downstream ENETs to a peer within the same downstream ENET, the
     Client returns an IPv6 ND Redirect message to inform the source that
     that target can be reached directly.  The contents of the Redirect
     message are the same as specified in [RFC4861], and should also
     include any RIOs with MNP information corresponding to the target.

     In the same fashion, when a Proxy/Server forwards an OAL packet (or
     original IP packet/parcel) from a Host or Client connected to one of
     its downstream (M)ANETs/INETs to a peer within the same downstream
     (M)ANET/INET, the Proxy/Server returns an IPv6 ND Redirect message.

     All other route optimization functions are conducted per the NS/
     NA(RO) messaging discussed in the previous sections.

  4.14.  Neighbor Unreachability Detection (NUD)

     AERO nodes perform Neighbor Unreachability Detection (NUD) per
     [RFC4861] either reactively in response to persistent link layer
     errors (see Section 4.11) or proactively to confirm reachability.
     The NUD algorithm is based on periodic control message exchanges and
     may further be seeded by IPv6 ND hints of forward progress, but care
     must be taken to avoid inferring reachability based on spoofed
     information.  For example, IPv6 ND message exchanges that include
     authentication codes and/or in-window Identifications may be
     considered as acceptable hints of forward progress, while spurious
     random carrier packets should be ignored.



  Templin                  Expires 6 October 2024                [Page 83]


  Internet-Draft                    AERO                        April 2024


     AERO nodes can perform NS/NA(NUD) exchanges over the OMNI link
     secured spanning tree (i.e. the same as described above) to test
     reachability without risk of DoS attacks from nodes pretending to be
     a neighbor.  These NS/NA(NUD) messages use the unicast GUAs/ULAs of
     the parties involved in the NUD test.  When only reachability
     information is required without updating any other NCE state, AERO
     nodes can instead perform NS/NA(NUD) exchanges directly between
     neighbors without employing the secured spanning tree as long as they
     include in-window Identifications and an authentication signature/
     checksum.

     After route optimization directs a source FHS peer to a target LHS
     peer with one or more link layer addresses, either node may invoke
     multilink forwarding state initialization to establish authentic
     intermediate system state between specific underlay interface pairs
     which also tests their reachability.  Thereafter, either node acting
     as the source may perform additional reachability probing through
     NS(NUD) messages over the SRT secured or unsecured spanning tree, or
     through NS(NUD) messages sent directly to an underlay interface of
     the target itself.  While testing a target underlay interface, the
     source can optionally continue to forward OAL packets/fragments via
     alternate interfaces or maintain a small queue of carrier packets
     until target reachability is confirmed.

     NS(NUD) messages are encapsulated, fragmented and transmitted as
     carrier packets the same as for ordinary original IP data packets/
     parcels.  The source encapsulates the NS(NUD) message the same as
     described in Section 4.13.2 and includes an Interface Attributes sub-
     option with ifIndex set to identify its underlay interface used for
     forwarding.  The source then includes an in-window Identification,
     performs OAL fragmentation followed by L2 encapsulation/fragmentation
     then forwards the resulting carrier packets into the unsecured
     spanning tree, either directly to the target if it is in the local
     segment or directly to a Gateway in the local segment.

     When the target receives the NS(NUD) carrier packets, it performs L2
     reassembly/decapsulation, verifies that it has a NCE for this source
     and that the Identification is in-window then performs OAL
     reassembly.  The target next verifies the NS(NUD) checksum/
     authentication signature, then searches for Interface Attributes in
     its NCE for the source that match the NS for the NA(NUD) reply.  The
     target then prepares the NA(NUD) with the source and destination
     addresses reversed, encapsulates and sets the OAL source and
     destination, includes an Interface Attributes sub-option in the
     NA(NUD) to identify the ifIndex of the underlay interface the NS(NUD)
     arrived on and sets the Target Address to the same value included in
     the NS(NUD).  The target next sets the R flag to 1, the S flag to 1
     and the O flag to 1, then includes an in-window Identification for



  Templin                  Expires 6 October 2024                [Page 84]


  Internet-Draft                    AERO                        April 2024


     the source.  The node then performs OAL fragmentation followed by L2
     encapsulation/fragmentation and forwards the resulting carrier
     packets into the unsecured spanning tree either directly to the
     source if it is in the local segment or directly to a Gateway in the
     local segment.

     When the source receives the NA(NUD), it marks the target underlay
     interface tested as "trusted".  Note that underlay interface states
     are maintained independently of the overall NCE REACHABLE state, and
     that a single NCE may have multiple target underlay interfaces in
     various "trusted/untrusted" states while the NCE state as a whole
     remains REACHABLE.

  4.15.  Mobility Management and Quality of Service (QoS)

     AERO is a fully Distributed Mobility Management (DMM) service in
     which each Proxy/Server is responsible for only a small subset of the
     Clients on the OMNI link.  This is in contrast to a Centralized
     Mobility Management (CMM) service where there are only one or a few
     network mobility collective entities for large Client populations.
     Clients coordinate with their associated FHS and MAP Proxy/Servers
     via RS/RA exchanges to maintain the DMM profile, and the AERO routing
     system tracks all current Client/Proxy/Server peering relationships.

     MAP Proxy/Servers provide a designated router service for their
     dependent Clients, while FHS Proxy/Servers provide a proxy conduit
     between the Client and both the MAP and OMNI link in general.
     Clients are responsible for maintaining neighbor relationships with
     their Proxy/Servers through periodic RS/RA exchanges, which also
     serves to confirm neighbor reachability.  When a Client's underlay
     interface attributes change, the Client is responsible for updating
     the MAP Proxy/Server through new RS/RA exchanges using the FHS Proxy/
     Server as a first-hop conduit.  The FHS Proxy/Server can also act as
     a proxy to perform some IPv6 ND exchanges on the Client's behalf
     without consuming bandwidth on the Client underlay interface.

     Note: when a Client's underlay interface address changes, the Client
     and/or its (former) FHS Proxy/Server for this interface must
     invalidate any AFVs based on the (changed) interface.  Future data
     packet forwarding will then trigger a new multilink forwarding NS/
     NA(MF) exchange to re-seed new AFVs in the path.

     Mobility management considerations are specified in the following
     sections.







  Templin                  Expires 6 October 2024                [Page 85]


  Internet-Draft                    AERO                        April 2024


  4.15.1.  Mobility Update Messaging

     Mobile Clients (and/or their MAP Proxy/Servers) accommodate mobility
     and/or multilink change events by sending secured uNA Mobility
     Management (MM) messages to each active neighbor.  When a node sends
     a uNA(MM) message to each specific neighbor on behalf of a mobile
     Client, it sets the IPv6 source address to its own MNP SRA GUA or SNP
     SRA ULA/GUA, sets the destination address to the neighbor's SRA ULA/
     GUA and sets the Target Address to one of the mobile Client's MNP SRA
     GUAs.  The uNA(MM) also includes an OMNI option with OMNI Interface
     Attributes and Traffic Selector sub-options for the mobile Client's
     underlay interfaces and includes an authentication signature if
     necessary.  The node then sets the uNA(MM) R flag to 1, S flag to 0
     and O flag to 1, then encapsulates the message in an OAL header with
     source set to its own SNP GUA/ULA and destination set to either the
     specific neighbor's SNP GUA or the FHS Proxy/Server's SNP SRA ULA.
     Following OAL fragmentation and L2 encapsulation/fragmentation, the
     carrier packets containing the uNA(MM) message will then follow the
     secured spanning tree and arrive at the specific neighbor.

     As discussed in Section 7.2.6 of [RFC4861], the transmission and
     reception of uNA(MM) messages is unreliable but provides a useful
     optimization.  In well-connected Internetworks with robust data links
     uNA(MM) messages will be delivered with high probability, but in any
     case the node can optionally send up to MAX_NEIGHBOR_ADVERTISEMENT
     uNA(MM)s to each neighbor to increase the likelihood that at least
     one will be received.  Alternatively, the node can set the SNR flag
     in the uNA(MM) OMNI option header to request a uNA(ACK) response
     (see: Section 4.5.1).

     When the FHS/LHS Proxy/Server receives a secured uNA(MM) message
     prepared as above, if the uNA(MM) destination was its own SNP SRA ULA
     the Proxy/Server uses the included OMNI option information to update
     its NCE for the target but does not reset ReachableTime since the
     receipt of a uNA(MM) message does not provide confirmation that any
     forward paths to the target Client are working.  If the destination
     was the SNP GUA of the FHS/LHS Client, the Proxy/Server instead
     changes the OAL source to its own SNP SRA GUA/ULA, includes an
     authentication signature if necessary, and includes an in-window
     Identification for this Client.  Finally, if the uNA(MM) message SNR
     flag was set, the node that processes the uNA(MM) also returns a
     uNA(ACK) response (see: Section 4.5.1).









  Templin                  Expires 6 October 2024                [Page 86]


  Internet-Draft                    AERO                        April 2024


  4.15.2.  Announcing Link-Layer Information Changes

     When a Client needs to change its underlay Interface Attributes and/
     or Traffic Selectors for one or more underlay interfaces (e.g., due
     to a mobility event), the Client sends RS messages to its MAP Proxy/
     Server (via first-hop FHS Proxy/Servers if necessary).  Each RS
     includes an OMNI option with Interface Attributes and/or Traffic
     Selector sub-options for the ifIndex in question.

     Note that the first FHS Proxy/Server may change due to the underlay
     interface change.  If the Client RS includes an OMNI Proxy/Server
     Departure sub-option for the former FHS Proxy/Server, the new FHS
     Proxy/Server can send a departure indication (see Section 4.15.5);
     otherwise, any stale state in the former FHS Proxy/Server will simply
     expire after ReachableTime expires with no effect on the MAP Proxy/
     Server.

     Up to MAX_RTR_SOLICITATIONS RS messages MAY be sent in parallel with
     sending carrier packets containing user data in case one or more RAs
     are lost.  If all RAs are lost, the Client SHOULD re-associate with a
     new Proxy/Server.

     After performing the RS/RA exchange, the Client sends uNA(MM)
     messages to all neighbors the same as described in the previous
     section.

  4.15.3.  Bringing New Links Into Service

     When a Client needs to bring new underlay interfaces into service
     (e.g., when it activates a new data link), it sends an RS message to
     the MAP Proxy/Server via a FHS Proxy/Server for the underlay
     interface (if necessary) with an OMNI option that includes an
     Interface Attributes sub-option with interface parameters and with
     link layer address information for the new link.  The Client then
     again sends uNA(MM) messages to all neighbors the same as described
     above.

  4.15.4.  Deactivating Existing Links

     When a Client needs to deactivate an existing underlay interface, it
     sends a uNA(MM) message toward the MAP Proxy/Server via an FHS Proxy/
     Server with an OMNI option with appropriate Interface Attributes
     values for the deactivated link.








  Templin                  Expires 6 October 2024                [Page 87]


  Internet-Draft                    AERO                        April 2024


     If the Client needs to send uNA(MM) messages over an underlay
     interface other than the one being deactivated, it MUST include
     Interface Attributes for any underlay interfaces being deactivated.
     The Client then again sends uNA(MM) messages to all neighbors the
     same as described above.

     Note that when a Client deactivates an underlay interface, neighbors
     that receive the ensuing uNA(MM) messages need not purge all
     references for the underlay interface from their neighbor cache
     entries.  The Client may reactivate or reuse the underlay interface
     and/or its ifIndex at a later point in time, when it will send new RS
     messages to an FHS Proxy/Server with fresh interface parameters to
     update any neighbors.

  4.15.5.  Moving Between Proxy/Servers

     The Client performs the procedures specified in Section 4.12.2 when
     it first associates with a new MAP Proxy/Server or renews its
     association with an existing MAP Proxy/Server.

     When a Client associates with a new MAP Proxy/Server, it sends RS
     messages to register its underlay interfaces with the new MAP while
     including the old MAP's GUA in the "Old MAP Proxy/Server GUA" field
     of a Proxy/Server Departure OMNI sub-option.  When the new MAP Proxy/
     Server returns the RA message via the FHS Proxy/Server (acting as a
     proxy), the FHS Proxy/Server sends a uNA(MM) to the old MAP Proxy/
     Server (i.e., if the GUA is non-zero and different from its own).
     The uNA(MM) has the MNP SRA GUA of the Client as the source and the
     SNP SRA GUA of the old MAP as the destination and with an OMNI Proxy/
     Server Departure sub-option as above.  The FHS Proxy/Server
     encapsulates the uNA(MM) in an OAL header with the SNP SRA GUA of the
     new MAP as the source and the SNP SRA GUA of the old MAP as the
     destination, then performs OAL fragmentation followed by L2
     encapsulation/fragmentation and forwards the resulting carrier
     packets via the secured spanning tree.

     When the old MAP Proxy/Server receives the carrier packets, it
     decapsulates and reassembles if necessary to obtain the uNA(MM) then
     changes the Client's NCE state to DEPARTED, resets DepartTime and
     caches the new MAP Proxy/Server GUA.  After a short delay (e.g., 2
     seconds) the old MAP Proxy/Server withdraws the Client's MNP(s) from
     the routing system.  While in the DEPARTED state, the old MAP Proxy/
     Server forwards any carrier packets received via the secured spanning
     tree destined to the Client's MNP GUAs or SNP GUA to the new MAP
     Proxy/Server's SNP GUA.  When DepartTime expires, the old MAP Proxy/
     Server deletes the Client's NCE.





  Templin                  Expires 6 October 2024                [Page 88]


  Internet-Draft                    AERO                        April 2024


     Mobility events may also cause a Client to change to a new FHS Proxy/
     Server over a specific underlay interface at any time such that a
     Client RS/RA exchange over the underlay interface will engage the new
     FHS Proxy/Server instead of the old.  The Client can arrange to
     inform the old FHS Proxy/Server of the departure by including a
     Proxy/Server Departure sub-option with a GUA for the "Old FHS Proxy/
     Server GUA", and the new FHS Proxy/Server will issue a uNA(MM) using
     the same procedures as outlined for the MAP above while using its own
     SNP SRA GUA as the source address.  This can often result in
     successful delivery of carrier packets that would otherwise be lost
     due to the mobility event.

     Clients SHOULD NOT move rapidly between MAP Proxy/Servers in order to
     avoid causing excessive oscillations in the AERO routing system.
     Examples of when a Client might wish to change to a different MAP
     Proxy/Server include a MAP Proxy/Server that has become unresponsive,
     topological movements of significant distance, movement to a new
     geographic region, movement to a new OMNI link segment, etc.

  4.16.  Multicast

     Each Client provides an IGMP (IPv4) [RFC2236] or MLD (IPv6) [RFC3810]
     proxy service for its ENETs and/or hosted applications [RFC4605] and
     acts as a Protocol Independent Multicast - Sparse-Mode (PIM-SM, or
     simply "PIM") Designated Router (DR) [RFC7761] on the OMNI link.
     Proxy/Servers act as OMNI link PIM routers for Clients on ANET, VPN/
     IPsec or Direct interfaces, and Relays also act as OMNI link PIM
     routers on behalf of nodes on other links/networks.

     Clients on VPN/IPsec, Direct or ANET underlay interfaces for which
     the ANET has deployed native multicast services forward IGMP/MLD
     messages into the ANET.  The IGMP/MLD messages may be further
     forwarded by a first-hop ANET access router acting as an IGMP/MLD-
     snooping switch [RFC4541], then ultimately delivered to an ANET (FHS)
     Proxy/Server.  The FHS Proxy/Server then acts as an ARS to send
     NS(AR) messages to an ARR for the multicast source.  Clients on ANET/
     INET underlay interfaces without native multicast services instead
     send NS(AR) messages as an ARS to cause their FHS Proxy/Server to
     forward the message to an ARR.  When the ARR prepares an NA(AR)
     response, it initiates PIM protocol messaging according to the
     Source-Specific Multicast (SSM) and Any-Source Multicast (ASM)
     operational modes as discussed in the following sections.









  Templin                  Expires 6 October 2024                [Page 89]


  Internet-Draft                    AERO                        April 2024


  4.16.1.  Source-Specific Multicast (SSM)

     When an ARS "X" (i.e., either a Client or Proxy/Server) acting as PIM
     router receives a Join/Prune message from a node on its downstream
     interfaces containing one or more ((S)ource, (G)roup) pairs, it
     updates its Multicast Routing Information Base (MRIB) accordingly.
     For each S belonging to a prefix reachable via X's non-OMNI
     interfaces, X then forwards the (S, G) Join/Prune to any PIM routers
     on those interfaces per [RFC7761].

     For each S belonging to a prefix reachable via X's OMNI interface, X
     sends an NS(AR) message (see: Section 4.13) using its own MNP SRA GUA
     or SNP GUA as the source address and the MNP/SNP GUA of S as both the
     destination and target addresses.  X then encapsulates the NS(AR) in
     an OAL header with source address set to its own GUA/ULA and
     destination address set to the GUA/ULA for S, then forwards the
     message into the secured spanning tree which delivers it to ARR "Y"
     that services S.  Y will then return an NA(AR) that includes an OMNI
     option with Interface Attributes for any underlay interfaces that are
     currently servicing S.

     When X processes the NA(AR) it selects one or more underlay
     interfaces for S and performs an NS/NA(MF) multilink forwarding
     exchange over the secured spanning tree while including a PIM Join/
     Prune message for each multicast group of interest in the OMNI
     option.  If S is located behind any Proxys "Z"*, each Z* then updates
     its MRIB accordingly and maintains the MNP SRA GUA or SNP GUA of X as
     the next hop in the reverse path.  Since Gateways forward messages
     not addressed to themselves without examining them, this means that
     the (reverse) multicast tree path is simply from each Z* (and/or S)
     to X with no other multicast-aware routers in the path.

     Following the initial combined Join/Prune and NS/NA(MF) messaging, X
     maintains a NCE for each S the same as if X was sending unicast data
     traffic to S.  In particular, X performs additional NS/NA(MF)
     exchanges to keep the NCE alive for up to t_periodic seconds
     [RFC7761].  If no new Joins are received within t_periodic seconds, X
     allows the NCE to expire.  Finally, if X receives any additional
     Join/Prune messages for (S,G) it forwards the messages over the
     secured spanning tree.

     Client C that holds an MNP for source S may later depart from a first
     Proxy/Server Z1 and/or connect via a new Proxy/Server Z2.  In that
     case, Y sends a uNA(MM) message to X the same as specified for
     unicast mobility in Section 4.15.  When X receives the uNA(MM)
     message, it updates its NCE for the XLA for source S and sends new
     Join messages in NS/NA(MF) exchanges addressed to the new target
     Client underlay interface connection for S.  There is no requirement



  Templin                  Expires 6 October 2024                [Page 90]


  Internet-Draft                    AERO                        April 2024


     to send any Prune messages to old Proxy/Server Z1 since source S will
     no longer source any multicast data traffic via Z1.  Instead, the
     multicast state for (S,G) in Proxy/Server Z1 will soon expire since
     no new Joins will arrive.

  4.16.2.  Any-Source Multicast (ASM)

     When an ARS "X" acting as a PIM router receives Join/Prune messages
     from a node on its downstream interfaces containing one or more (*,G)
     pairs, it updates its Multicast Routing Information Base (MRIB)
     accordingly.  X first performs an NS/NA(AR) exchange to receive
     address resolution information for Rendezvous Point (RP) "R" for each
     G.  X then includes a copy of each Join/Prune message in the OMNI
     option of an NS(MF) message with its own MNP SRA GUA or SNP GUA as
     the source address and the MNP SRA GUA or SNP GUA for R as the
     destination and target address, then encapsulates the NS(MF) message
     in an OAL header with its own GUA/ULA as the source and the GUA/ULA
     of R's Proxy/Server as the destination then sends the message into
     the secured spanning tree.

     For each source "S" that sends multicast traffic to group G via R,
     Client S* that aggregates S (or its Proxy/Server) encapsulates the
     original IP packets/parcels in PIM Register messages, includes the
     PIM Register messages in the OMNI options of uNA(MM) messages,
     performs OAL encapsulation and fragmentation with Identification
     values within the receive window for Client R* that aggregates R,
     then performs L2 encapsulation/fragmentation and forwards the
     resulting carrier packets.  Client R* may then elect to send a PIM
     Join to S* in the OMNI option of a uNA(MM) over the secured spanning
     tree.  This will result in an (S,G) tree rooted at S* with R as the
     next hop so that R will begin to receive two copies of the original
     IP packet/parcel; one native copy from the (S, G) tree and a second
     copy from the pre-existing (*, G) tree that still uses uNA(MM) PIM
     Register encapsulation.  R can then issue a uNA(MM) PIM Register-stop
     message over the secured spanning tree to suppress the Register-
     encapsulated stream.  At some later time, if Client S* moves to a new
     Proxy/Server, it resumes sending original IP packets/parcels via
     uNA(MM) PIM Register encapsulation via the new Proxy/Server.

     At the same time, as multicast listeners discover individual S's for
     a given G, they can initiate an (S,G) Join for each S under the same
     procedures discussed in Section 4.16.1.  Once the (S,G) tree is
     established, the listeners can send (S, G) Prune messages to R so
     that multicast original IP packets/parcels for group G sourced by S
     will only be delivered via the (S, G) tree and not from the (*, G)
     tree rooted at R.  All mobility considerations discussed for SSM
     apply.




  Templin                  Expires 6 October 2024                [Page 91]


  Internet-Draft                    AERO                        April 2024


  4.16.3.  Bi-Directional PIM (BIDIR-PIM)

     Bi-Directional PIM (BIDIR-PIM) [RFC5015] provides an alternate
     approach to ASM that treats the Rendezvous Point (RP) as a Designated
     Forwarder (DF).  Further considerations for BIDIR-PIM are out of
     scope.

  4.17.  Operation over Multiple OMNI Links

     An AERO Client can connect to multiple OMNI links the same as for any
     data link service.  In that case, the Client maintains a distinct
     OMNI interface for each link, e.g., 'omni0' for the first link,
     'omni1' for the second, 'omni2' for the third, etc.  Each OMNI link
     would include its own distinct set of Gateways and Proxy/Servers,
     thereby providing redundancy in case of failures.

     Each OMNI link could utilize the same or different ANET/INET link
     layer connections.  The links can be distinguished at the link layer
     via the SRT prefix in a similar fashion as for Virtual Local Area
     Network (VLAN) tagging (e.g., IEEE 802.1Q) and/or through assignment
     of distinct sets of MSPs on each link.  This gives rise to the
     opportunity for supporting multiple redundant networked paths (see:
     Section 4.2.4).

     The Client's network layer can select the outbound OMNI interface
     appropriate for a given traffic profile while (in the reverse
     direction) correspondent nodes must have some way of steering their
     original IP packets/parcels destined to a target via the correct OMNI
     link.

     In a first alternative, if each OMNI link services different MSPs the
     Client can receive a distinct MNP from each of the links.  IP routing
     will therefore assure that the correct OMNI link is used for both
     outbound and inbound traffic.  This can be accomplished using
     existing technologies and approaches, and without requiring any
     special supporting code in correspondent nodes or Gateways.

     In a second alternative, if each OMNI link services the same MSP(s)
     then each link could assign a distinct "OMNI link Anycast" address
     that is configured by all Gateways on the link.  Correspondent nodes
     can then perform Segment Routing to select the correct SRT, which
     will then direct the original IP packet/parcel over multiple hops to
     the target.








  Templin                  Expires 6 October 2024                [Page 92]


  Internet-Draft                    AERO                        April 2024


  4.18.  DNS Considerations

     AERO Client MNs and INET correspondent nodes consult the Domain Name
     System (DNS) the same as for any Internetworking node.  When
     correspondent nodes and Client MNs use different IP protocol versions
     (e.g., IPv4 correspondents and IPv6 MNs), the INET DNS must maintain
     A records for IPv4 address mappings to MNs which must then be
     populated in Relay NAT64 mapping caches.  In that way, an IPv4
     correspondent node can send original IPv4 packets/parcels to the IPv4
     address mapping of the target MN, and the Relay will translate the
     IPv4 header and destination address into an IPv6 header and IPv6
     destination address of the MN.

     When an AERO Client registers with an AERO Proxy/Server, the Proxy/
     Server can return the address(es) of DNS servers in RDNSS options
     [RFC6106].  The DNS server provides the IP addresses of other MNs and
     correspondent nodes in AAAA records for IPv6 or A records for IPv4.

  4.19.  Transition/Coexistence Considerations

     OAL encapsulation ensures that dissimilar INET partitions can be
     joined into a single unified OMNI link, even though the partitions
     themselves may have differing protocol versions and/or incompatible
     addressing plans.  However, a commonality can be achieved by
     incrementally distributing globally routable (i.e., native) IP
     prefixes to eventually reach all nodes (both mobile and fixed) in all
     OMNI link segments.  This can be accomplished by incrementally
     deploying AERO Gateways on each INET partition, with each Gateway
     distributing its MNPs and/or discovering FNPs on its INET links.

     This gives rise to the opportunity to eventually distribute native IP
     addresses to all nodes, and to present a unified OMNI link view even
     if the INET partitions remain in their current protocol and
     addressing plans.  In that way, the OMNI link can serve the dual
     purpose of providing a mobility/multilink service and a transition/
     coexistence service.  Alternatively, if an INET partition is
     transitioned to a native IP protocol version and addressing scheme
     compatible with the OMNI link MNP-based addressing scheme, the
     partition and OMNI link can be joined by Gateways.

     Relays that connect INETs/ENETs with dissimilar IP protocol versions
     may need to employ a network address and protocol translation
     function such as NAT64 [RFC6146].








  Templin                  Expires 6 October 2024                [Page 93]


  Internet-Draft                    AERO                        April 2024


  4.20.  Proxy/Server-Gateway Bidirectional Forwarding Detection

     In environments where rapid failure recovery is essential, Proxy/
     Servers and Gateways SHOULD use Bidirectional Forwarding Detection
     (BFD) [RFC5880].  Nodes that use BFD can quickly detect and react to
     failures so that cached information is re-established through
     alternate nodes.  BFD control messaging is carried only over well-
     connected ground domain networks (i.e., and not low-end radio links)
     and can therefore be tuned for rapid response.

     Proxy/Servers and Gateways can maintain BFD sessions in parallel with
     their BGP peerings.  If a Proxy/Server or Gateway fails, BGP peers
     will quickly re-establish routes through alternate paths the same as
     for common BGP operational practice.

  4.21.  Time-Varying MNPs

     In some use cases, it is desirable, beneficial and efficient for the
     Client to receive a constant MNP that travels with the Client
     wherever it moves.  For example, this would allow air traffic
     controllers to easily track aircraft, etc.  In other cases, however
     (e.g., intelligent transportation systems), the MN may be willing to
     sacrifice a modicum of efficiency in order to have time-varying MNPs
     that can be changed every so often to defeat adversarial tracking.

     The DHCPv6 service offers a way for Clients that desire time-varying
     MNPs to obtain short-lived prefixes (e.g., on the order of a small
     number of minutes).  In that case, the identity of the Client would
     not be bound to the MNP but rather to a Node Identification value
     (see: [I-D.templin-intarea-omni2]) that can serve as a Client ID seed
     for MNP prefix delegation.  The Client would then be obligated to
     renumber its internal networks whenever its MNP changes.  This should
     not present problems for Clients with automated network renumbering
     services, however it can limit the durations of ongoing sessions that
     would prefer to use a constant address.

  5.  Implementation Status

     AERO/OMNI Release-3.2 was tagged on March 30, 2021, and was subject
     to internal testing.  The implementation is not planned for public
     release.

     A new implementation architecture based on a clean-slate has been
     developed and will incorporate updated aspects of the AERO/OMNI
     specs, with the goal of producing a reference implementation for
     future release.





  Templin                  Expires 6 October 2024                [Page 94]


  Internet-Draft                    AERO                        April 2024


  6.  IANA Considerations

     The IANA has assigned the UDP port number "8060" for an experimental
     first edition of AERO [RFC6706].  This document together with OMNI
     [I-D.templin-intarea-omni2] reclaims UDP port number "8060" as the
     service port for AERO/OMNI UDP/IP encapsulation.  This document makes
     no IANA request, since the OMNI specification already provides IANA
     guidance.  (Note: although [RFC6706] was not widely implemented or
     deployed, it need not be obsoleted since its messages use the invalid
     ICMPv6 message type number '0' which implementations of this
     specification can easily distinguish and ignore.)

     No further IANA actions are required.

  7.  Security Considerations

     AERO Gateways establish security associations with AERO Proxy/Servers
     and Relays within their local OMNI link segments using secured
     tunnels over underlay interfaces.  The AERO Gateways of all OMNI link
     segments in turn configure secured tunnels with neighboring AERO
     Gateways for other OMNI link segments in a secured spanning tree
     topology.  Applicable security services include IPsec [RFC4301] with
     IKEv2 [RFC7296], etc.  (Note that secured direct point-to-point links
     can also be used instead of or in addition to network layer
     security.)  Together, these services are responsible for assuring
     connectionless integrity and data origin authentication with optional
     protection against replays for control messages that traverse the
     secured spanning tree.

     To prevent unauthorized local applications from congesting the
     secured spanning tree, Proxy/Servers and Gateways configure local
     access controls to permit only the BGP protocol service daemon to
     source routing protocol control messages with the ULA assigned to the
     OMNI interface as the source over the secured spanning tree.  This
     could be implemented as a port/address filtering configuration that
     permits only TCP port 179 (as defined in the IANA "Service Names and
     Port Numbers" registry) when using the ULA assigned to the OMNI
     interface.  To prevent malicious Clients from congesting the secured
     spanning tree, Proxy/Servers should also rate-limit the secured IPv6
     ND NS/NA messages they process for the same (source, target) pair,
     e.g., by applying IPv6 ND MAX_UNICAST_SOLICIT;
     MAX_NEIGHBOR_ADVERTISEMENT limits.

     To prevent spoofing, Proxy/Servers MUST silently discard without
     responding to any unsecured IPv6 ND messages with OMNI sub-options
     that would otherwise affect state.  Also, Proxy/Servers MUST silently
     discard without forwarding any original IP packets/parcels received
     from one of their own Clients (whether directly or following OAL



  Templin                  Expires 6 October 2024                [Page 95]


  Internet-Draft                    AERO                        April 2024


     reassembly) with a source address that does not match the Client's
     MNP and/or a destination address that does match the Client's MNP.
     Finally, Proxy/Servers MUST silently discard without forwarding any
     carrier packets that include an OAL packet/fragment with source and
     destination that both match the same MNP.

     AERO Clients that connect to secured ANETs need not apply additional
     security to their IPv6 ND messages, since the messages will be
     accepted and forwarded by a perimeter Proxy/Server that applies
     security over its INET-facing interface to the secured spanning tree
     (see above).  AERO Clients that connect to the open INET can use
     network and/or transport layer security services such as VPNs (e.g.,
     IPsec tunnels) or can by some other means establish a secured direct
     link to a Proxy/Server.  When a VPN or direct link may be
     impractical, however, INET Clients and Proxy/Servers SHOULD include
     and verify authentication signatures for IPv6 ND messages as
     specified in [I-D.templin-intarea-omni2].

     End systems SHOULD apply transport or higher layer security services
     such as QUIC-TLS [RFC9000], TLS/SSL [RFC8446], DTLS [RFC6347], etc.
     to provide a level of protection comparable to critical secured
     Internet services.  End systems that require host-based VPN services
     SHOULD use network and/or transport layer security services such as
     IPsec, TLS/SSL, DTLS, etc.  AERO Proxy/Servers and Clients can also
     provide a network-based VPN service on behalf of end systems, e.g.,
     if the end system is located within a secured enclave and cannot
     establish a VPN on its own behalf.

     For INET partitions that require strong network layer security in the
     data plane, two options for securing communications include 1)
     disable route optimization and direct all traffic over the secured
     spanning tree, or 2) enable on-demand secure tunnel establishment
     between Client neighbors.  Option 1) would result in longer routes
     than necessary and impose traffic concentration on critical
     infrastructure elements.  Option 2) could be coordinated between
     Clients using NS/NA messages with OMNI Host Identity Protocol (HIP)
     "Initiator/Responder" message sub-options [RFC7401]
     [I-D.templin-intarea-omni2] or QUIC-TLS protocol message sub-options
     [RFC9000][RFC9001] [RFC9002] to establish secured sessions.

     AERO Proxy/Servers and Gateways present targets for traffic
     amplification Denial of Service (DoS) attacks.  This concern is no
     different than for widely-deployed VPN security gateways in the
     Internet, where attackers could send spoofed packets to the gateways
     at high data rates.  This can be mitigated through the AERO/OMNI data
     origin authentication procedures, as well as connecting Proxy/Servers
     and Gateways over dedicated links with no connections to the Internet
     and/or when connections to the Internet are only permitted through



  Templin                  Expires 6 October 2024                [Page 96]


  Internet-Draft                    AERO                        April 2024


     well-managed firewalls.  Traffic amplification DoS attacks can also
     target an AERO Client's low data rate links.  This is a concern not
     only for Clients located on the open Internet but also for Clients in
     secured enclaves.  AERO Proxy/Servers and Proxys can institute rate
     limits that protect Clients from receiving carrier packet floods that
     could DoS low data rate links.

     AERO Relays must implement ingress filtering to avoid a spoofing
     attack in which spurious messages with ULA addresses are injected
     into an OMNI link from an outside attacker.  AERO Clients MUST ensure
     that their connectivity is not used by unauthorized nodes on their
     ENETs to gain access to a protected network, i.e., AERO Clients that
     act as routers MUST NOT provide routing services for unauthorized
     nodes.  (This concern is no different than for ordinary hosts that
     receive an IP address delegation but then "share" the address with
     other nodes via some form of Internet connection sharing such as
     tethering.)

     The AERO service for open INET Clients depends on a public key
     distribution service in which Client public keys and identities are
     maintained in a shared database accessible to all open INET Proxy/
     Servers.  Similarly, each Client must be able to determine the public
     key of each Proxy/Server, e.g. by consulting an online database.

     The PRL contains only public information, but MUST be well-managed
     and secured from unauthorized tampering.  The PRL can be conveyed to
     the Client in a similar fashion as in [RFC5214] (e.g., through data
     link layer login messaging, secure upload of a static file, DNS
     lookups, etc.).

     Security considerations for IPv6 fragmentation and reassembly are
     discussed in [I-D.templin-intarea-omni2].  In environments where
     spoofing is considered a threat, all OAL nodes SHOULD employ
     Identification window synchronization and OAL end systems SHOULD
     configure an (end-system-based) firewall.

     Security considerations for accepting link layer ICMP messages and
     reflected carrier packets are discussed throughout the document.

  8.  Acknowledgements

     Discussions in the IETF, aviation standards communities and private
     exchanges helped shape some of the concepts in this work.
     Individuals who contributed insights include Mikael Abrahamsson, Mark
     Andrews, Fred Baker, Bob Braden, Stewart Bryant, Scott Burleigh,
     Brian Carpenter, Wojciech Dec, Pavel Drasil, Ralph Droms, Adrian
     Farrel, Nick Green, Sri Gundavelli, Brian Haberman, Bernhard Haindl,
     Joel Halpern, Tom Herbert, Bob Hinden, Sascha Hlusiak, Lee Howard,



  Templin                  Expires 6 October 2024                [Page 97]


  Internet-Draft                    AERO                        April 2024


     Christian Huitema, Zdenek Jaron, Andre Kostur, Hubert Kuenig, Eliot
     Lear, Ted Lemon, Andy Malis, Satoru Matsushima, Tomek Mrugalski,
     Thomas Narten, Madhu Niraula, Alexandru Petrescu, Behcet Saikaya,
     Michal Skorepa, Dave Thaler, Joe Touch, Bernie Volz, Ryuji Wakikawa,
     Tony Whyman, Lloyd Wood and James Woodyatt.  Members of the IESG also
     provided valuable input during their review process that greatly
     improved the document.  Special thanks go to Stewart Bryant, Joel
     Halpern and Brian Haberman for their shepherding guidance during the
     publication of the AERO first edition.

     This work has further been encouraged and supported by Boeing
     colleagues including Akash Agarwal, Kyle Bae, M.  Wayne Benson, Dave
     Bernhardt, Cam Brodie, John Bush, Balaguruna Chidambaram, Irene Chin,
     Bruce Cornish, Claudiu Danilov, Sean Dickson, Don Dillenburg, Joe
     Dudkowski, Wen Fang, Samad Farooqui, Anthony Gregory, Jeff Holland,
     Seth Jahne, Brian Jaury, Greg Kimberly, Ed King, Madhuri Madhava
     Badgandi, Laurel Matthew, Gene MacLean III, Kyle Mikos, Rob
     Muszkiewicz, Sean O'Sullivan, Satish Raghavendran, Vijay Rajagopalan,
     Kristina Ross, Greg Saccone, Ron Sackman, Bhargava Raman Sai Prakash,
     Rod Santiago, Madhanmohan Savadamuthu, Kent Shuey, Brian Skeen, Mike
     Slane, Carrie Spiker, Katie Tran, Brendan Williams, Amelia Wilson,
     Julie Wulff, Yueli Yang, Eric Yeh and other members of the Boeing
     mobility, networking and autonomy teams.  Akash Agarwal, Kyle Bae,
     Wayne Benson, Madhuri Madhava Badgandi, Vijayasarathy Rajagopalan,
     Bhargava Raman Sai Prakash, Katie Tran and Eric Yeh are especially
     acknowledged for their work on the AERO implementation.  Chuck
     Klabunde is honored for his support and guidance, and we mourn his
     untimely loss.























  Templin                  Expires 6 October 2024                [Page 98]


  Internet-Draft                    AERO                        April 2024


     This work was inspired by the support and encouragement of countless
     outstanding colleagues, managers and program directors over the span
     of many decades.  Beginning in the late 1980s,' the Digital Equipment
     Corporation (DEC) Ultrix Engineering and DECnet Architects groups
     identified early issues with fragmentation and bridging links with
     diverse MTUs.  In the early 1990s, engagements at DEC Project Sequoia
     at UC Berkeley and the DEC Western Research Lab in Palo Alto included
     investigations into large-scale networked filesystems, ATM vs
     Internet and network security proxys.  In the mid-1990s to early
     2000s employment at the NASA Ames Research Center (Sterling Software)
     and SRI International supported early investigations of IPv6, ONR UAV
     Communications and the IETF.  An employment at Nokia where important
     IETF documents were published gave way to a present-day engagement
     with The Boeing Company.  The work matured at Boeing through major
     programs including Future Combat Systems, Advanced Airplane Program,
     DTN for the International Space Station, Mobility Vision Lab, CAST,
     Caravan, Airplane Internet of Things, the NASA UAS/CNS program, the
     FAA/ICAO ATN/IPS program and many others.  An attempt to name all who
     gave support and encouragement would double the current document size
     and result in many unintentional omissions - but to all a humble
     thanks.

     Earlier works on NBMA tunneling approaches are found in
     [RFC2529][RFC5214][RFC5569].

     Many of the constructs presented in this second edition of AERO are
     based on the author's earlier works, including:

     *  Intra-Site Automatic Tunnel Addressing Protocol (ISATAP) [RFC5214]

     *  The Subnetwork Encapsulation and Adaptation Layer (SEAL) [RFC5320]

     *  Virtual Enterprise Traversal (VET) [RFC5558]

     *  Routing and Addressing in Networks with Global Enterprise
        Recursion (RANGER) [RFC5720][RFC6139]

     *  The Internet Routing Overlay Network (IRON) [RFC6179]

     *  AERO, First Edition [RFC6706]

     Note that these works cite numerous earlier efforts that are not
     included here due to space limitations.  The authors of those earlier
     works are acknowledged for their insights.

     This work is aligned with the NASA Safe Autonomous Systems Operation
     (SASO) program under NASA contract number NNA16BD84C.




  Templin                  Expires 6 October 2024                [Page 99]


  Internet-Draft                    AERO                        April 2024


     This work is aligned with the FAA as per the SE2025 contract number
     DTFAWA-15-D-00030.

     This work is aligned with the Boeing Commercial Airplanes (BCA)
     Airplane Internet of Things (AIoT) and autonomy programs.

     This work is aligned with the Boeing Information Technology (BIT)
     MobileNet program.

     Honoring life, liberty and the pursuit of happiness.

  9.  References

  9.1.  Normative References

     [I-D.templin-intarea-omni2]
                Templin, F., "Transmission of IP Packets over Overlay
                Multilink Network (OMNI) Interfaces", Work in Progress,
                Internet-Draft, draft-templin-intarea-omni2-07, 2 April
                2024, <https://datatracker.ietf.org/doc/html/draft-
                templin-intarea-omni2-07>.

     [RFC0791]  Postel, J., "Internet Protocol", STD 5, RFC 791,
                DOI 10.17487/RFC0791, September 1981,
                <https://www.rfc-editor.org/info/rfc791>.

     [RFC0792]  Postel, J., "Internet Control Message Protocol", STD 5,
                RFC 792, DOI 10.17487/RFC0792, September 1981,
                <https://www.rfc-editor.org/info/rfc792>.

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC4191]  Draves, R. and D. Thaler, "Default Router Preferences and
                More-Specific Routes", RFC 4191, DOI 10.17487/RFC4191,
                November 2005, <https://www.rfc-editor.org/info/rfc4191>.

     [RFC4193]  Hinden, R. and B. Haberman, "Unique Local IPv6 Unicast
                Addresses", RFC 4193, DOI 10.17487/RFC4193, October 2005,
                <https://www.rfc-editor.org/info/rfc4193>.

     [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
                Border Gateway Protocol 4 (BGP-4)", RFC 4271,
                DOI 10.17487/RFC4271, January 2006,
                <https://www.rfc-editor.org/info/rfc4271>.




  Templin                  Expires 6 October 2024               [Page 100]


  Internet-Draft                    AERO                        April 2024


     [RFC4291]  Hinden, R. and S. Deering, "IP Version 6 Addressing
                Architecture", RFC 4291, DOI 10.17487/RFC4291, February
                2006, <https://www.rfc-editor.org/info/rfc4291>.

     [RFC4443]  Conta, A., Deering, S., and M. Gupta, Ed., "Internet
                Control Message Protocol (ICMPv6) for the Internet
                Protocol Version 6 (IPv6) Specification", STD 89,
                RFC 4443, DOI 10.17487/RFC4443, March 2006,
                <https://www.rfc-editor.org/info/rfc4443>.

     [RFC4861]  Narten, T., Nordmark, E., Simpson, W., and H. Soliman,
                "Neighbor Discovery for IP version 6 (IPv6)", RFC 4861,
                DOI 10.17487/RFC4861, September 2007,
                <https://www.rfc-editor.org/info/rfc4861>.

     [RFC4862]  Thomson, S., Narten, T., and T. Jinmei, "IPv6 Stateless
                Address Autoconfiguration", RFC 4862,
                DOI 10.17487/RFC4862, September 2007,
                <https://www.rfc-editor.org/info/rfc4862>.

     [RFC6890]  Cotton, M., Vegoda, L., Bonica, R., Ed., and B. Haberman,
                "Special-Purpose IP Address Registries", BCP 153,
                RFC 6890, DOI 10.17487/RFC6890, April 2013,
                <https://www.rfc-editor.org/info/rfc6890>.

     [RFC8028]  Baker, F. and B. Carpenter, "First-Hop Router Selection by
                Hosts in a Multi-Prefix Network", RFC 8028,
                DOI 10.17487/RFC8028, November 2016,
                <https://www.rfc-editor.org/info/rfc8028>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/info/rfc8174>.

     [RFC8200]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
                (IPv6) Specification", STD 86, RFC 8200,
                DOI 10.17487/RFC8200, July 2017,
                <https://www.rfc-editor.org/info/rfc8200>.

     [RFC8415]  Mrugalski, T., Siodelski, M., Volz, B., Yourtchenko, A.,
                Richardson, M., Jiang, S., Lemon, T., and T. Winters,
                "Dynamic Host Configuration Protocol for IPv6 (DHCPv6)",
                RFC 8415, DOI 10.17487/RFC8415, November 2018,
                <https://www.rfc-editor.org/info/rfc8415>.

     [RFC9268]  Hinden, R. and G. Fairhurst, "IPv6 Minimum Path MTU Hop-
                by-Hop Option", RFC 9268, DOI 10.17487/RFC9268, August
                2022, <https://www.rfc-editor.org/info/rfc9268>.



  Templin                  Expires 6 October 2024               [Page 101]


  Internet-Draft                    AERO                        April 2024


  9.2.  Informative References

     [BGP]      Huston, G., "BGP in 2015, http://potaroo.net", January
                2016.

     [EUI]      "IEEE Guidelines for Use of Extended Unique Identifier
                (EUI), Organizationally Unique Identifier (OUI), and
                Company ID, https://standards.ieee.org/wp-
                content/uploads/import/documents/tutorials/eui.pdf", 3
                August 2017.

     [I-D.bctb-6man-rfc6296-bis]
                Cullen, M., Baker, F., Trøan, O., and N. Buraglio, "RFC
                6296bis IPv6-to-IPv6 Network Prefix Translation", Work in
                Progress, Internet-Draft, draft-bctb-6man-rfc6296-bis-02,
                26 January 2024, <https://datatracker.ietf.org/doc/html/
                draft-bctb-6man-rfc6296-bis-02>.

     [I-D.ietf-6man-comp-rtg-hdr]
                Bonica, R., Kamite, Y., Alston, A., Henriques, D., and L.
                Jalil, "The IPv6 Compact Routing Header (CRH)", Work in
                Progress, Internet-Draft, draft-ietf-6man-comp-rtg-hdr-04,
                18 March 2024, <https://datatracker.ietf.org/doc/html/
                draft-ietf-6man-comp-rtg-hdr-04>.

     [I-D.ietf-intarea-tunnels]
                Touch, J. D. and M. Townsley, "IP Tunnels in the Internet
                Architecture", Work in Progress, Internet-Draft, draft-
                ietf-intarea-tunnels-13, 26 March 2023,
                <https://datatracker.ietf.org/doc/html/draft-ietf-intarea-
                tunnels-13>.

     [I-D.ietf-rtgwg-atn-bgp]
                Templin, F., Saccone, G., Dawra, G., Lindem, A., and V.
                Moreno, "A Simple BGP-based Mobile Routing System for the
                Aeronautical Telecommunications Network", Work in
                Progress, Internet-Draft, draft-ietf-rtgwg-atn-bgp-26, 29
                March 2024, <https://datatracker.ietf.org/doc/html/draft-
                ietf-rtgwg-atn-bgp-26>.

     [I-D.perkins-manet-aodvv2]
                Perkins, C. E., Dowdell, J., Steenbrink, L., and V.
                Pritchard, "Ad Hoc On-demand Distance Vector Version 2
                (AODVv2) Routing", Work in Progress, Internet-Draft,
                draft-perkins-manet-aodvv2-04, 3 March 2024,
                <https://datatracker.ietf.org/doc/html/draft-perkins-
                manet-aodvv2-04>.




  Templin                  Expires 6 October 2024               [Page 102]


  Internet-Draft                    AERO                        April 2024


     [I-D.templin-6man-parcels2]
                Templin, F., "IPv6 Parcels and Advanced Jumbos (AJs)",
                Work in Progress, Internet-Draft, draft-templin-6man-
                parcels2-02, 19 February 2024,
                <https://datatracker.ietf.org/doc/html/draft-templin-6man-
                parcels2-02>.

     [I-D.templin-intarea-parcels2]
                Templin, F., "IPv4 Parcels and Advanced Jumbos (AJs)",
                Work in Progress, Internet-Draft, draft-templin-intarea-
                parcels2-02, 19 February 2024,
                <https://datatracker.ietf.org/doc/html/draft-templin-
                intarea-parcels2-02>.

     [IEN48]    Cerf, V., "The Catenet Model For Internetworking,
                https://www.rfc-editor.org/ien/ien48.txt", July 1978.

     [IEN48-2]  Cerf, V., "The Catenet Model For Internetworking (with
                figures), http://www.postel.org/ien/pdf/ien048.pdf", July
                1978.

     [RFC1035]  Mockapetris, P., "Domain names - implementation and
                specification", STD 13, RFC 1035, DOI 10.17487/RFC1035,
                November 1987, <https://www.rfc-editor.org/info/rfc1035>.

     [RFC1256]  Deering, S., Ed., "ICMP Router Discovery Messages",
                RFC 1256, DOI 10.17487/RFC1256, September 1991,
                <https://www.rfc-editor.org/info/rfc1256>.

     [RFC1812]  Baker, F., Ed., "Requirements for IP Version 4 Routers",
                RFC 1812, DOI 10.17487/RFC1812, June 1995,
                <https://www.rfc-editor.org/info/rfc1812>.

     [RFC1918]  Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G.
                J., and E. Lear, "Address Allocation for Private
                Internets", BCP 5, RFC 1918, DOI 10.17487/RFC1918,
                February 1996, <https://www.rfc-editor.org/info/rfc1918>.

     [RFC2236]  Fenner, W., "Internet Group Management Protocol, Version
                2", RFC 2236, DOI 10.17487/RFC2236, November 1997,
                <https://www.rfc-editor.org/info/rfc2236>.

     [RFC2464]  Crawford, M., "Transmission of IPv6 Packets over Ethernet
                Networks", RFC 2464, DOI 10.17487/RFC2464, December 1998,
                <https://www.rfc-editor.org/info/rfc2464>.






  Templin                  Expires 6 October 2024               [Page 103]


  Internet-Draft                    AERO                        April 2024


     [RFC2529]  Carpenter, B. and C. Jung, "Transmission of IPv6 over IPv4
                Domains without Explicit Tunnels", RFC 2529,
                DOI 10.17487/RFC2529, March 1999,
                <https://www.rfc-editor.org/info/rfc2529>.

     [RFC3056]  Carpenter, B. and K. Moore, "Connection of IPv6 Domains
                via IPv4 Clouds", RFC 3056, DOI 10.17487/RFC3056, February
                2001, <https://www.rfc-editor.org/info/rfc3056>.

     [RFC3810]  Vida, R., Ed. and L. Costa, Ed., "Multicast Listener
                Discovery Version 2 (MLDv2) for IPv6", RFC 3810,
                DOI 10.17487/RFC3810, June 2004,
                <https://www.rfc-editor.org/info/rfc3810>.

     [RFC4301]  Kent, S. and K. Seo, "Security Architecture for the
                Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
                December 2005, <https://www.rfc-editor.org/info/rfc4301>.

     [RFC4380]  Huitema, C., "Teredo: Tunneling IPv6 over UDP through
                Network Address Translations (NATs)", RFC 4380,
                DOI 10.17487/RFC4380, February 2006,
                <https://www.rfc-editor.org/info/rfc4380>.

     [RFC4389]  Thaler, D., Talwar, M., and C. Patel, "Neighbor Discovery
                Proxies (ND Proxy)", RFC 4389, DOI 10.17487/RFC4389, April
                2006, <https://www.rfc-editor.org/info/rfc4389>.

     [RFC4511]  Sermersheim, J., Ed., "Lightweight Directory Access
                Protocol (LDAP): The Protocol", RFC 4511,
                DOI 10.17487/RFC4511, June 2006,
                <https://www.rfc-editor.org/info/rfc4511>.

     [RFC4541]  Christensen, M., Kimball, K., and F. Solensky,
                "Considerations for Internet Group Management Protocol
                (IGMP) and Multicast Listener Discovery (MLD) Snooping
                Switches", RFC 4541, DOI 10.17487/RFC4541, May 2006,
                <https://www.rfc-editor.org/info/rfc4541>.

     [RFC4605]  Fenner, B., He, H., Haberman, B., and H. Sandick,
                "Internet Group Management Protocol (IGMP) / Multicast
                Listener Discovery (MLD)-Based Multicast Forwarding
                ("IGMP/MLD Proxying")", RFC 4605, DOI 10.17487/RFC4605,
                August 2006, <https://www.rfc-editor.org/info/rfc4605>.

     [RFC5015]  Handley, M., Kouvelas, I., Speakman, T., and L. Vicisano,
                "Bidirectional Protocol Independent Multicast (BIDIR-
                PIM)", RFC 5015, DOI 10.17487/RFC5015, October 2007,
                <https://www.rfc-editor.org/info/rfc5015>.



  Templin                  Expires 6 October 2024               [Page 104]


  Internet-Draft                    AERO                        April 2024


     [RFC5214]  Templin, F., Gleeson, T., and D. Thaler, "Intra-Site
                Automatic Tunnel Addressing Protocol (ISATAP)", RFC 5214,
                DOI 10.17487/RFC5214, March 2008,
                <https://www.rfc-editor.org/info/rfc5214>.

     [RFC5320]  Templin, F., Ed., "The Subnetwork Encapsulation and
                Adaptation Layer (SEAL)", RFC 5320, DOI 10.17487/RFC5320,
                February 2010, <https://www.rfc-editor.org/info/rfc5320>.

     [RFC5340]  Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF
                for IPv6", RFC 5340, DOI 10.17487/RFC5340, July 2008,
                <https://www.rfc-editor.org/info/rfc5340>.

     [RFC5522]  Eddy, W., Ivancic, W., and T. Davis, "Network Mobility
                Route Optimization Requirements for Operational Use in
                Aeronautics and Space Exploration Mobile Networks",
                RFC 5522, DOI 10.17487/RFC5522, October 2009,
                <https://www.rfc-editor.org/info/rfc5522>.

     [RFC5558]  Templin, F., Ed., "Virtual Enterprise Traversal (VET)",
                RFC 5558, DOI 10.17487/RFC5558, February 2010,
                <https://www.rfc-editor.org/info/rfc5558>.

     [RFC5569]  Despres, R., "IPv6 Rapid Deployment on IPv4
                Infrastructures (6rd)", RFC 5569, DOI 10.17487/RFC5569,
                January 2010, <https://www.rfc-editor.org/info/rfc5569>.

     [RFC5614]  Ogier, R. and P. Spagnolo, "Mobile Ad Hoc Network (MANET)
                Extension of OSPF Using Connected Dominating Set (CDS)
                Flooding", RFC 5614, DOI 10.17487/RFC5614, August 2009,
                <https://www.rfc-editor.org/info/rfc5614>.

     [RFC5720]  Templin, F., "Routing and Addressing in Networks with
                Global Enterprise Recursion (RANGER)", RFC 5720,
                DOI 10.17487/RFC5720, February 2010,
                <https://www.rfc-editor.org/info/rfc5720>.

     [RFC5880]  Katz, D. and D. Ward, "Bidirectional Forwarding Detection
                (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010,
                <https://www.rfc-editor.org/info/rfc5880>.

     [RFC6081]  Thaler, D., "Teredo Extensions", RFC 6081,
                DOI 10.17487/RFC6081, January 2011,
                <https://www.rfc-editor.org/info/rfc6081>.







  Templin                  Expires 6 October 2024               [Page 105]


  Internet-Draft                    AERO                        April 2024


     [RFC6106]  Jeong, J., Park, S., Beloeil, L., and S. Madanapalli,
                "IPv6 Router Advertisement Options for DNS Configuration",
                RFC 6106, DOI 10.17487/RFC6106, November 2010,
                <https://www.rfc-editor.org/info/rfc6106>.

     [RFC6139]  Russert, S., Ed., Fleischman, E., Ed., and F. Templin,
                Ed., "Routing and Addressing in Networks with Global
                Enterprise Recursion (RANGER) Scenarios", RFC 6139,
                DOI 10.17487/RFC6139, February 2011,
                <https://www.rfc-editor.org/info/rfc6139>.

     [RFC6145]  Li, X., Bao, C., and F. Baker, "IP/ICMP Translation
                Algorithm", RFC 6145, DOI 10.17487/RFC6145, April 2011,
                <https://www.rfc-editor.org/info/rfc6145>.

     [RFC6146]  Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful
                NAT64: Network Address and Protocol Translation from IPv6
                Clients to IPv4 Servers", RFC 6146, DOI 10.17487/RFC6146,
                April 2011, <https://www.rfc-editor.org/info/rfc6146>.

     [RFC6147]  Bagnulo, M., Sullivan, A., Matthews, P., and I. van
                Beijnum, "DNS64: DNS Extensions for Network Address
                Translation from IPv6 Clients to IPv4 Servers", RFC 6147,
                DOI 10.17487/RFC6147, April 2011,
                <https://www.rfc-editor.org/info/rfc6147>.

     [RFC6179]  Templin, F., Ed., "The Internet Routing Overlay Network
                (IRON)", RFC 6179, DOI 10.17487/RFC6179, March 2011,
                <https://www.rfc-editor.org/info/rfc6179>.

     [RFC6221]  Miles, D., Ed., Ooghe, S., Dec, W., Krishnan, S., and A.
                Kavanagh, "Lightweight DHCPv6 Relay Agent", RFC 6221,
                DOI 10.17487/RFC6221, May 2011,
                <https://www.rfc-editor.org/info/rfc6221>.

     [RFC6347]  Rescorla, E. and N. Modadugu, "Datagram Transport Layer
                Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347,
                January 2012, <https://www.rfc-editor.org/info/rfc6347>.

     [RFC6438]  Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
                for Equal Cost Multipath Routing and Link Aggregation in
                Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011,
                <https://www.rfc-editor.org/info/rfc6438>.

     [RFC6621]  Macker, J., Ed., "Simplified Multicast Forwarding",
                RFC 6621, DOI 10.17487/RFC6621, May 2012,
                <https://www.rfc-editor.org/info/rfc6621>.




  Templin                  Expires 6 October 2024               [Page 106]


  Internet-Draft                    AERO                        April 2024


     [RFC6706]  Templin, F., Ed., "Asymmetric Extended Route Optimization
                (AERO)", RFC 6706, DOI 10.17487/RFC6706, August 2012,
                <https://www.rfc-editor.org/info/rfc6706>.

     [RFC7181]  Clausen, T., Dearlove, C., Jacquet, P., and U. Herberg,
                "The Optimized Link State Routing Protocol Version 2",
                RFC 7181, DOI 10.17487/RFC7181, April 2014,
                <https://www.rfc-editor.org/info/rfc7181>.

     [RFC7296]  Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., and T.
                Kivinen, "Internet Key Exchange Protocol Version 2
                (IKEv2)", STD 79, RFC 7296, DOI 10.17487/RFC7296, October
                2014, <https://www.rfc-editor.org/info/rfc7296>.

     [RFC7333]  Chan, H., Ed., Liu, D., Seite, P., Yokota, H., and J.
                Korhonen, "Requirements for Distributed Mobility
                Management", RFC 7333, DOI 10.17487/RFC7333, August 2014,
                <https://www.rfc-editor.org/info/rfc7333>.

     [RFC7401]  Moskowitz, R., Ed., Heer, T., Jokela, P., and T.
                Henderson, "Host Identity Protocol Version 2 (HIPv2)",
                RFC 7401, DOI 10.17487/RFC7401, April 2015,
                <https://www.rfc-editor.org/info/rfc7401>.

     [RFC7761]  Fenner, B., Handley, M., Holbrook, H., Kouvelas, I.,
                Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent
                Multicast - Sparse Mode (PIM-SM): Protocol Specification
                (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, March
                2016, <https://www.rfc-editor.org/info/rfc7761>.

     [RFC8175]  Ratliff, S., Jury, S., Satterwhite, D., Taylor, R., and B.
                Berry, "Dynamic Link Exchange Protocol (DLEP)", RFC 8175,
                DOI 10.17487/RFC8175, June 2017,
                <https://www.rfc-editor.org/info/rfc8175>.

     [RFC8402]  Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
                Decraene, B., Litkowski, S., and R. Shakir, "Segment
                Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
                July 2018, <https://www.rfc-editor.org/info/rfc8402>.

     [RFC8446]  Rescorla, E., "The Transport Layer Security (TLS) Protocol
                Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
                <https://www.rfc-editor.org/info/rfc8446>.

     [RFC8754]  Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J.,
                Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header
                (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020,
                <https://www.rfc-editor.org/info/rfc8754>.



  Templin                  Expires 6 October 2024               [Page 107]


  Internet-Draft                    AERO                        April 2024


     [RFC9000]  Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
                Multiplexed and Secure Transport", RFC 9000,
                DOI 10.17487/RFC9000, May 2021,
                <https://www.rfc-editor.org/info/rfc9000>.

     [RFC9001]  Thomson, M., Ed. and S. Turner, Ed., "Using TLS to Secure
                QUIC", RFC 9001, DOI 10.17487/RFC9001, May 2021,
                <https://www.rfc-editor.org/info/rfc9001>.

     [RFC9002]  Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection
                and Congestion Control", RFC 9002, DOI 10.17487/RFC9002,
                May 2021, <https://www.rfc-editor.org/info/rfc9002>.

     [RFC9365]  Jeong, J., Ed., "IPv6 Wireless Access in Vehicular
                Environments (IPWAVE): Problem Statement and Use Cases",
                RFC 9365, DOI 10.17487/RFC9365, March 2023,
                <https://www.rfc-editor.org/info/rfc9365>.

     [RFC9374]  Moskowitz, R., Card, S., Wiethuechter, A., and A. Gurtov,
                "DRIP Entity Tag (DET) for Unmanned Aircraft System Remote
                ID (UAS RID)", RFC 9374, DOI 10.17487/RFC9374, March 2023,
                <https://www.rfc-editor.org/info/rfc9374>.

  Appendix A.  Non-Normative Considerations

     AERO can be applied to a multitude of Internetworking scenarios, with
     each having its own adaptations.  The following considerations are
     provided as non-normative guidance:

  A.1.  Implementation Strategies for Route Optimization

     Address resolution and route optimization as discussed in
     Section 4.13 results in the creation of NCEs.  The NCE state is set
     to REACHABLE for at most ReachableTime seconds.  In order to refresh
     the NCE lifetime before the ReachableTime timer expires, the
     specification requires implementations to issue a new NS/NA(AR)
     exchange to reset ReachableTime while data messages are still
     flowing.  However, the decision of when to initiate a new NS/NA(AR)
     exchange and to perpetuate the process is left as an implementation
     detail.











  Templin                  Expires 6 October 2024               [Page 108]


  Internet-Draft                    AERO                        April 2024


     One possible strategy may be to monitor the NCE watching for data
     messages for (ReachableTime - 5) seconds.  If any data messages have
     been sent to the neighbor within this timeframe, then send an NS(AR)
     to receive a new NA(AR).  If no data messages have been sent, wait
     for 5 additional seconds and send an immediate NS(AR) if any data
     packets are sent within this "expiration pending" 5 second window.
     If no additional data messages are sent within the 5 second window,
     reset the NCE state to STALE.

     The monitoring of the neighbor data traffic therefore becomes an
     ongoing process during the NCE lifetime.  If the NCE expires, future
     data messages will trigger a new NS/NA(AR) exchange while the
     messages themselves may be delivered over longer paths until route
     optimization state is re-established.

  A.2.  Implicit Mobility Management

     OMNI interface neighbors MAY provide a configuration option that
     allows them to perform implicit mobility management in which no IPv6
     ND messaging is used.  In that case, the Client only transmits
     carrier packets over a single interface at a time, and the neighbor
     always observes carrier packets arriving from the Client from the
     same L2 source address.

     If the Client's underlay interface address changes (either due to a
     readdressing of the original interface or switching to a new
     interface) the neighbor immediately updates the NCE for the Client
     and begins accepting and sending carrier packets according to the
     Client's new address.  This implicit mobility method applies to use
     cases such as cellphones with both WiFi and Cellular interfaces where
     only one of the interfaces is active at a given time, and the Client
     automatically switches over to the backup interface if the primary
     interface fails.

  A.3.  Direct Underlying Interfaces

     When a Client's OMNI interface is configured over a Direct interface,
     the neighbor at the other end of the Direct link can receive original
     IP packets/parcels without any encapsulation.  In that case, the
     Client sends packets/parcels over the Direct link according to
     traffic selectors.  If the Direct interface is selected, then the
     Client's packets/parcels are transmitted directly to the peer without
     traversing an ANET/INET.  If other interfaces are selected, then the
     Client's packets/parcels are transmitted via a different interface,
     which may result in the inclusion of Proxy/Servers and Gateways in
     the communications path.  Direct interfaces must be tested
     periodically for reachability, e.g., via NUD.




  Templin                  Expires 6 October 2024               [Page 109]


  Internet-Draft                    AERO                        April 2024


  A.4.  AERO Critical Infrastructure Considerations

     AERO Gateways can be either Commercial off-the Shelf (COTS) standard
     IP routers or virtual machines in the cloud.  Gateways must be
     provisioned, supported and managed by the INET administrative
     authority, and connected to the Gateways of other INETs via inter-
     domain peerings.  Cost for purchasing, configuring and managing
     Gateways is nominal even for very large OMNI links.

     AERO INET Proxy/Servers can be standard dedicated server platforms,
     but most often will be deployed as virtual machines in the cloud.
     The only requirements for INET Proxy/Servers are that they can run
     the AERO/OMNI code and have at least one network interface connection
     to the INET.  INET Proxy/Servers must be provisioned, supported and
     managed by the INET administrative authority.  Cost for purchasing,
     configuring and managing cloud Proxy/Servers is nominal especially
     for virtual machines.

     AERO ANET Proxy/Servers are most often standard dedicated server
     platforms with one underlay interface connected to the ANET and a
     second interface connected to an INET.  As with INET Proxy/Servers,
     the only requirements are that they can run the AERO/OMNI code and
     have at least one interface connection to the INET.  ANET Proxy/
     Servers must be provisioned, supported and managed by the ANET
     administrative authority.  Cost for purchasing, configuring and
     managing Proxys is nominal, and borne by the ANET administrative
     authority.

     AERO Relays are simply Proxy/Servers connected to INETs and/or ENETs
     that provide forwarding services for non-MNP destinations.  The Relay
     connects to the OMNI link and engages in eBGP peering with one or
     more Gateways as a stub AS.  The Relay then injects its MNPs and/or
     non-MNP prefixes into the BGP routing system, and provisions the
     prefixes to its downstream-attached networks.  The Relay can perform
     ARS/ARR services the same as for any Proxy/Server, and can route
     between the MNP and non-MNP address spaces.

  A.5.  AERO Server Failure Implications

     AERO Proxy/Servers do not present a single point of failure in the
     architecture since all Proxy/Servers on the link provide identical
     services and loss of a Proxy/Server does not imply immediate and/or
     comprehensive communication failures.  Proxy/Server failure can be
     quickly detected and conveyed by Bidirectional Forward Detection
     (BFD) and/or proactive NUD allowing Clients to migrate to new Proxy/
     Servers.





  Templin                  Expires 6 October 2024               [Page 110]


  Internet-Draft                    AERO                        April 2024


     If a Proxy/Server fails, peer carrier packet forwarding to Clients
     will continue by virtue of the neighbor cache entries that have
     already been established through address resolution and route
     optimization.  If a Client also experiences mobility events at
     roughly the same time the Proxy/Server fails, uNA(MM) messages may be
     lost but neighbor cache entries in the DEPARTED state will ensure
     that carrier packet forwarding to the Client's new locations will
     continue for up to DepartTime seconds.

     If a Client is left without a Proxy/Server for a considerable length
     of time (e.g., greater than ReachableTime seconds) then existing
     neighbor cache entries will eventually expire and both ongoing and
     new communications will fail.  The original source will continue to
     retransmit until the Client has established a new Proxy/Server
     relationship, after which time communications can continue .

     Therefore, links that provide many Proxy/Servers with high
     availability profiles are responsive to loss of individual
     infrastructure elements, since Clients can quickly establish new
     Proxy/Server relationships in event of failures.

  A.6.  AERO Client / Server Architecture

     The AERO architectural model is client / server in the control plane,
     with route optimization in the data plane.  The same as for common
     Internet services, the AERO Client discovers the addresses of AERO
     Proxy/Servers and connects to one or more of them.  The AERO service
     is analogous to common Internet services such as google.com,
     yahoo.com, cnn.com, etc.  However, there is only one AERO service for
     the link and all Proxy/Servers provide identical services.

     Common Internet services provide differing strategies for advertising
     server addresses to clients.  The strategy is conveyed through the
     DNS resource records returned in response to name resolution queries.
     As of January 2020 Internet-based 'nslookup' services were used to
     determine the following:

     *  When a client resolves the domainname "google.com", the DNS always
        returns one A record (i.e., an IPv4 address) and one AAAA record
        (i.e., an IPv6 address).  The client receives the same addresses
        each time it resolves the domainname via the same DNS resolver,
        but may receive different addresses when it resolves the
        domainname via different DNS resolvers.  But, in each case,
        exactly one A and one AAAA record are returned.

     *  When a client resolves the domainname "ietf.org", the DNS always
        returns one A record and one AAAA record with the same addresses
        regardless of which DNS resolver is used.



  Templin                  Expires 6 October 2024               [Page 111]


  Internet-Draft                    AERO                        April 2024


     *  When a client resolves the domainname "yahoo.com", the DNS always
        returns a list of 4 A records and 4 AAAA records.  Each time the
        client resolves the domainname via the same DNS resolver, the same
        list of addresses are returned but in randomized order (i.e.,
        consistent with a DNS round-robin strategy).  But, interestingly,
        the same addresses are returned (albeit in randomized order) when
        the domainname is resolved via different DNS resolvers.

     *  When a client resolves the domainname "amazon.com", the DNS always
        returns a list of 3 A records and no AAAA records.  As with
        "yahoo.com", the same three A records are returned from any
        worldwide Internet connection point in randomized order.

     The above example strategies show differing approaches to Internet
     resilience and service distribution offered by major Internet
     services.  The Google approach exposes only a single IPv4 and a
     single IPv6 address to clients.  Clients can then select whichever IP
     protocol version offers the best response, but will always use the
     same IP address according to the current Internet connection point.
     This means that the IP address offered by the network must lead to a
     highly-available server and/or service distribution point.  In other
     words, resilience is predicated on high availability within the
     network and with no client-initiated failovers expected (i.e., it is
     all-or-nothing from the client's perspective).  However, Google does
     provide for worldwide distributed service distribution by virtue of
     the fact that each Internet connection point responds with a
     different IPv6 and IPv4 address.  The IETF approach is like google
     (all-or-nothing from the client's perspective), but provides only a
     single IPv4 or IPv6 address on a worldwide basis.  This means that
     the addresses must be made highly-available at the network level with
     no client failover possibility, and if there is any worldwide service
     distribution it would need to be conducted by a network element that
     is reached via the IP address acting as a service distribution point.

     In contrast to the Google and IETF philosophies, Yahoo and Amazon
     both provide clients with a (short) list of IP addresses with Yahoo
     providing both IP protocol versions and Amazon as IPv4-only.  The
     order of the list is randomized with each name service query
     response, with the effect of round-robin load balancing for service
     distribution.  With a short list of addresses, there is still
     expectation that the network will implement high availability for
     each address but in case any single address fails the client can
     switch over to using a different address.  The balance then becomes
     one of function in the network vs function in the end system.

     The same implications observed for common highly-available services
     in the Internet apply also to the AERO client/server architecture.
     When an AERO Client connects to one or more ANETs, it discovers one



  Templin                  Expires 6 October 2024               [Page 112]


  Internet-Draft                    AERO                        April 2024


     or more AERO Proxy/Server addresses through the mechanisms discussed
     in earlier sections.  Each Proxy/Server address presumably leads to a
     fault-tolerant clustering arrangement such as supported by Linux-HA,
     Extended Virtual Synchrony or Paxos.  Such an arrangement has
     precedence in common Internet service deployments in lightweight
     virtual machines without requiring expensive hardware deployment.
     Similarly, common Internet service deployments set service IP
     addresses on service distribution points that may relay requests to
     many different servers.

     For AERO, the expectation is that a combination of the Google/IETF
     and Yahoo/Amazon philosophies would be employed.  The AERO Client
     connects to different ANET access points and can receive 1-2 Proxy/
     Server ULAs at each point.  It then selects one AERO Proxy/Server
     address, and engages in RS/RA exchanges with the same Proxy/Server
     from all ANET connections.  The Client remains with this Proxy/Server
     unless or until the Proxy/Server fails, in which case it can switch
     over to an alternate Proxy/Server.  The Client can likewise switch
     over to a different Proxy/Server at any time if there is some reason
     for it to do so.  So, the AERO expectation is for a balance of
     function in the network and end system, with fault tolerance and
     resilience at both levels.

  Appendix B.  Change Log

     << RFC Editor - remove prior to publication >>

     Changes from earlier versions:

     *  Submit for review.

  Author's Address

     Fred L. Templin (editor)
     Boeing Research & Technology
     P.O. Box 3707
     Seattle, WA 98124
     United States of America
     Email: fltemplin@acm.org












  Templin                  Expires 6 October 2024               [Page 113]


