Thanks a lot for your reviews. We will go over your comments in detail.
Post by Y. Richard YangDanny, Christian,
Very interesting work. I finished a review of Sec. 6, and below is
the second part of my review.
Richard
====
6. Proposed Approach
The primary design goal for ALTO-based Broker-assisted Multi-domain
Orchestration is to discover resource and topology information from
different administrative domains involved in the federation, while
also safeguarding the privacy and autonomy of every domain.
[yry] Connect to all requirements above?
In the architectural proposal shown in Figure 1, a broker component
is conceived to be working as coordinator of a set of MdOs, whose key
[yry] coordinator -> a coordinator; whose it is not clear righ away
what "whose" refers to, MdOs or broker?
components are: the Inter-domain Resource (IdR), the Inter-domain
Topology (IdT) and the ALTO Server.
BROKER COMPONENT
[yry] COMPONENTS
+--------------------------------------------+
| |
| +-----------------+ |
| | | |
XXXXXXXXXXXXXXXXXXX ALTO SERVER(s) | |
X | | + |
X | +----------------+\ |
X | / \ |
X | / \ |
X | +------+/+-------+ +---------------+ |
X | | Inter-domain | | Inter-domain | |
X | | Topology (IdT) | | Resource (IdR)| |
X | +-^------^-------+ +---^-------^---+ |
X | | | | | |
X +--------------------------------------------+
X | | | |
X | | | |
+--X--------+---+--------------------+ |
| | | |
| | +------------+------------+---+
| MdO1 | | |
| +<------------->+ +---+
+---------------+ | MdO2 | |
| | |
+-+--------------+ |
| |
Legend: | MdO(n) |
XXX ALTO Protocol +------------------+
Figure 1: Broker-assisted Multi-operator Network Architecture
[yry] Does it help to label all, or key edges, not only the single
XXX, on which protocol (protocols) will be used? You mentioned
BGP/BGP-LS, ...
6.1. Inter-domain Resource (IdR) Component
It creates a hierarchical database that contains inter-domain
resource information such as resource availability (i.e., CPU,
memory, and storage), Virtual Network Functions (VNFs) and Physical
Network Functions (PNFs) supported and Service Access Points (SAPs)
to access those resources. UNIFY [UNIFY.NFFG], TOSCA [TOSCA], ETSI-
[yry] It can be ambiguous what Service Access Points refer to,
only VNFs/PNFs or all? The idea of a hierarchical database appears to
be not fully specified.
NFV [ETSI-NFV-MANO], among other data models can be used to create
the interface between IdR and MdOs.
[yry] IdR not defined yet---appeared only in the section title.
6.2. Inter-domain Topology (IdT) Component
A hierarchical TED (Traffic Engineering Database) that contains
inter-domain network topology information including additional key
parameters (e.g., throughput and latency of links). This information
can be retrieved from each MdO through BGP-LS or REST interfaces.
6.3. ALTO Server Functionalities
The ALTO server component is the core of the broker layer. Multiple
logically centralized ALTO servers use the information collected from
IdR and IdT modules to create and provide abstract maps with a
[yry] module vs component
simplified view, yet enough information about MdOs involved in the
federation. This information includes domain-level topology, storage
resources, computation resources, networking resources and PNF/VNF
capabilities.
[yry] Consistency: missing CPU, as above?
As an ALTO client, each MdO sends ALTO service queries to the ALTO
server. This server provides aggregated inter-domain information
exposed as set ALTO base services defined in [RFC7285], e.g., Network
Map, Cost Map and ALTO extension services, e.g., Property
Map [DRAFT-PM], Multi-Cost Map [RFC8189], Path Vector [DRAFT-PV].
For example, when a source MdO receives a customer service request,
it checks whether or not it can deliver the full service. If it is
unable to do so, the MdO consumes from the ALTO Server the Property
Map service to have a clear global view of the resource information
offered by other MdOs. This information allows discovering which
candidate MdOs may be contacted to deliver the remaining requirements
of a requested end-to-end service deployment. The connectivity
information among discovered MdOs can be retrieved by a Cost Map
service, responding, for instance, a path vector with the AS-level
topology distance between the source MdO and candidate MdOs.
6.4. Filtered Cost Map Extension
[yry] A bit disruptive transition. Why only Filtered Cost Map?
It helps to state the basic ALTO services that broker uses and
state that this is the one needs extension---I actually think
this should be a new service, instead of filtered cost map.
The ALTO server MUST provide connectivity information for every SG
link in the SG path for an E2E requirement. This information is the
AS-level topology distance in the form of path vector, and it
includes all possible ways for each (source node, destination node)
pair in the SG link.
[yry] Is it necessary to limit to only AS-hop distance? I assume
that other distances such as bw, latency can be useful as well.
In this section, we introduce a non-normative overview of the
Filtered Cost Map defined in Section 6.1 of [DRAFT-PV] [1].
The specifications for the "Media Types", "HTTP method",
"Capabilities" and "Uses" (described in Section 6.1 of [DRAFT-PV]
[2]) are unchanged.
6.4.1. Accept Input Parameters
The ReqFilteredCostMap object in Section 6.1.2 of [DRAFT-PV] [3] is
object {
[NFFG sg;]
} ReqFilteredCostMap;
[yry] If sg is optional, the req can have an empty body. What is
the response to an empty request? Non-optional makes more sense,
to me.
[yry] One key question that we should think: is it possible that
NFFG is an example of a more *generic* query model, for example,
waypoint model, so that the generic query is
src -> dst: waypoints?
object {
JSONString nfs<1..*>;
JSONString saps<1..*>;
NextHops sg_links<1..*>;
REQs reqs<1..*>;
} NFFG;
object {
JSONNumber id;
JSONString src-node;
JSONString dst-node;
} NextHops;
object {
JSONString id;
JSONString src-node;
JSONString dst-node;
JSONNumber sg-path<1..*>;
} REQs;
sg: If present, the ALTO Server MUST allow the request input to
include an SG with a formatted body as an NFFG object. An NFFG
object contains NFs, SAPs, SG links representing logical
connections between NFs, SAPs or both and E2E requirements as a
list of ids of SG links.
It is worth noting that further versions of this draft will define a
more elaborated NFFG object to support extended parameters such as
monitoring parameters, resource requirements, etc.
[yry] Also, need consistency requirement, for example, src-node string
appears in union of nfs, saps ...
6.4.2. Response
If the ALTO client includes the path vector cost mode in the "cost-
type" or "multi-cost-types" field of the input parameter, the
response for each SG link in each E2E requirement MUST be encoded as
a JSONArray of JSONArrays of JSONStrings. Anyone of the sub-arrays
indicates a potential candidate path calculated as the per-domain
topological distance corresponding to the amount of traversing
domains.
[yry] The encoding is quite interesting, but without an example,
and a bit explanation, it can be hard to understand the nested
data structure. I feel that the model can be a generic model,
such as waypoint, beyond SG; see above.
Moreover, as defined in Section 6.3.6 of [DRAFT-PV] [4], If an ALTO
client sends a request of the media type "application/alto-
costmapfilter+json" and accepts "multipart/related", the ALTO server
MUST provide path vector information along with the associated
Property Map information (e.g., entry points of the corresponding
foreign domains), in the same body of the response.
Section 6.5.2 gives an example of the Filtered Cost Map query and the
corresponding responses.
6.5. Examples of Message Exchange
This section list a couple of examples of the Property Map and
Filtered Cost Map queries and the corresponding responses. These
responses are based on the information in Table 1 and Table 2 of a
use case implementation described in Appendix A.
6.5.1. Property Map Service
In this example, the ALTO client wants to retrieve the entire
Property Map for PID entities with the "entry-point", "cpu", "mem",
"storage", "port" and "nf" properties.
GET /propmap/full/inet-ucmspn HTTP/1.1
Host: alto.example.com
Accept: application/alto-propmap+json,application/alto-error+json
HTTP/1.1 200 OK
Content-Length: ###
Content-Type: application/alto-propmap+json
{
"property-map": {
"pid:AS1": {
"entry-point": [ "http://172.25.0.10:8888/escape" ],
"cpu": [ "50.0" ],
"mem": [ "60.0" ],
"storage": [ "70.0" ],
"port": [ "SAP1" ],
"nf": [ "NF1", "NF3" ]
},
"pid:AS2": {
"entry-point": [ "http://172.26.0.10:8888/escape" ],
"cpu": [ "10.0" ],
"mem": [ "20.0" ],
"storage": [ "30.0" ],
"nf": [ "NF2" ]
},
"pid:AS3": {
"entry-point": [ "http://172.27.0.10:8888/escape" ],
"cpu": [ "80.0" ],
"mem": [ "90.0" ],
"storage": [ "100.0" ],
"port": [ "SAP2" ],
"nf": [ "NF1", "NF3" ]
}
}
}
[yry] A key issue that one may encounter in multi-domain
aggregation is ambiguity of aggregated resources. Is the semantics
that each resource is the *local* resource? If so, is there
a naming/scoping requirement on naming entities?
[yry] If you do hierarchical brokers and each broker can hide the
details, the resource aggregation can become ambiguous: (1) broker 1 can
say it has 10 units of CPUs, and broker 2 can say the same, but they
may aggregate the 10 from the same individual domain.
6.5.2. Filtered Cost Map Service
The following example uses the Filtered Cost Map service to request
the path vector for a given E2E requirement. The SG request
information in Table 2 is used to describe the service, and it is
composed of three NFs (NF1, NF2, and NF3) and two SAPs (SAP1 and
SAP2). Links connecting the NFs and SAPs ("sg_links" tag) are also
included, followed by an E2E requirement ("reqs" tag) with
information about the order in which NFs are traversed from SAP1 to
SAP2.
Note that the request accepts "multipart/related" media type. This
means the ALTO server will include associated property information in
the same response.
POST /costmap/pv HTTP/1.1
Host: alto.example.com
Accept: multipart/related, application/alto-costmap+json,
application/alto-propmap+json, application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-costmapfilter+json
{
"cost-type": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
"sg": {
"nfs": [ "NF1", "NF2", "NF3" ],
"saps": [ "SAP1", "SAP2" ],
"sg_links":[
{
"id": 2,
"src-node": "SAP1",
"dst-node": "NF1",
},
{
"id": 2,
"src-node": "NF1",
"dst-node": "NF2",
},
{
"id": 3,
"src-node": "NF2",
"dst-node": "NF3",
},
{
"id": 4,
"src-node": "NF3",
"dst-node": "SAP2",
}
],
"reqs": [
{
"id": 1,
"src-node": "SAP1",
"dst-node": "SAP2",
"sg-path": [ 1, 2, 3, 4 ]
}
]
}
}
The ALTO server returns connectivity information for the E2E
requirement provided by the ALTO Client request of the above example.
Also, the response includes Property Map information for each element
in the path vector. In this case, it is retrieved a Property Map
with the "entry-point" property, i.e., the URL of the MdO entry point
for the corresponding network.
HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: multipart/related; boundary=example
--example
Content-Type: application/alto-endpointcost+json
{
"meta": {
"cost-type": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
},
"cost-map": {
"SAP1": {
"SAP2": {
"SAP1": {
"NF1": [
[ "AS1" ], [ "AS1", "AS2", "AS3" ]
]
},
"NF1": {
"NF2": [
[ "AS1", "AS2" ], [ "AS3", "AS2" ]
]
},
"NF2": {
"NF3": [
[ "AS2", "AS1" ], [ "AS2", "AS3" ]
]
},
"NF3": {
"SAP2": [
[ "AS1", "AS2", "AS3" ], [ "AS3" ]
]
}
}
}
}
}
--example
Content-Type: application/alto-propmap+json
{
"property-map": {
"pid:AS1": { "entry-point": "http://172.25.0.10:8888/escape" },
"pid:AS2": { "entry-point": "http://172.26.0.10:8888/escape" },
"pid:AS3": { "entry-point": "http://172.27.0.10:8888/escape" }
}
}
--example--
[yry] Very interesting design. It helps if you add some text to go over
the meaning, such as [ "AS1" ], [ "AS1", "AS2", "AS3" ] are two
alternative
paths.
====
Post by Y. Richard YangDear Danny, Christian,
I started to read this draft. It addresses an important, timely issue.
Below is the first part (on Sections 1-6) of my review, and I will send the
second part on Sections 7-9 by tomorrow.
Great work!
Richard
====
Existing proposals for the network service orchestration are
intrinsically conceived for single administrative domain scenarios.
[yry] Add citations right after existing proposals?
For example, in the standard service orchestration model described in
ETSI NFV MANO framework [ETSI-NFV-MANO], one orchestrator is supposed to
work within one administrative domain. The analysis of
orchestration and management of network Services over multiple
[yry] case, network vs Services
administrative domains have begun to be addressed by ETSI
in [ETSI-NFV-MANO-MDO].
[yry] The last sentence gives a related work. To make reading easier,
it may help to differentiate between proposals (first sentence) and
analysis (this sentence).
All such proposals described above envision the potential
introduction of new business model approaches, including federation
models [PPP-5:2013] among administrative domains. In this context,
this document considers each network operator involved in the
community advertises its abstracted capabilities (e.g., software/
hardware resources, physical/virtual network functions, etc.) to a
broker (i.e., 3rd party). This latter, in its turn, provides or
[yry] latter -> broker
assists coordinate E2E network services spanning multi-domain
networks.
5. Problem Statement and Challenges
The provision of a complete E2E network service requires chaining
services provided by multiple network operator with multiple
[yry] operator -> operators
technologies. In this multi-domain environment, the orchestration
process will require an advertise mechanism through which single
[yry] advertise -> advertisement, single -> individual
domains can describe their capabilities, resources, and VNFs in an
interoperable manner. Moreover, a discovery mechanism is also
necessary so that source domains can obtain candidate domains (with
[yry] Although one can infer what source/candidate domains mean,
it can help if you define them.
the corresponding connectivity information) which can provide a part
of the service and/or slide in an E2ENS requirement.
[yry] slide -> slice?
In order to the advertising and discovery process works in a proper
[yry] "In order to" -> "In order that"
Lack of Abstractions: Multiple vendors with heterogeneous
technologies need an information model to adequately represent
in confidentiality-preserving fashion the resource and topology
information.
Scalability: Involves the distribution of topology and resource
information in a peer-to-peer fashion (MdO-to-MdO). Multi-
[yry] Note that the previous item mentions generic information model
but here mentions only topology and resource information. It can
help if
the scoping is clarified up front: only topology/resource.
operator multi-domain environments where the information
distribution is advertised in a peer-to-peer model scales
linearly. It means more MdO interconnections one has, the more
[yry] Why linearly? There can be ambiguity. Assume the total
transmission load of one domain grows linearly with n, which is
the number of domains. Total system scales w/ n^2. But there can be
other peer-to-peer systems such as CHORD which publish
(advertises) with a constant cost. It helps to specify some more details.
it "costs" to distribute.
Flexibility: Considers that a distributed approach does not allow
domains without physical infrastructure (e.g., without BGP or
BGP-LS) to advertise resource capabilities and networking
resources. Such procedures consist in deploying and configuring
physical peering points for these domains.
[yry] The description above is not clear to me yet. One can run BGP
w/o a physical infrastructure. It helps if you can clarify this challenge.
Complexity: Refers to the discovery mechanism to pre-select
candidate domains, accounting for resources and capabilities,
necessary for an E2E network service deployment. An intrinsic
complexity exists in the process of assembling, logically
organizing, and enabling abstraction views of different
resources and capabilities in multi-domain scenarios.
[yry] A summary comment on the 4 items. Are they challenges, issues
or requirements? The wording appears to have multiple aspects: lacking
of abstractions seems to imply issues; flexibility seems to imply
requirements... Is the set complete, for example how about security,
autonomy, privacy ...---some you touch right below? One more comment is
that the terms used (e.g., flexibility) can be too generic. If you go a bit
more specific, for example, Discovery Flexibility, it may help.
====
On Mon, Jul 2, 2018 at 8:25 PM Danny Alex Lachos Perez <
Post by Danny Alex Lachos PerezDear WG members
This new version (-01) considers a section on benefits and open
questions (section 7) where we analyze the advantages and open issues in
the broker-assisted architecture. Moreover, we removed the Property Map
Extension section because the current Property Map draft [0] already
supports property values encoded as JSONArray.
Other changes include (i) update the Problem Statement and Challenges
section and (ii) many minor style and grammar edits.
Comments and feedback will be highly appreciated.
Thank you very much
[0]
https://tools.ietf.org/html/draft-ietf-alto-unified-props-new-04#section-4.6
Ss
Danny Lachos
A new version of I-D, draft-lachosrothenberg-alto-brokermdo-01.txt
has been successfully submitted by Danny Alex Lachos Perez and posted
to the
IETF repository.
Name: draft-lachosrothenberg-alto-brokermdo
Revision: 01
Title: ALTO-based Broker-assisted Multi-domain Orchestration
Document date: 2018-07-02
Group: Individual Submission
Pages: 22
https://www.ietf.org/internet-drafts/draft-lachosrothenberg-alto-brokermdo-01.txt
https://datatracker.ietf.org/doc/draft-lachosrothenberg-alto-brokermdo/
https://tools.ietf.org/html/draft-lachosrothenberg-alto-brokermdo-01
https://datatracker.ietf.org/doc/html/draft-lachosrothenberg-alto-brokermdo
https://www.ietf.org/rfcdiff?url2=draft-lachosrothenberg-alto-brokermdo-01
Evolving networking scenarios (e.g., 5G) demand new multiple
administrative domain (aka multi-domain) orchestration models. This
document proposes the use of Application-Layer Traffic Optimization
(ALTO) services to offer topology and resources addressing network
service discovery and provisioning by multi-domain orchestrators.
The ALTO services with the proposed protocol extension offer
aggregated views on various types of resources contributing to a more
simple and scalable solution for resource and service discovery in
multi-domain, multi-technology environments.
Please note that it may take a couple of minutes from the time of
submission
until the htmlized version and diff are available at tools.ietf.org.
The IETF Secretariat
_______________________________________________
alto mailing list
https://www.ietf.org/mailman/listinfo/alto
--
--
=====================================
| Professor of Computer Science |
| http://www.cs.yale.edu/~yry/ |
=====================================
--
--
=====================================
| Professor of Computer Science |
| http://www.cs.yale.edu/~yry/ |
=====================================