Append and delete configuration

[anjan-keysight]

Issue #409

Redocly View:

Config:
https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-traffic-generator/models/dev-append-delete-config/artifacts/openapi.yaml&nocors#tag/Configuration

Objectives:

There is a long pending request for support for appending and deleting configuration resources to existing configuration.
One such end use case is when we want to provide support for multiple endpoint for traffic. At present, due to maximum stream limitation per port, we can not support RSVP traffic with 64K LSPs or 1000s of DHCP clients with address field set as auto ( not known before running DHCP) . With multiple endpoint support in same flow configuration, we can pack multiple LSPs into one stream.

Furthermore, in such a scenario, resolving "auto" mode for LSP fields for multiple sub-streams, which involves learning LSP data and then internally crafting all the sub-streams into a composite data pattern. This becomes complex to implement and difficult for end user to follow and have control over.

The simplest approach is to be able to allow user to configure and start protocol, then collect control plane parameters
like MPLS label, DHCP IPs etc. Finally these learned information can then be used to define and configure the sub-flows,
likely for a full mesh traffic endpoints. User can then append the traffic portion of the configuration and start the traffic.

Another use case is to provide flexibility to user to configure and run batches traffic streams one after another, after
removing the previous batch of streams for the same running protocol endpoints. Append config in conjunction with delete
config can be used as outlined in the examples below.

In some cases, the contents of the flows to be created are just not known when creating the initial config. This PR also handles these usecases
e.g. open-traffic-generator/ixia-c#56 (comment)

Note: This PR addresses appending and deleting configuration of resource type flows only for our initial delivery. How we would
like to append / delete other resource types in future is addressed in this PR.

Also note, the append / delete (flow) configuration APIs will expect all traffic to be in stopped state. Any running flow will be implicitly stopped and a warning returned with these new API calls. User will then have to explicitly start the stopped flows, which will result in stats reset.

Sample Code Snippet

config := gosnappi.NewConfig()
RsvpIngressTunnel := 10

// add ports
p1 := config.Ports().Add().SetName("p1").SetLocation(opts.IxiaCPorts()[0])
p2 := config.Ports().Add().SetName("p2").SetLocation(opts.IxiaCPorts()[1])

// add devices
d1 := config.Devices().Add().SetName("p1d1")
d2 := config.Devices().Add().SetName("p2d1")

// add protocol stacks for device d1
d1Eth1 := d1.Ethernets().
	Add().
	SetName("p1d1Eth1").
	SetMac("00:21:00:00:00:11").
	SetMtu(1450)

d1Eth1.
	Connection().
	SetPortName(p1.Name())

d1Eth1.
	Ipv4Addresses().
	Add().
	SetName("p1d1ipv4").
	SetAddress("21.1.1.1").
	SetGateway("21.1.1.2").
	SetPrefix(24)

// isis router
d1isis := d1.Isis().
	SetName("p1d1isis").
	SetSystemId("640000000001")

d1isis.Basic().SetIpv4TeRouterId("21.1.1.1")
d1isis.Basic().SetHostname("ixia-Ingress")

//isis interface
d1isisint := d1isis.Interfaces().
	Add().
	SetName("p1d1int").
	SetEthName("p1d1Eth1").
	SetNetworkType(gosnappi.IsisInterfaceNetworkType.POINT_TO_POINT).
	SetLevelType(gosnappi.IsisInterfaceLevelType.LEVEL_2).
	SetMetric(10)

//isis advanced settings
d1isisint.
	Advanced().SetAutoAdjustSupportedProtocols(false)
// RSVP
d1rsvp := d1.Rsvp().SetName("IxiaIngress")
d1rsvp.Ipv4Interfaces().
	Add().SetIpv4Name("p1d1ipv4").
	SetNeighborIp("21.1.1.2")

d1rsvpLsp := d1rsvp.LspIpv4Interfaces().Add().SetIpv4Name("p1d1ipv4")
// d1rsvpLsp.P2PEgressIpv4Lsps().SetName("Rsvp-EG-1")

for i := int32(1); i <= RsvpIngressTunnel; i++ {
	name := fmt.Sprint("Ingress_lsp", i)
	d1RsvpIngress := d1rsvpLsp.P2PIngressIpv4Lsps().Add()
	d1RsvpIngress.SetName(name)
	d1RsvpIngress.SetRemoteAddress("22.1.1.1").
		SetRefreshInterval(30).
		SetTimeoutMultiplier(3).
		SetTunnelId(uint32(i))
	d1RsvpIngress.Ero().SetPrependNeighborIp("prepend_loose")
	d1RsvpIngress.Ero().Subobjects().Add().SetType("ipv4").
		SetIpv4Address("22.1.1.1").
		SetPrefixLength(24).
		SetHopType("loose")

}
// add protocol stacks for device d2
d2Eth1 := d2.Ethernets().
	Add().
	SetName("p2d1Eth1").
	SetMac("22:00:00:00:00:12").
	SetMtu(1450)

d2Eth1.
	Connection().
	SetPortName(p2.Name())

d2Eth1.
	Ipv4Addresses().
	Add().
	SetName("p2d1ipv4").
	SetAddress("22.1.1.1").
	SetGateway("22.1.1.2").
	SetPrefix(24)

// isis router
d2isis := d2.Isis().
	SetName("p2d1isis").
	SetSystemId("650000000001")

d2isis.Basic().SetIpv4TeRouterId("22.1.1.1")
d2isis.Basic().SetHostname("ixia-Egress")

//isis interface
d2isisint := d2isis.Interfaces().
	Add().
	SetName("p2d1int").
	SetEthName("p2d1Eth1").
	SetNetworkType(gosnappi.IsisInterfaceNetworkType.POINT_TO_POINT).
	SetLevelType(gosnappi.IsisInterfaceLevelType.LEVEL_2).
	SetMetric(10)
//isis advanced settings
d2isisint.
	Advanced().SetAutoAdjustSupportedProtocols(false)

// RSVP
d2rsvp := d2.Rsvp().SetName("IxiaEgress")
d2rsvp.Ipv4Interfaces().
	Add().SetIpv4Name("p2d1ipv4").
	SetNeighborIp("22.1.1.2")

d2rsvpLsp := d2rsvp.LspIpv4Interfaces().Add().SetIpv4Name("p2d1ipv4")
d2RsvpEgress := d2rsvpLsp.P2PEgressIpv4Lsps()

d2RsvpEgress.SetName("Rsvp-EG-2").
	SetEnableFixedLabel(false).
	SetRefreshInterval(30).
	SetReservationStyle("fixed_filter").
	SetTimeoutMultiplier(3)

flowCount := int(RsvpIngressTunnel)
if _, err := client.SetConfig(config); err != nil {
	t.Fatal(err)
}
if err := client.StartProtocol(); err != nil {
	t.Fatal(err)
}

//ISIS metrics
err = api.WaitFor(
	func() (bool, error) {
		return client.AllIsisSessionUp(config, gosnappi.IsisInterfaceLevelType.LEVEL_2, 3)
	}, opts, nil,
)

reqLabel := gosnappi.NewStatesRequest()
reqLabel.RsvpLsps().SetRsvpRouterNames([]string{"IxiaIngress"})
resLabel, err := client.GetStates(reqLabel)
if err != nil {
	t.Fatal(err)
}
learnedLabels := []uint32{}
for _, rsvpLsp := range resLabel.Items() {
	if rsvpLsp.RsvpRouterName() == "IxiaIngress" {
		for _, lsp := range rsvpLsp.Ipv4Lsps.Items() {
			if lsp.SourceAddress() == "21.1.1.1" && lsp.DestinationAddress() == "22.1.1.1" {
				learnedLabels[lsp.TunnelId()] = lsp.LabelIn()
			}
		}
	}
}
// misc error checking and learned info data integrity goes here

// reset initial config and specifically add traffic details only
ca := gosnappi.NewConfigAppend()
caf := ca.ConfigAppendList().Add().Flows()
flow := caf.Add()
flow.Metrics().SetEnable(true)
flow.Duration().FixedPackets().SetPackets(500)
flow.Rate().SetPps(10)

// add endpoints and packet description flow
flow.SetName("f1").
	TxRx().Device().
	SetTxNames([]string{d1Eth1.Name()}).
	SetRxNames([]string{d2RsvpEgress.Name()})

f1Eth := flow.Packet().Add().Ethernet()
f1Eth.Src().SetValue(d1Eth1.Mac())
f1Eth.Dst().Auto()

f1Mpls := flow.Packet().Add().Mpls()
f1Mpls.Label().SetValues(learnedLabels)

f1Ip := flow.Packet().Add().Ipv4()
f1Ip.Src().SetValue("21.1.1.1")
f1Ip.Dst().SetValue("22.1.1.1")


if _, err := client.AppendConfig(ca); err != nil {
	t.Fatal(err)
}
if err := client.StartTransmit(); err != nil {
	t.Fatal(err)
}

// After traffic metrics measurement and verification, we remove streams
if err := client.StopTransmit(); err != nil {
	t.Fatal(err)
}

cd := gosnappi.NewConfigDelete()
cd.ConfigDeleteList().Add().SetFlows([]string{"f1"})
if _, err := client.DeleteConfig(cd); err != nil {
	t.Fatal(err)
}

// Now we can reconfigure and start a new stream
ca = gosnappi.NewConfigAppend()
caf = ca.ConfigAppendList().Add().Flows()
flow := caf.Add()
flow.Metrics().SetEnable(true)
flow.Duration().FixedPackets().SetPackets(500)
flow.Rate().SetPps(10)

// add endpoints and packet description flow
flow.SetName("f1").
	TxRx().Device().
	SetTxNames([]string{d2RsvpEgress.Name()}).
	SetRxNames([]string{d1Eth1.Name()})

f1Eth := flow.Packet().Add().Ethernet()
f1Eth.Src().SetValue(d2Eth1.Mac())
f1Eth.Dst().Auto()

f1Ip := flow.Packet().Add().Ipv4()
f1Ip.Src().SetValue("22.1.1.1")
f1Ip.Dst().SetValue("21.1.1.1")

if _, err := client.AppendConfig(ca); err != nil {
	t.Fatal(err)
}
if err := client.StartTransmit(); err != nil {
	t.Fatal(err)
}
.

[apratimmukherjee]

Looks good to me.

[abhijit-dhar]

From code snippet it appears that traffic-stop is required before starting the appended/deleted flow, if there are already some running flows. So, after adding the new flows we need to do start transmit again (this was not obvious). As a consequence, the previous traffic's stats will get reset and in the next stat-fetch we will see old existing stats are starting from beginning. This is also not obvious from the document. May be implicit from the script but not explicit.

Suggestion

• In a note please mention that traffic/start stop is mandatory step for using this feature. if you are not handling this stop/start implicitly issue a warning/error. Explicitly mention where any why traffic need to be stop/start
• If start/stop is implicitly handled, the also a warning is needed if a traffic stream was already running then its stats will be reset,

As a user I want to write as little code as possible. So, make the stop traffic implicit when starting the new flow, with a warning that previously running flow's stats will be reset.

[anjan-keysight]

@abhijit-dhar Added a note with regard to the traffic state.

[anjan-keysight]

@jasdeep-hundal gentle reminder to pls review the PR. Starting Monday we plan to proceed with the next set of deliverables based on this PR.