Skip to content

ON-16435: Replace arch checks with nic capabilies #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

ON-16435: Replace arch checks with nic capabilies #75

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5546a0d
ON-16435: Add getter for nic res flags
ligallag-amd Apr 3, 2025
c9bf7ae
ON-16435: Add new ZF_RES_NIC_FLAGS and use them over arch checks
ligallag-amd Apr 3, 2025
bf7eff7
ON-16435: Add ef10ct arch check where capabilitiy is unclear
ligallag-amd Apr 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/include/zf_internal/private/zf_stack_def.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,9 @@ struct zf_stack {
#define ZF_RES_NIC_FLAG_VLAN_FILTERS 0x1
#define ZF_RES_NIC_FLAG_RX_LL 0x2
#define ZF_RES_NIC_FLAG_TX_LL 0x4
#define ZF_RES_NIC_FLAG_RX_REF 0x8
#define ZF_RES_NIC_FLAG_CTPIO_ONLY 0x10
#define ZF_RES_NIC_FLAG_PIO 0x20

#include <onload/version.h>
#define ZF_VERSION_LENGTH_MAX OO_VER_STR_LEN
Expand Down Expand Up @@ -341,4 +344,9 @@ static inline const ef_vi* zf_stack_nic_tx_vi(const zf_stack* st, int nicno) {
static inline bool zf_stack_nic_has_tx_vi(const zf_stack* st, int nicno) {
return st->nic[nicno].tx_vi.inited; }

static inline unsigned* zf_stack_res_nic_flags(zf_stack* st, int nicno) {
struct zf_stack_impl* sti = ZF_CONTAINER(struct zf_stack_impl, st, st);
return &sti->nic[nicno].flags;
}

#endif /* __ZF_INTERNAL_STACK_DEF_H__ */
5 changes: 3 additions & 2 deletions src/include/zf_internal/tx_send.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ zf_send(struct zf_tx* restrict tx,
return -EAGAIN;
#endif

if( vi->nic_type.arch == EF_VI_ARCH_EFCT ) {
if( *zf_stack_res_nic_flags(st, nicno) & ZF_RES_NIC_FLAG_CTPIO_ONLY ) {
/* FIXME get X3 overhead sorted properly */
if(ZF_UNLIKELY( ef_vi_transmit_space_bytes(vi) < (int)tot_len + 128 )) {
if( st->flags & ZF_STACK_FLAG_TRANSMIT_WARM_ENABLED )
Expand Down Expand Up @@ -207,7 +207,7 @@ zf_send(struct zf_tx* restrict tx,
!ctpio_is_allowed(st_nic, tot_len) &&
pio_is_available(st_nic) &&
tot_len <= max_pio_len(st_nic) && iov_cnt <= 2 ) {
zf_assert_nequal(vi->nic_type.arch, EF_VI_ARCH_EFCT);
zf_assert(*zf_stack_res_nic_flags(st, nicno) & ZF_RES_NIC_FLAG_PIO);
int pio_buf_no = st_nic->pio.busy & 1;
int orig_pio_ofs = pio_buf_no ? max_pio_len(st_nic) : 0;
rc = ef10_ef_vi_transmitv_copy_pio(vi, orig_pio_ofs, iov, iov_cnt, 1);
Expand Down Expand Up @@ -288,6 +288,7 @@ zf_send(struct zf_tx* restrict tx,
}
}
zf_assert_nequal(vi->nic_type.arch, EF_VI_ARCH_EFCT);
zf_assert_nequal(vi->nic_type.arch, EF_VI_ARCH_EF10CT);
Comment on lines 290 to +291
Copy link
Collaborator

@sianj-xilinx sianj-xilinx Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assert that CTPIO_ONLY capability wasn't set?

/* Strictly speaking, we should call ef_vi_transmit_ctpio_fallback()
* to post a CTPIO fallback descriptor. However, that entry point is
* actually identical to this one, and this code path is also used
Expand Down
2 changes: 1 addition & 1 deletion src/include/zf_internal/zf_stack.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ ZF_HOT static inline void zf_stack_refill_rx_ring(struct zf_stack* st,
int nic, unsigned max_count)
{
ef_vi* vi = &st->nic[nic].vi;
if( vi->nic_type.arch == EF_VI_ARCH_EFCT )
if( *zf_stack_res_nic_flags(st, nic) & ZF_RES_NIC_FLAG_RX_REF )
return;
unsigned space = ef_vi_receive_space(vi);
if( space < st->nic[nic].rx_ring_refill_batch_size )
Expand Down
9 changes: 5 additions & 4 deletions src/lib/zf/private/reactor.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -508,10 +508,11 @@ __zf_reactor_perform(struct zf_stack* st, unsigned spin_cnt)
* from the future in the next RX buffer. */
pkt_id next_packet_id = 0;
char* next_packet;
bool is_efct = vi->nic_type.arch == EF_VI_ARCH_EFCT;
bool is_rx_ref = *zf_stack_res_nic_flags(st, nic) &
Copy link
Collaborator

@sianj-xilinx sianj-xilinx Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're using these flags on the critical path. The zf_stack_impl is used for cold path state. If we need the checks on the hot path that information should move to zf_stack_nic. That might just be the RX_REF capability - probably needs a bit of care to ensure that this is done in a way that doesn't impact performance.

ZF_RES_NIC_FLAG_RX_REF;
bool packet_present;

if( is_efct ) {
if( is_rx_ref ) {
char* vpkt = (char*)efct_vi_rx_future_peek(vi);
packet_present = vpkt != NULL;
if( packet_present ) {
Expand Down Expand Up @@ -643,7 +644,7 @@ zf_reactor_purge_event(struct zf_stack* st, int nic, ef_vi* vi, ef_event* ev)
case EF_EVENT_TYPE_RX:
zf_assert_nequal(EF_EVENT_RX_SOP(*ev), 0);
zf_assert_equal(EF_EVENT_RX_CONT(*ev), 0);
zf_assert_nequal(vi->nic_type.arch, EF_VI_ARCH_EFCT);
zf_assert(!(*zf_stack_res_nic_flags(st, nic) & ZF_RES_NIC_FLAG_RX_REF));
zf_pool_free_pkt(&st->pool, EF_EVENT_RX_RQ_ID(*ev));
zf_log_event_trace(st, "%s: purged event %x\n", __FUNCTION__,
EF_EVENT_RX_RQ_ID(*ev));
Expand All @@ -652,7 +653,7 @@ zf_reactor_purge_event(struct zf_stack* st, int nic, ef_vi* vi, ef_event* ev)
zf_reactor_handle_tx_event(st, nic, vi, ev);
return ZF_REACTOR_PURGE_STATUS_TX;
case EF_EVENT_TYPE_RX_REF:
zf_assert_equal(vi->nic_type.arch, EF_VI_ARCH_EFCT);
zf_assert(*zf_stack_res_nic_flags(st, nic) & ZF_RES_NIC_FLAG_RX_REF);
efct_vi_rxpkt_release(vi, ev->rx_ref.pkt_id);
zf_log_event_trace(st, "%s: purged event %x\n", __FUNCTION__, ev->rx_ref.pkt_id);
return ZF_REACTOR_PURGE_STATUS_RX;
Expand Down
39 changes: 29 additions & 10 deletions src/lib/zf/private/stack_alloc.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ static int zf_stack_init_pio(struct zf_stack_impl* sti, struct zf_attr* attr,
struct zf_stack_res_nic* sti_nic = &sti->nic[nicno];
ef_vi* vi = zf_stack_nic_tx_vi(st_nic);

if( st_nic->vi.nic_type.arch == EF_VI_ARCH_EFCT &&
if( !(*zf_stack_res_nic_flags(st, nicno) & ZF_RES_NIC_FLAG_PIO) &&
attr->pio >= PIO_MUST_USE ) {
zf_log_stack_warn(st,
"PIO not supported by efct interface but pio=%d. "
Expand Down Expand Up @@ -491,13 +491,16 @@ static void zf_stack_init_state(struct zf_stack_impl* sti,
static int zf_stack_init_nic_capabilities(struct zf_stack* st, int nicno)
{
struct zf_stack_impl* sti = ZF_CONTAINER(struct zf_stack_impl, st, st);
unsigned* res_nic_flags = zf_stack_res_nic_flags(st, nicno);
unsigned long vlan_filters;
unsigned long variant;
unsigned long is_rx_ref;
unsigned long is_ctpio_only;
ef_driver_handle dh = sti->nic[nicno].dh;
ef_pd* pd = &sti->nic[nicno].pd;
int rc;

sti->nic[nicno].flags = 0;
*res_nic_flags = 0;

rc = ef_pd_capabilities_get(dh, pd, dh, EF_VI_CAP_RX_FW_VARIANT, &variant);
if( rc != 0 ) {
Expand All @@ -512,7 +515,7 @@ static int zf_stack_init_nic_capabilities(struct zf_stack* st, int nicno)
return -EOPNOTSUPP;
}
if( variant == MC_CMD_GET_CAPABILITIES_OUT_RXDP_LOW_LATENCY )
sti->nic[nicno].flags |= ZF_RES_NIC_FLAG_RX_LL;
*res_nic_flags |= ZF_RES_NIC_FLAG_RX_LL;

rc = ef_pd_capabilities_get(dh, pd, dh, EF_VI_CAP_TX_FW_VARIANT, &variant);
if( rc != 0 ) {
Expand All @@ -527,12 +530,27 @@ static int zf_stack_init_nic_capabilities(struct zf_stack* st, int nicno)
return -EOPNOTSUPP;
}
if( variant == MC_CMD_GET_CAPABILITIES_OUT_TXDP_LOW_LATENCY )
sti->nic[nicno].flags |= ZF_RES_NIC_FLAG_TX_LL;
*res_nic_flags |= ZF_RES_NIC_FLAG_TX_LL;

rc = ef_pd_capabilities_get(dh, pd, dh, EF_VI_CAP_RX_FILTER_TYPE_IP_VLAN,
&vlan_filters);
if( rc == 0 && vlan_filters != 0 )
sti->nic[nicno].flags |= ZF_RES_NIC_FLAG_VLAN_FILTERS;
*res_nic_flags |= ZF_RES_NIC_FLAG_VLAN_FILTERS;

rc = ef_pd_capabilities_get(dh, pd, dh, EF_VI_CAP_RX_REF,
&is_rx_ref);
if( rc == 0 && is_rx_ref != 0 )
*res_nic_flags |= ZF_RES_NIC_FLAG_RX_REF;

rc = ef_pd_capabilities_get(dh, pd, dh, EF_VI_CAP_CTPIO_ONLY,
&is_ctpio_only);
if( rc == 0 && is_ctpio_only != 0 )
*res_nic_flags |= ZF_RES_NIC_FLAG_CTPIO_ONLY;

rc = ef_pd_capabilities_get(dh, pd, dh, EF_VI_CAP_PIO,
&is_ctpio_only);
if( rc == 0 && is_ctpio_only != 0 )
*res_nic_flags |= ZF_RES_NIC_FLAG_PIO;

return 0;
}
Expand All @@ -546,6 +564,7 @@ int zf_stack_init_nic_resources(struct zf_stack_impl* sti,
zf_stack* st = &sti->st;
struct zf_stack_nic* st_nic = &st->nic[nicno];
struct zf_stack_res_nic* sti_nic = &sti->nic[nicno];
unsigned* res_nic_flags = zf_stack_res_nic_flags(st, nicno);
int rc;

/* Open driver. */
Expand Down Expand Up @@ -574,8 +593,8 @@ int zf_stack_init_nic_resources(struct zf_stack_impl* sti,
if( rc < 0 )
goto fail1;

if( !(sti->nic[nicno].flags & ZF_RES_NIC_FLAG_RX_LL) ||
!(sti->nic[nicno].flags & ZF_RES_NIC_FLAG_TX_LL) ) {
if( !(*res_nic_flags & ZF_RES_NIC_FLAG_RX_LL) ||
!(*res_nic_flags & ZF_RES_NIC_FLAG_TX_LL) ) {
zf_log_stack_warn(st, "Interface %s is not in low latency mode.\n",
sti->nic[nicno].if_name);
zf_log_stack_warn(st, "Low latency mode is recommended for best "
Expand Down Expand Up @@ -636,9 +655,9 @@ int zf_stack_init_nic_resources(struct zf_stack_impl* sti,
);

if ( attr->rx_ring_max != 0 ) {
/* For EFCT, we store the timestamp in a fake prefix when copying from
* the shared rx buffer into our own packet buffer. */
if( st_nic->vi.nic_type.arch == EF_VI_ARCH_EFCT )
/* For RX_REF nics, we store the timestamp in a fake prefix when copying
* from the shared rx buffer into our own packet buffer. */
if( *res_nic_flags & ZF_RES_NIC_FLAG_RX_REF )
st_nic->rx_prefix_len = ES_DZ_RX_PREFIX_SIZE;
else
st_nic->rx_prefix_len = ef_vi_receive_prefix_len(&st_nic->vi);
Expand Down
2 changes: 1 addition & 1 deletion src/lib/zf/private/stack_fast.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ zf_stack_handle_rx(struct zf_stack* st, int nic, const char* iov_base,
if( rx_prefix_len ){
ef_vi* vi = &(st->nic[nic].vi);

if( vi->nic_type.arch != EF_VI_ARCH_EFCT ) {
if( *zf_stack_res_nic_flags(st, nic) & ZF_RES_NIC_FLAG_RX_REF ) {
ef_eventq_state* evqs = &(vi->ep_state->evq);

/* To overcome the restrictions that come with using ef_vi's timestamping
Expand Down
2 changes: 1 addition & 1 deletion src/lib/zf/rx.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ zfrr_hw_filter_init_vlan(struct zf_stack* st, int nicno, int proto,
struct zf_stack_impl* sti = ZF_CONTAINER(struct zf_stack_impl, st, st);

if( proto == IPPROTO_UDP && is_multicast(laddr) &&
sti->nic[nicno].flags & ZF_RES_NIC_FLAG_VLAN_FILTERS &&
*zf_stack_res_nic_flags(st, nicno) & ZF_RES_NIC_FLAG_VLAN_FILTERS &&
sti->sti_vlan_id != ZF_NO_VLAN ) {
ef_filter_spec_set_vlan(spec, sti->sti_vlan_id);
}
Expand Down
9 changes: 5 additions & 4 deletions src/lib/zf/stack.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,12 +242,13 @@ int zf_stack_free(struct zf_stack* stack)
unsigned tx_packets = ef_vi_transmit_fill_level(zf_stack_nic_tx_vi(stack, nicno)) - pkts_in_pio;
unsigned rx_packets = ef_vi_receive_fill_level(&stack->nic[nicno].vi);

/* For X3 ef_vi_transmit_fill_level and PIO states are bogus */
if( zf_stack_nic_tx_vi(stack, nicno)->nic_type.arch != EF_VI_ARCH_EFCT )
/* For CTPIO only boards, ef_vi_transmit_fill_level and PIO states are
* bogus */
if( !(*zf_stack_res_nic_flags(stack, nicno) &
ZF_RES_NIC_FLAG_CTPIO_ONLY) ) {
pkts_accounted += tx_packets;

if( stack->nic[nicno].vi.nic_type.arch != EF_VI_ARCH_EFCT )
pkts_accounted += rx_packets;
}

zf_log_stack_trace(stack,
"%s: VI %d: busy pkts=%d vs %d: rx_hw=%d tx_hw=%d "
Expand Down
3 changes: 2 additions & 1 deletion src/lib/zf/tx_warm.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ int enable_tx_warm(struct zf_tx* tx, zf_tx_warm_state* state)
zf_log_stack_trace(st, "%s: TX warm enabled\n", __func__);
char* ctpio_warm_buf = NULL;
state->ctpio_warm_buf_id = PKT_INVALID;
if( vi->vi_flags & EF_VI_TX_CTPIO && vi->nic_type.arch != EF_VI_ARCH_EFCT ) {
if( vi->vi_flags & EF_VI_TX_CTPIO && vi->nic_type.arch != EF_VI_ARCH_EFCT
&& vi->nic_type.arch != EF_VI_ARCH_EF10CT ) {
Comment on lines +19 to +20
Copy link
Collaborator

@sianj-xilinx sianj-xilinx Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is on whether we need to post a dma fallback, which we don't if we have the CTPIO_ONLY capability, so I think that's what we should use.

int rc = zft_alloc_pkt(&st->pool, &state->ctpio_warm_buf_id);
if( rc < 0 )
return rc;
Expand Down
2 changes: 1 addition & 1 deletion src/lib/zf/udp_tx.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ try_clean_tx_completions(struct zf_udp_tx* udp_tx)
int iterations = 0;
#endif

if( vi->nic_type.arch != EF_VI_ARCH_EFCT )
if( *zf_stack_res_nic_flags(st, nicno) & ZF_RES_NIC_FLAG_CTPIO_ONLY )
return false;
for( ; ; ) {
ef_event ev;
Expand Down
3 changes: 2 additions & 1 deletion src/lib/zf/zf_stackdump.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ void dump_nic(SkewPointer<zf_stack_impl> stimpl, int index)
return;

zf_dump("nic%d: vi=%d vi_flags=%x nic_flags=%x intf=%s index=%d hw=%u%c%u\n",
index, ef_vi_instance(vi), ef_vi_flags(vi), stimpl->nic[index].flags,
index, ef_vi_instance(vi), ef_vi_flags(vi),
*zf_stack_res_nic_flags(&stimpl->st, index),
stimpl->nic[index].if_name, stimpl->nic[index].ifindex,
vi->nic_type.arch, vi->nic_type.variant, vi->nic_type.revision);
#if 0
Expand Down
Loading