ON-16435: Replace arch checks with nic capabilies

[ligallag-amd]

ON-16435: Add getter for nic res flags

This makes it easier to change whether these flags are in zf_stack or
zf_stack_impl. Keeping them in the latter struct may have impact on
cache hit rate in the hotpath where these flags may be queried.

ON-16435: Add new ZF_RES_NIC_FLAGS and use them over arch checks

ON-16435: Add ef10ct arch check where capabilitiy is unclear

The final commit is probably not what we want but I am at a bit of a loss for what capabilities should be used in those cases. I therefore added ef10ct arch checks. Opening this as a draft because of this.

I'm also wondering about the performance implications of this change. It queries the flags in the hot path of RX and TX. I wonder if it's therefore worth moving the location of these flags into zf_stack::nic. Or perhaps we should use vi::nic_type::nic_flags instead for the capability checks.

Testing C-Model

zfsink

ZF_ATTR="interface=enp1s0f0" EF_VI_RXQ_SIZE=4096 EF_VI_PD_FLAGS=llct ./zfsink 192.168.135.105:2000
efct_design_parameters: unsupported rx_superbuf_bytes, 0 != 1048576
efct_design_parameters: rx_superbuf_bytes = 0 = 0, treating it as it were the standard value
# pkt-rate  bandwidth(Mbps)      total-pkts
         0                0                0
         0                0                0
         0                0                0
        25                0               25
       398                0              423
---- Some time later (packet rate on C-Model is low) ----
       400                0             5120
       400                0             5520
       400                0             5920
        80                0             6000
         0                0             6000

zfsend

ZF_ATTR="interface=enp1s0f0" EF_VI_PD_FLAGS=llct ./zfsend -u -i 5000 192.168.135.105:2000 230.1.2.3:2000

All packets received at the other end. I had to modify zfsend to sleep 1000us between sends (not sure of the sends are batched) because otherwise the sends would fail and zfsend would hit an assertion. This is fine because the C-Model cannot keep up with the sends.

[sianj-xilinx]

You're using these flags on the critical path. The zf_stack_impl is used for cold path state. If we need the checks on the hot path that information should move to zf_stack_nic. That might just be the RX_REF capability - probably needs a bit of care to ensure that this is done in a way that doesn't impact performance.

[sianj-xilinx]

Assert that CTPIO_ONLY capability wasn't set?

[sianj-xilinx]

The check here is on whether we need to post a dma fallback, which we don't if we have the CTPIO_ONLY capability, so I think that's what we should use.