iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch '200GbE' of git://git.kernel.org/pub/scm/linux/kernel/git/tnguy/next-queue

Browse Source 
Tony Nguyen says:

====================
Introduce Intel IDPF driver

Pavan Kumar Linga says:

This patch series introduces the Intel Infrastructure Data Path Function
(IDPF) driver. It is used for both physical and virtual functions. Except
for some of the device operations the rest of the functionality is the
same for both PF and VF. IDPF uses virtchnl version2 opcodes and
structures defined in the virtchnl2 header file which helps the driver
to learn the capabilities and register offsets from the device
Control Plane (CP) instead of assuming the default values.

The format of the series follows the driver init flow to interface open.
To start with, probe gets called and kicks off the driver initialization
by spawning the 'vc_event_task' work queue which in turn calls the
'hard reset' function. As part of that, the mailbox is initialized which
is used to send/receive the virtchnl messages to/from the CP. Once that is
done, 'core init' kicks in which requests all the required global resources
from the CP and spawns the 'init_task' work queue to create the vports.

Based on the capability information received, the driver creates the said
number of vports (one or many) where each vport is associated to a netdev.
Also, each vport has its own resources such as queues, vectors etc.
From there, rest of the netdev_ops and data path are added.

IDPF implements both single queue which is traditional queueing model
as well as split queue model. In split queue model, it uses separate queue
for both completion descriptors and buffers which helps to implement
out-of-order completions. It also helps to implement asymmetric queues,
for example multiple RX completion queues can be processed by a single
RX buffer queue and multiple TX buffer queues can be processed by a
single TX completion queue. In single queue model, same queue is used
for both descriptor completions as well as buffer completions. It also
supports features such as generic checksum offload, generic receive
offload (hardware GRO) etc.
---
v7:
Patch 2:
 * removed pci_[disable|enable]_pcie_error_reporting as they are dropped
   from the core
Patch 4, 9:
 * used 'kasprintf' instead of 'snprintf' to avoid providing explicit
   character string size which also fixes "-Wformat-truncation" warnings
Patch 14:
 * used 'ethtool_sprintf' instead of 'snprintf' to avoid providing explicit
   character string size which also fixes "-Wformat-truncation" warning
 * add string format argument to the 'ethtool_sprintf' to avoid warning on
   "-Wformat-security"

v6: https://lore.kernel.org/netdev/20230825235954.894050-1-pavan.kumar.linga@intel.com/
Note: 'Acked-by' was only added to patches 1, 2, 12 and not to the other
   patches because of the changes in v6

Patch 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 15:
 * renamed 'reset_lock' to 'vport_ctrl_lock' to reflect the lock usage
 * to avoid defensive programming, used 'vport_ctrl_lock' for the user
   callbacks that access the 'vport' to prevent the hardware reset thread
   from releasing the 'vport', when the user callback is in progress
 * added some variables to netdev private structure to avoid vport access
   if possible from ethtool and ndo callbacks
 * moved 'mac_filter_list_lock' and MAC related flags to vport_config
   structure and refactored mac filter flow to handle asynchronous
   ndo mac filter callbacks
 * stop the queues before starting the reset flow to avoid TX hangs
 * removed 'sw_mutex' and 'stop_mutex' as they are not needed anymore
 * added missing clear bit in 'init_task' error path
 * renamed labels appropriately
Patch 8:
 * replaced page_pool_put_page with page_pool_put_full_page
 * for the page pool max_len, used PAGE_SIZE
Patch 10, 11, 13:
 * made use of the 'netif_txq_maybe_stop', '__netif_txq_completed_wake'
   helper macros
Patch 13:
 * removed IDPF_HR_RESET_IN_PROG flag check in idpf_tx_singleq_start
   as it is defensive
Patch 14:
 * removed max descriptor check as the core does that
 * removed unnecessary error messages
 * removed the stats that are common between the ones reported by ethtool
   and ip link
 * replaced snprintf with ethtool_sprintf
 * added a comment to explain the reason for the max queue check
 * as the netdev queues are set on alloc, there is no need to set
   them again on reset unless there is a queue change, so move the
   'idpf_set_real_num_queues' to 'idpf_initiate_soft_reset'
 Patch 15:
 * reworded the 'configure SRIOV' in the commit message

v5: https://lore.kernel.org/netdev/20230816004305.216136-1-anthony.l.nguyen@intel.com/
Most Patches:
 * wrapped line limit to 80 chars to those which don't effect readability
Patch 12:
 * in skb_add_rx_frag, offset 'headlen' w.r.t page_offset when adding a
   frag to avoid adding the header again
Patch 14:
 * added NULL check for 'rxq' when dereferencing it in page_pool_get_stats

v4: https://lore.kernel.org/netdev/20230808003416.3805142-1-anthony.l.nguyen@intel.com/
Patch 1:
 * s/virtcnl/virtchnl
 * removed the kernel doc for the error code definitions that don't exist
 * reworded the summary part in the virtchnl2 header
Patch 3:
 * don't set local variable to NULL on error
 * renamed sq_send_command_out label with err_unlock
 * don't use __GFP_ZERO in dma_alloc_coherent
Patch 4:
 * introduced mailbox workqueue to process mailbox interrupts
Patch 3, 4, 5, 6, 7, 8, 9, 11, 15:
 * removed unnecessary variable 0-init
Patch 3, 5, 7, 8, 9, 15:
 * removed defensive programming checks wherever applicable
 * removed IDPF_CAP_FIELD_LAST as it can be treated as defensive
   programming
Patch 3, 4, 5, 6, 7:
 * replaced IDPF_DFLT_MBX_BUF_SIZE with IDPF_CTLQ_MAX_BUF_LEN
Patch 2 to 15:
 * add kernel-doc for idpf.h and idpf_txrx.h enums and structures
Patch 4, 5, 15:
 * adjusted the destroy sequence of the workqueues as per the alloc
   sequence
Patch 4, 5, 9, 15:
 * scrub unnecessary flags in 'idpf_flags'
   - IDPF_REMOVE_IN_PROG flag can take care of the cases where
     IDPF_REL_RES_IN_PROG is used, removed the later one
   - IDPF_REQ_[TX|RX]_SPLITQ are replaced with struct variables
   - IDPF_CANCEL_[SERVICE|STATS]_TASK are redundant as the work queue
     doesn't get rescheduled again after 'cancel_delayed_work_sync'
   - IDPF_HR_CORE_RESET is removed as there is no set_bit for this flag
   - IDPF_MB_INTR_TRIGGER is removed as it is not needed anymore with the
     mailbox workqueue implementation
Patch 7 to 15:
 * replaced the custom buffer recycling code with page pool API
 * switched the header split buffer allocations from using a bunch of
   pages to using one large chunk of DMA memory
 * reordered some of the flows in vport_open to support page pool
Patch 8, 12:
 * don't suppress the alloc errors by using __GFP_NOWARN
Patch 9:
 * removed dyn_ctl_clrpba_m as it is not being used
Patch 14:
 * introduced enum idpf_vport_reset_cause instead of using vport flags
 * introduced page pool stats

v3: https://lore.kernel.org/netdev/20230616231341.2885622-1-anthony.l.nguyen@intel.com/
Patch 5:
 * instead of void, used 'struct virtchnl2_create_vport' type for
   vport_params_recvd and vport_params_reqd and removed the typecasting
 * used u16/u32 as needed instead of int for variables which cannot be
   negative and updated in all the places whereever applicable
Patch 6:
 * changed the commit message to "add ptypes and MAC filter support"
 * used the sender Signed-off-by as the last tag on all the patches
 * removed unnecessary variables 0-init
 * instead of fixing the code in this commit, fixed it in the commit
   where the change was introduced first
 * moved get_type_info struct on to the stack instead of memory alloc
 * moved mutex_lock and ptype_info memory alloc outside while loop and
   adjusted the return flow
 * used 'break' instead of 'continue' in ptype id switch case

v2: https://lore.kernel.org/netdev/20230614171428.1504179-1-anthony.l.nguyen@intel.com/
Patch 2:
 * added "Intel(R)" to the DRV_SUMMARY and Makefile.
Patch 4, 5, 6, 15:
 * replaced IDPF_VC_MSG_PENDING flag with mutex 'vc_buf_lock' for the
   adapter related virtchnl opcodes.
 * get the mutex lock in the virtchnl send thread itself instead of
   in receive thread.
Patch 5, 6, 7, 8, 9, 11, 14, 15:
 * replaced IDPF_VPORT_VC_MSG_PENDING flag with mutex 'vc_buf_lock' for
   the vport related virtchnl opcodes.
 * get the mutex lock in the virtchnl send thread itself instead of
   in receive thread.
Patch 6:
 * converted get_ptype_info logic from 1:N to 1:1 message exchange for
   better handling of mutex lock.
Patch 15:
 * introduced 'stats_lock' spinlock to avoid concurrent stats update.

v1: https://lore.kernel.org/netdev/20230530234501.2680230-1-anthony.l.nguyen@intel.com/

====================

Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
David S. Miller 2023-09-16 12:00:56 +01:00
commit b6a7eeb44a
26 changed files with 19191 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Documentation/networking/device_drivers/ethernet/index.rst
View File

@ -32,6 +32,7 @@ Contents:
intel/e1000
intel/e1000e
intel/fm10k
intel/idpf
intel/igb
intel/igbvf
intel/ixgbe

160
Documentation/networking/device_drivers/ethernet/intel/idpf.rst Normal file
View File

@ -0,0 +1,160 @@
.. SPDX-License-Identifier: GPL-2.0+
==========================================================================
idpf Linux* Base Driver for the Intel(R) Infrastructure Data Path Function
==========================================================================
Intel idpf Linux driver.
Copyright(C) 2023 Intel Corporation.
.. contents::
The idpf driver serves as both the Physical Function (PF) and Virtual Function
(VF) driver for the Intel(R) Infrastructure Data Path Function.
Driver information can be obtained using ethtool, lspci, and ip.
For questions related to hardware requirements, refer to the documentation
supplied with your Intel adapter. All hardware requirements listed apply to use
with Linux.
Identifying Your Adapter
========================
For information on how to identify your adapter, and for the latest Intel
network drivers, refer to the Intel Support website:
http://www.intel.com/support
Additional Features and Configurations
======================================
ethtool
-------
The driver utilizes the ethtool interface for driver configuration and
diagnostics, as well as displaying statistical information. The latest ethtool
version is required for this functionality. If you don't have one yet, you can
obtain it at:
https://kernel.org/pub/software/network/ethtool/
Viewing Link Messages
---------------------
Link messages will not be displayed to the console if the distribution is
restricting system messages. In order to see network driver link messages on
your console, set dmesg to eight by entering the following::
# dmesg -n 8
.. note::
This setting is not saved across reboots.
Jumbo Frames
------------
Jumbo Frames support is enabled by changing the Maximum Transmission Unit (MTU)
to a value larger than the default value of 1500.
Use the ip command to increase the MTU size. For example, enter the following
where <ethX> is the interface number::
# ip link set mtu 9000 dev <ethX>
# ip link set up dev <ethX>
.. note::
The maximum MTU setting for jumbo frames is 9706. This corresponds to the
maximum jumbo frame size of 9728 bytes.
.. note::
This driver will attempt to use multiple page sized buffers to receive
each jumbo packet. This should help to avoid buffer starvation issues when
allocating receive packets.
.. note::
Packet loss may have a greater impact on throughput when you use jumbo
frames. If you observe a drop in performance after enabling jumbo frames,
enabling flow control may mitigate the issue.
Performance Optimization
========================
Driver defaults are meant to fit a wide variety of workloads, but if further
optimization is required, we recommend experimenting with the following
settings.
Interrupt Rate Limiting
-----------------------
This driver supports an adaptive interrupt throttle rate (ITR) mechanism that
is tuned for general workloads. The user can customize the interrupt rate
control for specific workloads, via ethtool, adjusting the number of
microseconds between interrupts.
To set the interrupt rate manually, you must disable adaptive mode::
# ethtool -C <ethX> adaptive-rx off adaptive-tx off
For lower CPU utilization:
- Disable adaptive ITR and lower Rx and Tx interrupts. The examples below
affect every queue of the specified interface.
- Setting rx-usecs and tx-usecs to 80 will limit interrupts to about
12,500 interrupts per second per queue::
# ethtool -C <ethX> adaptive-rx off adaptive-tx off rx-usecs 80
tx-usecs 80
For reduced latency:
- Disable adaptive ITR and ITR by setting rx-usecs and tx-usecs to 0
using ethtool::
# ethtool -C <ethX> adaptive-rx off adaptive-tx off rx-usecs 0
tx-usecs 0
Per-queue interrupt rate settings:
- The following examples are for queues 1 and 3, but you can adjust other
queues.
- To disable Rx adaptive ITR and set static Rx ITR to 10 microseconds or
about 100,000 interrupts/second, for queues 1 and 3::
# ethtool --per-queue <ethX> queue_mask 0xa --coalesce adaptive-rx off
rx-usecs 10
- To show the current coalesce settings for queues 1 and 3::
# ethtool --per-queue <ethX> queue_mask 0xa --show-coalesce
Virtualized Environments
------------------------
In addition to the other suggestions in this section, the following may be
helpful to optimize performance in VMs.
- Using the appropriate mechanism (vcpupin) in the VM, pin the CPUs to
individual LCPUs, making sure to use a set of CPUs included in the
device's local_cpulist: /sys/class/net/<ethX>/device/local_cpulist.
- Configure as many Rx/Tx queues in the VM as available. (See the idpf driver
documentation for the number of queues supported.) For example::
# ethtool -L <virt_interface> rx <max> tx <max>
Support
=======
For general information, go to the Intel support website at:
http://www.intel.com/support/
If an issue is identified with the released source code on a supported kernel
with a supported adapter, email the specific information related to the issue
to intel-wired-lan@lists.osuosl.org.
Trademarks
==========
Intel is a trademark or registered trademark of Intel Corporation or its
subsidiaries in the United States and/or other countries.
* Other names and brands may be claimed as the property of others.

12
drivers/net/ethernet/intel/Kconfig
View File

@ -355,5 +355,17 @@ config IGC
To compile this driver as a module, choose M here. The module
will be called igc.
config IDPF
tristate "Intel(R) Infrastructure Data Path Function Support"
depends on PCI_MSI
select DIMLIB
select PAGE_POOL
select PAGE_POOL_STATS
help
This driver supports Intel(R) Infrastructure Data Path Function
devices.
To compile this driver as a module, choose M here. The module
will be called idpf.
endif # NET_VENDOR_INTEL

1
drivers/net/ethernet/intel/Makefile
View File

@ -15,3 +15,4 @@ obj-$(CONFIG_I40E) += i40e/
obj-$(CONFIG_IAVF) += iavf/
obj-$(CONFIG_FM10K) += fm10k/
obj-$(CONFIG_ICE) += ice/
obj-$(CONFIG_IDPF) += idpf/

18
drivers/net/ethernet/intel/idpf/Makefile Normal file
View File

@ -0,0 +1,18 @@
# SPDX-License-Identifier: GPL-2.0-only
# Copyright (C) 2023 Intel Corporation
# Makefile for Intel(R) Infrastructure Data Path Function Linux Driver
obj-$(CONFIG_IDPF) += idpf.o
idpf-y := \
idpf_controlq.o \
idpf_controlq_setup.o \
idpf_dev.o \
idpf_ethtool.o \
idpf_lib.o \
idpf_main.o \
idpf_singleq_txrx.o \
idpf_txrx.o \
idpf_virtchnl.o \
idpf_vf_dev.o

968
drivers/net/ethernet/intel/idpf/idpf.h Normal file
View File

@ -0,0 +1,968 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_H_
#define _IDPF_H_
/* Forward declaration */
struct idpf_adapter;
struct idpf_vport;
struct idpf_vport_max_q;
#include <net/pkt_sched.h>
#include <linux/aer.h>
#include <linux/etherdevice.h>
#include <linux/pci.h>
#include <linux/bitfield.h>
#include <linux/sctp.h>
#include <linux/ethtool.h>
#include <net/gro.h>
#include <linux/dim.h>
#include "virtchnl2.h"
#include "idpf_lan_txrx.h"
#include "idpf_txrx.h"
#include "idpf_controlq.h"
#define GETMAXVAL(num_bits) GENMASK((num_bits) - 1, 0)
#define IDPF_NO_FREE_SLOT 0xffff
/* Default Mailbox settings */
#define IDPF_NUM_FILTERS_PER_MSG 20
#define IDPF_NUM_DFLT_MBX_Q 2 /* includes both TX and RX */
#define IDPF_DFLT_MBX_Q_LEN 64
#define IDPF_DFLT_MBX_ID -1
/* maximum number of times to try before resetting mailbox */
#define IDPF_MB_MAX_ERR 20
#define IDPF_NUM_CHUNKS_PER_MSG(struct_sz, chunk_sz) \
((IDPF_CTLQ_MAX_BUF_LEN - (struct_sz)) / (chunk_sz))
#define IDPF_WAIT_FOR_EVENT_TIMEO_MIN 2000
#define IDPF_WAIT_FOR_EVENT_TIMEO 60000
#define IDPF_MAX_WAIT 500
/* available message levels */
#define IDPF_AVAIL_NETIF_M (NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK)
#define IDPF_DIM_PROFILE_SLOTS 5
#define IDPF_VIRTCHNL_VERSION_MAJOR VIRTCHNL2_VERSION_MAJOR_2
#define IDPF_VIRTCHNL_VERSION_MINOR VIRTCHNL2_VERSION_MINOR_0
/**
* struct idpf_mac_filter
* @list: list member field
* @macaddr: MAC address
* @remove: filter should be removed (virtchnl)
* @add: filter should be added (virtchnl)
*/
struct idpf_mac_filter {
struct list_head list;
u8 macaddr[ETH_ALEN];
bool remove;
bool add;
};
/**
* enum idpf_state - State machine to handle bring up
* @__IDPF_STARTUP: Start the state machine
* @__IDPF_VER_CHECK: Negotiate virtchnl version
* @__IDPF_GET_CAPS: Negotiate capabilities
* @__IDPF_INIT_SW: Init based on given capabilities
* @__IDPF_STATE_LAST: Must be last, used to determine size
*/
enum idpf_state {
__IDPF_STARTUP,
__IDPF_VER_CHECK,
__IDPF_GET_CAPS,
__IDPF_INIT_SW,
__IDPF_STATE_LAST,
};
/**
* enum idpf_flags - Hard reset causes.
* @IDPF_HR_FUNC_RESET: Hard reset when TxRx timeout
* @IDPF_HR_DRV_LOAD: Set on driver load for a clean HW
* @IDPF_HR_RESET_IN_PROG: Reset in progress
* @IDPF_REMOVE_IN_PROG: Driver remove in progress
* @IDPF_MB_INTR_MODE: Mailbox in interrupt mode
* @IDPF_FLAGS_NBITS: Must be last
*/
enum idpf_flags {
IDPF_HR_FUNC_RESET,
IDPF_HR_DRV_LOAD,
IDPF_HR_RESET_IN_PROG,
IDPF_REMOVE_IN_PROG,
IDPF_MB_INTR_MODE,
IDPF_FLAGS_NBITS,
};
/**
* enum idpf_cap_field - Offsets into capabilities struct for specific caps
* @IDPF_BASE_CAPS: generic base capabilities
* @IDPF_CSUM_CAPS: checksum offload capabilities
* @IDPF_SEG_CAPS: segmentation offload capabilities
* @IDPF_RSS_CAPS: RSS offload capabilities
* @IDPF_HSPLIT_CAPS: Header split capabilities
* @IDPF_RSC_CAPS: RSC offload capabilities
* @IDPF_OTHER_CAPS: miscellaneous offloads
*
* Used when checking for a specific capability flag since different capability
* sets are not mutually exclusive numerically, the caller must specify which
* type of capability they are checking for.
*/
enum idpf_cap_field {
IDPF_BASE_CAPS = -1,
IDPF_CSUM_CAPS = offsetof(struct virtchnl2_get_capabilities,
csum_caps),
IDPF_SEG_CAPS = offsetof(struct virtchnl2_get_capabilities,
seg_caps),
IDPF_RSS_CAPS = offsetof(struct virtchnl2_get_capabilities,
rss_caps),
IDPF_HSPLIT_CAPS = offsetof(struct virtchnl2_get_capabilities,
hsplit_caps),
IDPF_RSC_CAPS = offsetof(struct virtchnl2_get_capabilities,
rsc_caps),
IDPF_OTHER_CAPS = offsetof(struct virtchnl2_get_capabilities,
other_caps),
};
/**
* enum idpf_vport_state - Current vport state
* @__IDPF_VPORT_DOWN: Vport is down
* @__IDPF_VPORT_UP: Vport is up
* @__IDPF_VPORT_STATE_LAST: Must be last, number of states
*/
enum idpf_vport_state {
__IDPF_VPORT_DOWN,
__IDPF_VPORT_UP,
__IDPF_VPORT_STATE_LAST,
};
/**
* struct idpf_netdev_priv - Struct to store vport back pointer
* @adapter: Adapter back pointer
* @vport: Vport back pointer
* @vport_id: Vport identifier
* @vport_idx: Relative vport index
* @state: See enum idpf_vport_state
* @netstats: Packet and byte stats
* @stats_lock: Lock to protect stats update
*/
struct idpf_netdev_priv {
struct idpf_adapter *adapter;
struct idpf_vport *vport;
u32 vport_id;
u16 vport_idx;
enum idpf_vport_state state;
struct rtnl_link_stats64 netstats;
spinlock_t stats_lock;
};
/**
* struct idpf_reset_reg - Reset register offsets/masks
* @rstat: Reset status register
* @rstat_m: Reset status mask
*/
struct idpf_reset_reg {
void __iomem *rstat;
u32 rstat_m;
};
/**
* struct idpf_vport_max_q - Queue limits
* @max_rxq: Maximum number of RX queues supported
* @max_txq: Maixmum number of TX queues supported
* @max_bufq: In splitq, maximum number of buffer queues supported
* @max_complq: In splitq, maximum number of completion queues supported
*/
struct idpf_vport_max_q {
u16 max_rxq;
u16 max_txq;
u16 max_bufq;
u16 max_complq;
};
/**
* struct idpf_reg_ops - Device specific register operation function pointers
* @ctlq_reg_init: Mailbox control queue register initialization
* @intr_reg_init: Traffic interrupt register initialization
* @mb_intr_reg_init: Mailbox interrupt register initialization
* @reset_reg_init: Reset register initialization
* @trigger_reset: Trigger a reset to occur
*/
struct idpf_reg_ops {
void (*ctlq_reg_init)(struct idpf_ctlq_create_info *cq);
int (*intr_reg_init)(struct idpf_vport *vport);
void (*mb_intr_reg_init)(struct idpf_adapter *adapter);
void (*reset_reg_init)(struct idpf_adapter *adapter);
void (*trigger_reset)(struct idpf_adapter *adapter,
enum idpf_flags trig_cause);
};
/**
* struct idpf_dev_ops - Device specific operations
* @reg_ops: Register operations
*/
struct idpf_dev_ops {
struct idpf_reg_ops reg_ops;
};
/* These macros allow us to generate an enum and a matching char * array of
* stringified enums that are always in sync. Checkpatch issues a bogus warning
* about this being a complex macro; but it's wrong, these are never used as a
* statement and instead only used to define the enum and array.
*/
#define IDPF_FOREACH_VPORT_VC_STATE(STATE) \
STATE(IDPF_VC_CREATE_VPORT) \
STATE(IDPF_VC_CREATE_VPORT_ERR) \
STATE(IDPF_VC_ENA_VPORT) \
STATE(IDPF_VC_ENA_VPORT_ERR) \
STATE(IDPF_VC_DIS_VPORT) \
STATE(IDPF_VC_DIS_VPORT_ERR) \
STATE(IDPF_VC_DESTROY_VPORT) \
STATE(IDPF_VC_DESTROY_VPORT_ERR) \
STATE(IDPF_VC_CONFIG_TXQ) \
STATE(IDPF_VC_CONFIG_TXQ_ERR) \
STATE(IDPF_VC_CONFIG_RXQ) \
STATE(IDPF_VC_CONFIG_RXQ_ERR) \
STATE(IDPF_VC_ENA_QUEUES) \
STATE(IDPF_VC_ENA_QUEUES_ERR) \
STATE(IDPF_VC_DIS_QUEUES) \
STATE(IDPF_VC_DIS_QUEUES_ERR) \
STATE(IDPF_VC_MAP_IRQ) \
STATE(IDPF_VC_MAP_IRQ_ERR) \
STATE(IDPF_VC_UNMAP_IRQ) \
STATE(IDPF_VC_UNMAP_IRQ_ERR) \
STATE(IDPF_VC_ADD_QUEUES) \
STATE(IDPF_VC_ADD_QUEUES_ERR) \
STATE(IDPF_VC_DEL_QUEUES) \
STATE(IDPF_VC_DEL_QUEUES_ERR) \
STATE(IDPF_VC_ALLOC_VECTORS) \
STATE(IDPF_VC_ALLOC_VECTORS_ERR) \
STATE(IDPF_VC_DEALLOC_VECTORS) \
STATE(IDPF_VC_DEALLOC_VECTORS_ERR) \
STATE(IDPF_VC_SET_SRIOV_VFS) \
STATE(IDPF_VC_SET_SRIOV_VFS_ERR) \
STATE(IDPF_VC_GET_RSS_LUT) \
STATE(IDPF_VC_GET_RSS_LUT_ERR) \
STATE(IDPF_VC_SET_RSS_LUT) \
STATE(IDPF_VC_SET_RSS_LUT_ERR) \
STATE(IDPF_VC_GET_RSS_KEY) \
STATE(IDPF_VC_GET_RSS_KEY_ERR) \
STATE(IDPF_VC_SET_RSS_KEY) \
STATE(IDPF_VC_SET_RSS_KEY_ERR) \
STATE(IDPF_VC_GET_STATS) \
STATE(IDPF_VC_GET_STATS_ERR) \
STATE(IDPF_VC_ADD_MAC_ADDR) \
STATE(IDPF_VC_ADD_MAC_ADDR_ERR) \
STATE(IDPF_VC_DEL_MAC_ADDR) \
STATE(IDPF_VC_DEL_MAC_ADDR_ERR) \
STATE(IDPF_VC_GET_PTYPE_INFO) \
STATE(IDPF_VC_GET_PTYPE_INFO_ERR) \
STATE(IDPF_VC_LOOPBACK_STATE) \
STATE(IDPF_VC_LOOPBACK_STATE_ERR) \
STATE(IDPF_VC_NBITS)
#define IDPF_GEN_ENUM(ENUM) ENUM,
#define IDPF_GEN_STRING(STRING) #STRING,
enum idpf_vport_vc_state {
IDPF_FOREACH_VPORT_VC_STATE(IDPF_GEN_ENUM)
};
extern const char * const idpf_vport_vc_state_str[];
/**
* enum idpf_vport_reset_cause - Vport soft reset causes
* @IDPF_SR_Q_CHANGE: Soft reset queue change
* @IDPF_SR_Q_DESC_CHANGE: Soft reset descriptor change
* @IDPF_SR_MTU_CHANGE: Soft reset MTU change
* @IDPF_SR_RSC_CHANGE: Soft reset RSC change
*/
enum idpf_vport_reset_cause {
IDPF_SR_Q_CHANGE,
IDPF_SR_Q_DESC_CHANGE,
IDPF_SR_MTU_CHANGE,
IDPF_SR_RSC_CHANGE,
};
/**
* enum idpf_vport_flags - Vport flags
* @IDPF_VPORT_DEL_QUEUES: To send delete queues message
* @IDPF_VPORT_SW_MARKER: Indicate TX pipe drain software marker packets
* processing is done
* @IDPF_VPORT_FLAGS_NBITS: Must be last
*/
enum idpf_vport_flags {
IDPF_VPORT_DEL_QUEUES,
IDPF_VPORT_SW_MARKER,
IDPF_VPORT_FLAGS_NBITS,
};
struct idpf_port_stats {
struct u64_stats_sync stats_sync;
u64_stats_t rx_hw_csum_err;
u64_stats_t rx_hsplit;
u64_stats_t rx_hsplit_hbo;
u64_stats_t rx_bad_descs;
u64_stats_t tx_linearize;
u64_stats_t tx_busy;
u64_stats_t tx_drops;
u64_stats_t tx_dma_map_errs;
struct virtchnl2_vport_stats vport_stats;
};
/**
* struct idpf_vport - Handle for netdevices and queue resources
* @num_txq: Number of allocated TX queues
* @num_complq: Number of allocated completion queues
* @txq_desc_count: TX queue descriptor count
* @complq_desc_count: Completion queue descriptor count
* @compln_clean_budget: Work budget for completion clean
* @num_txq_grp: Number of TX queue groups
* @txq_grps: Array of TX queue groups
* @txq_model: Split queue or single queue queuing model
* @txqs: Used only in hotpath to get to the right queue very fast
* @crc_enable: Enable CRC insertion offload
* @num_rxq: Number of allocated RX queues
* @num_bufq: Number of allocated buffer queues
* @rxq_desc_count: RX queue descriptor count. *MUST* have enough descriptors
* to complete all buffer descriptors for all buffer queues in
* the worst case.
* @num_bufqs_per_qgrp: Buffer queues per RX queue in a given grouping
* @bufq_desc_count: Buffer queue descriptor count
* @bufq_size: Size of buffers in ring (e.g. 2K, 4K, etc)
* @num_rxq_grp: Number of RX queues in a group
* @rxq_grps: Total number of RX groups. Number of groups * number of RX per
* group will yield total number of RX queues.
* @rxq_model: Splitq queue or single queue queuing model
* @rx_ptype_lkup: Lookup table for ptypes on RX
* @adapter: back pointer to associated adapter
* @netdev: Associated net_device. Each vport should have one and only one
* associated netdev.
* @flags: See enum idpf_vport_flags
* @vport_type: Default SRIOV, SIOV, etc.
* @vport_id: Device given vport identifier
* @idx: Software index in adapter vports struct
* @default_vport: Use this vport if one isn't specified
* @base_rxd: True if the driver should use base descriptors instead of flex
* @num_q_vectors: Number of IRQ vectors allocated
* @q_vectors: Array of queue vectors
* @q_vector_idxs: Starting index of queue vectors
* @max_mtu: device given max possible MTU
* @default_mac_addr: device will give a default MAC to use
* @rx_itr_profile: RX profiles for Dynamic Interrupt Moderation
* @tx_itr_profile: TX profiles for Dynamic Interrupt Moderation
* @port_stats: per port csum, header split, and other offload stats
* @link_up: True if link is up
* @link_speed_mbps: Link speed in mbps
* @vc_msg: Virtchnl message buffer
* @vc_state: Virtchnl message state
* @vchnl_wq: Wait queue for virtchnl messages
* @sw_marker_wq: workqueue for marker packets
* @vc_buf_lock: Lock to protect virtchnl buffer
*/
struct idpf_vport {
u16 num_txq;
u16 num_complq;
u32 txq_desc_count;
u32 complq_desc_count;
u32 compln_clean_budget;
u16 num_txq_grp;
struct idpf_txq_group *txq_grps;
u32 txq_model;
struct idpf_queue **txqs;
bool crc_enable;
u16 num_rxq;
u16 num_bufq;
u32 rxq_desc_count;
u8 num_bufqs_per_qgrp;
u32 bufq_desc_count[IDPF_MAX_BUFQS_PER_RXQ_GRP];
u32 bufq_size[IDPF_MAX_BUFQS_PER_RXQ_GRP];
u16 num_rxq_grp;
struct idpf_rxq_group *rxq_grps;
u32 rxq_model;
struct idpf_rx_ptype_decoded rx_ptype_lkup[IDPF_RX_MAX_PTYPE];
struct idpf_adapter *adapter;
struct net_device *netdev;
DECLARE_BITMAP(flags, IDPF_VPORT_FLAGS_NBITS);
u16 vport_type;
u32 vport_id;
u16 idx;
bool default_vport;
bool base_rxd;
u16 num_q_vectors;
struct idpf_q_vector *q_vectors;
u16 *q_vector_idxs;
u16 max_mtu;
u8 default_mac_addr[ETH_ALEN];
u16 rx_itr_profile[IDPF_DIM_PROFILE_SLOTS];
u16 tx_itr_profile[IDPF_DIM_PROFILE_SLOTS];
struct idpf_port_stats port_stats;
bool link_up;
u32 link_speed_mbps;
char vc_msg[IDPF_CTLQ_MAX_BUF_LEN];
DECLARE_BITMAP(vc_state, IDPF_VC_NBITS);
wait_queue_head_t vchnl_wq;
wait_queue_head_t sw_marker_wq;
struct mutex vc_buf_lock;
};
/**
* enum idpf_user_flags
* @__IDPF_PROMISC_UC: Unicast promiscuous mode
* @__IDPF_PROMISC_MC: Multicast promiscuous mode
* @__IDPF_USER_FLAGS_NBITS: Must be last
*/
enum idpf_user_flags {
__IDPF_PROMISC_UC = 32,
__IDPF_PROMISC_MC,
__IDPF_USER_FLAGS_NBITS,
};
/**
* struct idpf_rss_data - Associated RSS data
* @rss_key_size: Size of RSS hash key
* @rss_key: RSS hash key
* @rss_lut_size: Size of RSS lookup table
* @rss_lut: RSS lookup table
* @cached_lut: Used to restore previously init RSS lut
*/
struct idpf_rss_data {
u16 rss_key_size;
u8 *rss_key;
u16 rss_lut_size;
u32 *rss_lut;
u32 *cached_lut;
};
/**
* struct idpf_vport_user_config_data - User defined configuration values for
* each vport.
* @rss_data: See struct idpf_rss_data
* @num_req_tx_qs: Number of user requested TX queues through ethtool
* @num_req_rx_qs: Number of user requested RX queues through ethtool
* @num_req_txq_desc: Number of user requested TX queue descriptors through
* ethtool
* @num_req_rxq_desc: Number of user requested RX queue descriptors through
* ethtool
* @user_flags: User toggled config flags
* @mac_filter_list: List of MAC filters
*
* Used to restore configuration after a reset as the vport will get wiped.
*/
struct idpf_vport_user_config_data {
struct idpf_rss_data rss_data;
u16 num_req_tx_qs;
u16 num_req_rx_qs;
u32 num_req_txq_desc;
u32 num_req_rxq_desc;
DECLARE_BITMAP(user_flags, __IDPF_USER_FLAGS_NBITS);
struct list_head mac_filter_list;
};
/**
* enum idpf_vport_config_flags - Vport config flags
* @IDPF_VPORT_REG_NETDEV: Register netdev
* @IDPF_VPORT_UP_REQUESTED: Set if interface up is requested on core reset
* @IDPF_VPORT_ADD_MAC_REQ: Asynchronous add ether address in flight
* @IDPF_VPORT_DEL_MAC_REQ: Asynchronous delete ether address in flight
* @IDPF_VPORT_CONFIG_FLAGS_NBITS: Must be last
*/
enum idpf_vport_config_flags {
IDPF_VPORT_REG_NETDEV,
IDPF_VPORT_UP_REQUESTED,
IDPF_VPORT_ADD_MAC_REQ,
IDPF_VPORT_DEL_MAC_REQ,
IDPF_VPORT_CONFIG_FLAGS_NBITS,
};
/**
* struct idpf_avail_queue_info
* @avail_rxq: Available RX queues
* @avail_txq: Available TX queues
* @avail_bufq: Available buffer queues
* @avail_complq: Available completion queues
*
* Maintain total queues available after allocating max queues to each vport.
*/
struct idpf_avail_queue_info {
u16 avail_rxq;
u16 avail_txq;
u16 avail_bufq;
u16 avail_complq;
};
/**
* struct idpf_vector_info - Utility structure to pass function arguments as a
* structure
* @num_req_vecs: Vectors required based on the number of queues updated by the
* user via ethtool
* @num_curr_vecs: Current number of vectors, must be >= @num_req_vecs
* @index: Relative starting index for vectors
* @default_vport: Vectors are for default vport
*/
struct idpf_vector_info {
u16 num_req_vecs;
u16 num_curr_vecs;
u16 index;
bool default_vport;
};
/**
* struct idpf_vector_lifo - Stack to maintain vector indexes used for vector
* distribution algorithm
* @top: Points to stack top i.e. next available vector index
* @base: Always points to start of the free pool
* @size: Total size of the vector stack
* @vec_idx: Array to store all the vector indexes
*
* Vector stack maintains all the relative vector indexes at the *adapter*
* level. This stack is divided into 2 parts, first one is called as 'default
* pool' and other one is called 'free pool'. Vector distribution algorithm
* gives priority to default vports in a way that at least IDPF_MIN_Q_VEC
* vectors are allocated per default vport and the relative vector indexes for
* those are maintained in default pool. Free pool contains all the unallocated
* vector indexes which can be allocated on-demand basis. Mailbox vector index
* is maintained in the default pool of the stack.
*/
struct idpf_vector_lifo {
u16 top;
u16 base;
u16 size;
u16 *vec_idx;
};
/**
* struct idpf_vport_config - Vport configuration data
* @user_config: see struct idpf_vport_user_config_data
* @max_q: Maximum possible queues
* @req_qs_chunks: Queue chunk data for requested queues
* @mac_filter_list_lock: Lock to protect mac filters
* @flags: See enum idpf_vport_config_flags
*/
struct idpf_vport_config {
struct idpf_vport_user_config_data user_config;
struct idpf_vport_max_q max_q;
void *req_qs_chunks;
spinlock_t mac_filter_list_lock;
DECLARE_BITMAP(flags, IDPF_VPORT_CONFIG_FLAGS_NBITS);
};
/**
* struct idpf_adapter - Device data struct generated on probe
* @pdev: PCI device struct given on probe
* @virt_ver_maj: Virtchnl version major
* @virt_ver_min: Virtchnl version minor
* @msg_enable: Debug message level enabled
* @mb_wait_count: Number of times mailbox was attempted initialization
* @state: Init state machine
* @flags: See enum idpf_flags
* @reset_reg: See struct idpf_reset_reg
* @hw: Device access data
* @num_req_msix: Requested number of MSIX vectors
* @num_avail_msix: Available number of MSIX vectors
* @num_msix_entries: Number of entries in MSIX table
* @msix_entries: MSIX table
* @req_vec_chunks: Requested vector chunk data
* @mb_vector: Mailbox vector data
* @vector_stack: Stack to store the msix vector indexes
* @irq_mb_handler: Handler for hard interrupt for mailbox
* @tx_timeout_count: Number of TX timeouts that have occurred
* @avail_queues: Device given queue limits
* @vports: Array to store vports created by the driver
* @netdevs: Associated Vport netdevs
* @vport_params_reqd: Vport params requested
* @vport_params_recvd: Vport params received
* @vport_ids: Array of device given vport identifiers
* @vport_config: Vport config parameters
* @max_vports: Maximum vports that can be allocated
* @num_alloc_vports: Current number of vports allocated
* @next_vport: Next free slot in pf->vport[] - 0-based!
* @init_task: Initialization task
* @init_wq: Workqueue for initialization task
* @serv_task: Periodically recurring maintenance task
* @serv_wq: Workqueue for service task
* @mbx_task: Task to handle mailbox interrupts
* @mbx_wq: Workqueue for mailbox responses
* @vc_event_task: Task to handle out of band virtchnl event notifications
* @vc_event_wq: Workqueue for virtchnl events
* @stats_task: Periodic statistics retrieval task
* @stats_wq: Workqueue for statistics task
* @caps: Negotiated capabilities with device
* @vchnl_wq: Wait queue for virtchnl messages
* @vc_state: Virtchnl message state
* @vc_msg: Virtchnl message buffer
* @dev_ops: See idpf_dev_ops
* @num_vfs: Number of allocated VFs through sysfs. PF does not directly talk
* to VFs but is used to initialize them
* @crc_enable: Enable CRC insertion offload
* @req_tx_splitq: TX split or single queue model to request
* @req_rx_splitq: RX split or single queue model to request
* @vport_ctrl_lock: Lock to protect the vport control flow
* @vector_lock: Lock to protect vector distribution
* @queue_lock: Lock to protect queue distribution
* @vc_buf_lock: Lock to protect virtchnl buffer
*/
struct idpf_adapter {
struct pci_dev *pdev;
u32 virt_ver_maj;
u32 virt_ver_min;
u32 msg_enable;
u32 mb_wait_count;
enum idpf_state state;
DECLARE_BITMAP(flags, IDPF_FLAGS_NBITS);
struct idpf_reset_reg reset_reg;
struct idpf_hw hw;
u16 num_req_msix;
u16 num_avail_msix;
u16 num_msix_entries;
struct msix_entry *msix_entries;
struct virtchnl2_alloc_vectors *req_vec_chunks;
struct idpf_q_vector mb_vector;
struct idpf_vector_lifo vector_stack;
irqreturn_t (*irq_mb_handler)(int irq, void *data);
u32 tx_timeout_count;
struct idpf_avail_queue_info avail_queues;
struct idpf_vport **vports;
struct net_device **netdevs;
struct virtchnl2_create_vport **vport_params_reqd;
struct virtchnl2_create_vport **vport_params_recvd;
u32 *vport_ids;
struct idpf_vport_config **vport_config;
u16 max_vports;
u16 num_alloc_vports;
u16 next_vport;
struct delayed_work init_task;
struct workqueue_struct *init_wq;
struct delayed_work serv_task;
struct workqueue_struct *serv_wq;
struct delayed_work mbx_task;
struct workqueue_struct *mbx_wq;
struct delayed_work vc_event_task;
struct workqueue_struct *vc_event_wq;
struct delayed_work stats_task;
struct workqueue_struct *stats_wq;
struct virtchnl2_get_capabilities caps;
wait_queue_head_t vchnl_wq;
DECLARE_BITMAP(vc_state, IDPF_VC_NBITS);
char vc_msg[IDPF_CTLQ_MAX_BUF_LEN];
struct idpf_dev_ops dev_ops;
int num_vfs;
bool crc_enable;
bool req_tx_splitq;
bool req_rx_splitq;
struct mutex vport_ctrl_lock;
struct mutex vector_lock;
struct mutex queue_lock;
struct mutex vc_buf_lock;
};
/**
* idpf_is_queue_model_split - check if queue model is split
* @q_model: queue model single or split
*
* Returns true if queue model is split else false
*/
static inline int idpf_is_queue_model_split(u16 q_model)
{
return q_model == VIRTCHNL2_QUEUE_MODEL_SPLIT;
}
#define idpf_is_cap_ena(adapter, field, flag) \
idpf_is_capability_ena(adapter, false, field, flag)
#define idpf_is_cap_ena_all(adapter, field, flag) \
idpf_is_capability_ena(adapter, true, field, flag)
bool idpf_is_capability_ena(struct idpf_adapter *adapter, bool all,
enum idpf_cap_field field, u64 flag);
#define IDPF_CAP_RSS (\
VIRTCHNL2_CAP_RSS_IPV4_TCP |\
VIRTCHNL2_CAP_RSS_IPV4_TCP |\
VIRTCHNL2_CAP_RSS_IPV4_UDP |\
VIRTCHNL2_CAP_RSS_IPV4_SCTP |\
VIRTCHNL2_CAP_RSS_IPV4_OTHER |\
VIRTCHNL2_CAP_RSS_IPV6_TCP |\
VIRTCHNL2_CAP_RSS_IPV6_TCP |\
VIRTCHNL2_CAP_RSS_IPV6_UDP |\
VIRTCHNL2_CAP_RSS_IPV6_SCTP |\
VIRTCHNL2_CAP_RSS_IPV6_OTHER)
#define IDPF_CAP_RSC (\
VIRTCHNL2_CAP_RSC_IPV4_TCP |\
VIRTCHNL2_CAP_RSC_IPV6_TCP)
#define IDPF_CAP_HSPLIT (\
VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |\
VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6)
#define IDPF_CAP_RX_CSUM_L4V4 (\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP)
#define IDPF_CAP_RX_CSUM_L4V6 (\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP)
#define IDPF_CAP_RX_CSUM (\
VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP)
#define IDPF_CAP_SCTP_CSUM (\
VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |\
VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |\
VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP)
#define IDPF_CAP_TUNNEL_TX_CSUM (\
VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL |\
VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL)
/**
* idpf_get_reserved_vecs - Get reserved vectors
* @adapter: private data struct
*/
static inline u16 idpf_get_reserved_vecs(struct idpf_adapter *adapter)
{
return le16_to_cpu(adapter->caps.num_allocated_vectors);
}
/**
* idpf_get_default_vports - Get default number of vports
* @adapter: private data struct
*/
static inline u16 idpf_get_default_vports(struct idpf_adapter *adapter)
{
return le16_to_cpu(adapter->caps.default_num_vports);
}
/**
* idpf_get_max_vports - Get max number of vports
* @adapter: private data struct
*/
static inline u16 idpf_get_max_vports(struct idpf_adapter *adapter)
{
return le16_to_cpu(adapter->caps.max_vports);
}
/**
* idpf_get_max_tx_bufs - Get max scatter-gather buffers supported by the device
* @adapter: private data struct
*/
static inline unsigned int idpf_get_max_tx_bufs(struct idpf_adapter *adapter)
{
return adapter->caps.max_sg_bufs_per_tx_pkt;
}
/**
* idpf_get_min_tx_pkt_len - Get min packet length supported by the device
* @adapter: private data struct
*/
static inline u8 idpf_get_min_tx_pkt_len(struct idpf_adapter *adapter)
{
u8 pkt_len = adapter->caps.min_sso_packet_len;
return pkt_len ? pkt_len : IDPF_TX_MIN_PKT_LEN;
}
/**
* idpf_get_reg_addr - Get BAR0 register address
* @adapter: private data struct
* @reg_offset: register offset value
*
* Based on the register offset, return the actual BAR0 register address
*/
static inline void __iomem *idpf_get_reg_addr(struct idpf_adapter *adapter,
resource_size_t reg_offset)
{
return (void __iomem *)(adapter->hw.hw_addr + reg_offset);
}
/**
* idpf_is_reset_detected - check if we were reset at some point
* @adapter: driver specific private structure
*
* Returns true if we are either in reset currently or were previously reset.
*/
static inline bool idpf_is_reset_detected(struct idpf_adapter *adapter)
{
if (!adapter->hw.arq)
return true;
return !(readl(idpf_get_reg_addr(adapter, adapter->hw.arq->reg.len)) &
adapter->hw.arq->reg.len_mask);
}
/**
* idpf_is_reset_in_prog - check if reset is in progress
* @adapter: driver specific private structure
*
* Returns true if hard reset is in progress, false otherwise
*/
static inline bool idpf_is_reset_in_prog(struct idpf_adapter *adapter)
{
return (test_bit(IDPF_HR_RESET_IN_PROG, adapter->flags) ||
test_bit(IDPF_HR_FUNC_RESET, adapter->flags) ||
test_bit(IDPF_HR_DRV_LOAD, adapter->flags));
}
/**
* idpf_netdev_to_vport - get a vport handle from a netdev
* @netdev: network interface device structure
*/
static inline struct idpf_vport *idpf_netdev_to_vport(struct net_device *netdev)
{
struct idpf_netdev_priv *np = netdev_priv(netdev);
return np->vport;
}
/**
* idpf_netdev_to_adapter - Get adapter handle from a netdev
* @netdev: Network interface device structure
*/
static inline struct idpf_adapter *idpf_netdev_to_adapter(struct net_device *netdev)
{
struct idpf_netdev_priv *np = netdev_priv(netdev);
return np->adapter;
}
/**
* idpf_is_feature_ena - Determine if a particular feature is enabled
* @vport: Vport to check
* @feature: Netdev flag to check
*
* Returns true or false if a particular feature is enabled.
*/
static inline bool idpf_is_feature_ena(const struct idpf_vport *vport,
netdev_features_t feature)
{
return vport->netdev->features & feature;
}
/**
* idpf_get_max_tx_hdr_size -- get the size of tx header
* @adapter: Driver specific private structure
*/
static inline u16 idpf_get_max_tx_hdr_size(struct idpf_adapter *adapter)
{
return le16_to_cpu(adapter->caps.max_tx_hdr_size);
}
/**
* idpf_vport_ctrl_lock - Acquire the vport control lock
* @netdev: Network interface device structure
*
* This lock should be used by non-datapath code to protect against vport
* destruction.
*/
static inline void idpf_vport_ctrl_lock(struct net_device *netdev)
{
struct idpf_netdev_priv *np = netdev_priv(netdev);
mutex_lock(&np->adapter->vport_ctrl_lock);
}
/**
* idpf_vport_ctrl_unlock - Release the vport control lock
* @netdev: Network interface device structure
*/
static inline void idpf_vport_ctrl_unlock(struct net_device *netdev)
{
struct idpf_netdev_priv *np = netdev_priv(netdev);
mutex_unlock(&np->adapter->vport_ctrl_lock);
}
void idpf_statistics_task(struct work_struct *work);
void idpf_init_task(struct work_struct *work);
void idpf_service_task(struct work_struct *work);
void idpf_mbx_task(struct work_struct *work);
void idpf_vc_event_task(struct work_struct *work);
void idpf_dev_ops_init(struct idpf_adapter *adapter);
void idpf_vf_dev_ops_init(struct idpf_adapter *adapter);
int idpf_vport_adjust_qs(struct idpf_vport *vport);
int idpf_init_dflt_mbx(struct idpf_adapter *adapter);
void idpf_deinit_dflt_mbx(struct idpf_adapter *adapter);
int idpf_vc_core_init(struct idpf_adapter *adapter);
void idpf_vc_core_deinit(struct idpf_adapter *adapter);
int idpf_intr_req(struct idpf_adapter *adapter);
void idpf_intr_rel(struct idpf_adapter *adapter);
int idpf_get_reg_intr_vecs(struct idpf_vport *vport,
struct idpf_vec_regs *reg_vals);
u16 idpf_get_max_tx_hdr_size(struct idpf_adapter *adapter);
int idpf_send_delete_queues_msg(struct idpf_vport *vport);
int idpf_send_add_queues_msg(const struct idpf_vport *vport, u16 num_tx_q,
u16 num_complq, u16 num_rx_q, u16 num_rx_bufq);
int idpf_initiate_soft_reset(struct idpf_vport *vport,
enum idpf_vport_reset_cause reset_cause);
int idpf_send_enable_vport_msg(struct idpf_vport *vport);
int idpf_send_disable_vport_msg(struct idpf_vport *vport);
int idpf_send_destroy_vport_msg(struct idpf_vport *vport);
int idpf_send_get_rx_ptype_msg(struct idpf_vport *vport);
int idpf_send_ena_dis_loopback_msg(struct idpf_vport *vport);
int idpf_send_get_set_rss_key_msg(struct idpf_vport *vport, bool get);
int idpf_send_get_set_rss_lut_msg(struct idpf_vport *vport, bool get);
int idpf_send_dealloc_vectors_msg(struct idpf_adapter *adapter);
int idpf_send_alloc_vectors_msg(struct idpf_adapter *adapter, u16 num_vectors);
void idpf_deinit_task(struct idpf_adapter *adapter);
int idpf_req_rel_vector_indexes(struct idpf_adapter *adapter,
u16 *q_vector_idxs,
struct idpf_vector_info *vec_info);
int idpf_vport_alloc_vec_indexes(struct idpf_vport *vport);
int idpf_send_get_stats_msg(struct idpf_vport *vport);
int idpf_get_vec_ids(struct idpf_adapter *adapter,
u16 *vecids, int num_vecids,
struct virtchnl2_vector_chunks *chunks);
int idpf_recv_mb_msg(struct idpf_adapter *adapter, u32 op,
void *msg, int msg_size);
int idpf_send_mb_msg(struct idpf_adapter *adapter, u32 op,
u16 msg_size, u8 *msg);
void idpf_set_ethtool_ops(struct net_device *netdev);
int idpf_vport_alloc_max_qs(struct idpf_adapter *adapter,
struct idpf_vport_max_q *max_q);
void idpf_vport_dealloc_max_qs(struct idpf_adapter *adapter,
struct idpf_vport_max_q *max_q);
int idpf_add_del_mac_filters(struct idpf_vport *vport,
struct idpf_netdev_priv *np,
bool add, bool async);
int idpf_set_promiscuous(struct idpf_adapter *adapter,
struct idpf_vport_user_config_data *config_data,
u32 vport_id);
int idpf_send_disable_queues_msg(struct idpf_vport *vport);
void idpf_vport_init(struct idpf_vport *vport, struct idpf_vport_max_q *max_q);
u32 idpf_get_vport_id(struct idpf_vport *vport);
int idpf_vport_queue_ids_init(struct idpf_vport *vport);
int idpf_queue_reg_init(struct idpf_vport *vport);
int idpf_send_config_queues_msg(struct idpf_vport *vport);
int idpf_send_enable_queues_msg(struct idpf_vport *vport);
int idpf_send_create_vport_msg(struct idpf_adapter *adapter,
struct idpf_vport_max_q *max_q);
int idpf_check_supported_desc_ids(struct idpf_vport *vport);
void idpf_vport_intr_write_itr(struct idpf_q_vector *q_vector,
u16 itr, bool tx);
int idpf_send_map_unmap_queue_vector_msg(struct idpf_vport *vport, bool map);
int idpf_send_set_sriov_vfs_msg(struct idpf_adapter *adapter, u16 num_vfs);
int idpf_sriov_configure(struct pci_dev *pdev, int num_vfs);
#endif /* !_IDPF_H_ */

621
drivers/net/ethernet/intel/idpf/idpf_controlq.c Normal file
View File

@ -0,0 +1,621 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
#include "idpf_controlq.h"
/**
* idpf_ctlq_setup_regs - initialize control queue registers
* @cq: pointer to the specific control queue
* @q_create_info: structs containing info for each queue to be initialized
*/
static void idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
struct idpf_ctlq_create_info *q_create_info)
{
/* set control queue registers in our local struct */
cq->reg.head = q_create_info->reg.head;
cq->reg.tail = q_create_info->reg.tail;
cq->reg.len = q_create_info->reg.len;
cq->reg.bah = q_create_info->reg.bah;
cq->reg.bal = q_create_info->reg.bal;
cq->reg.len_mask = q_create_info->reg.len_mask;
cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
cq->reg.head_mask = q_create_info->reg.head_mask;
}
/**
* idpf_ctlq_init_regs - Initialize control queue registers
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
* @is_rxq: true if receive control queue, false otherwise
*
* Initialize registers. The caller is expected to have already initialized the
* descriptor ring memory and buffer memory
*/
static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
bool is_rxq)
{
/* Update tail to post pre-allocated buffers for rx queues */
if (is_rxq)
wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
/* For non-Mailbox control queues only TAIL need to be set */
if (cq->q_id != -1)
return;
/* Clear Head for both send or receive */
wr32(hw, cq->reg.head, 0);
/* set starting point */
wr32(hw, cq->reg.bal, lower_32_bits(cq->desc_ring.pa));
wr32(hw, cq->reg.bah, upper_32_bits(cq->desc_ring.pa));
wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
}
/**
* idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
* @cq: pointer to the specific Control queue
*
* Record the address of the receive queue DMA buffers in the descriptors.
* The buffers must have been previously allocated.
*/
static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
{
int i;
for (i = 0; i < cq->ring_size; i++) {
struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
/* No buffer to post to descriptor, continue */
if (!bi)
continue;
desc->flags =
cpu_to_le16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
desc->opcode = 0;
desc->datalen = cpu_to_le16(bi->size);
desc->ret_val = 0;
desc->v_opcode_dtype = 0;
desc->v_retval = 0;
desc->params.indirect.addr_high =
cpu_to_le32(upper_32_bits(bi->pa));
desc->params.indirect.addr_low =
cpu_to_le32(lower_32_bits(bi->pa));
desc->params.indirect.param0 = 0;
desc->params.indirect.sw_cookie = 0;
desc->params.indirect.v_flags = 0;
}
}
/**
* idpf_ctlq_shutdown - shutdown the CQ
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
*
* The main shutdown routine for any controq queue
*/
static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
{
mutex_lock(&cq->cq_lock);
/* free ring buffers and the ring itself */
idpf_ctlq_dealloc_ring_res(hw, cq);
/* Set ring_size to 0 to indicate uninitialized queue */
cq->ring_size = 0;
mutex_unlock(&cq->cq_lock);
mutex_destroy(&cq->cq_lock);
}
/**
* idpf_ctlq_add - add one control queue
* @hw: pointer to hardware struct
* @qinfo: info for queue to be created
* @cq_out: (output) double pointer to control queue to be created
*
* Allocate and initialize a control queue and add it to the control queue list.
* The cq parameter will be allocated/initialized and passed back to the caller
* if no errors occur.
*
* Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
*/
int idpf_ctlq_add(struct idpf_hw *hw,
struct idpf_ctlq_create_info *qinfo,
struct idpf_ctlq_info **cq_out)
{
struct idpf_ctlq_info *cq;
bool is_rxq = false;
int err;
cq = kzalloc(sizeof(*cq), GFP_KERNEL);
if (!cq)
return -ENOMEM;
cq->cq_type = qinfo->type;
cq->q_id = qinfo->id;
cq->buf_size = qinfo->buf_size;
cq->ring_size = qinfo->len;
cq->next_to_use = 0;
cq->next_to_clean = 0;
cq->next_to_post = cq->ring_size - 1;
switch (qinfo->type) {
case IDPF_CTLQ_TYPE_MAILBOX_RX:
is_rxq = true;
fallthrough;
case IDPF_CTLQ_TYPE_MAILBOX_TX:
err = idpf_ctlq_alloc_ring_res(hw, cq);
break;
default:
err = -EBADR;
break;
}
if (err)
goto init_free_q;
if (is_rxq) {
idpf_ctlq_init_rxq_bufs(cq);
} else {
/* Allocate the array of msg pointers for TX queues */
cq->bi.tx_msg = kcalloc(qinfo->len,
sizeof(struct idpf_ctlq_msg *),
GFP_KERNEL);
if (!cq->bi.tx_msg) {
err = -ENOMEM;
goto init_dealloc_q_mem;
}
}
idpf_ctlq_setup_regs(cq, qinfo);
idpf_ctlq_init_regs(hw, cq, is_rxq);
mutex_init(&cq->cq_lock);
list_add(&cq->cq_list, &hw->cq_list_head);
*cq_out = cq;
return 0;
init_dealloc_q_mem:
/* free ring buffers and the ring itself */
idpf_ctlq_dealloc_ring_res(hw, cq);
init_free_q:
kfree(cq);
return err;
}
/**
* idpf_ctlq_remove - deallocate and remove specified control queue
* @hw: pointer to hardware struct
* @cq: pointer to control queue to be removed
*/
void idpf_ctlq_remove(struct idpf_hw *hw,
struct idpf_ctlq_info *cq)
{
list_del(&cq->cq_list);
idpf_ctlq_shutdown(hw, cq);
kfree(cq);
}
/**
* idpf_ctlq_init - main initialization routine for all control queues
* @hw: pointer to hardware struct
* @num_q: number of queues to initialize
* @q_info: array of structs containing info for each queue to be initialized
*
* This initializes any number and any type of control queues. This is an all
* or nothing routine; if one fails, all previously allocated queues will be
* destroyed. This must be called prior to using the individual add/remove
* APIs.
*/
int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
struct idpf_ctlq_create_info *q_info)
{
struct idpf_ctlq_info *cq, *tmp;
int err;
int i;
INIT_LIST_HEAD(&hw->cq_list_head);
for (i = 0; i < num_q; i++) {
struct idpf_ctlq_create_info *qinfo = q_info + i;
err = idpf_ctlq_add(hw, qinfo, &cq);
if (err)
goto init_destroy_qs;
}
return 0;
init_destroy_qs:
list_for_each_entry_safe(cq, tmp, &hw->cq_list_head, cq_list)
idpf_ctlq_remove(hw, cq);
return err;
}
/**
* idpf_ctlq_deinit - destroy all control queues
* @hw: pointer to hw struct
*/
void idpf_ctlq_deinit(struct idpf_hw *hw)
{
struct idpf_ctlq_info *cq, *tmp;
list_for_each_entry_safe(cq, tmp, &hw->cq_list_head, cq_list)
idpf_ctlq_remove(hw, cq);
}
/**
* idpf_ctlq_send - send command to Control Queue (CTQ)
* @hw: pointer to hw struct
* @cq: handle to control queue struct to send on
* @num_q_msg: number of messages to send on control queue
* @q_msg: pointer to array of queue messages to be sent
*
* The caller is expected to allocate DMAable buffers and pass them to the
* send routine via the q_msg struct / control queue specific data struct.
* The control queue will hold a reference to each send message until
* the completion for that message has been cleaned.
*/
int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
{
struct idpf_ctlq_desc *desc;
int num_desc_avail;
int err = 0;
int i;
mutex_lock(&cq->cq_lock);
/* Ensure there are enough descriptors to send all messages */
num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
err = -ENOSPC;
goto err_unlock;
}
for (i = 0; i < num_q_msg; i++) {
struct idpf_ctlq_msg *msg = &q_msg[i];
desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
desc->opcode = cpu_to_le16(msg->opcode);
desc->pfid_vfid = cpu_to_le16(msg->func_id);
desc->v_opcode_dtype = cpu_to_le32(msg->cookie.mbx.chnl_opcode);
desc->v_retval = cpu_to_le32(msg->cookie.mbx.chnl_retval);
desc->flags = cpu_to_le16((msg->host_id & IDPF_HOST_ID_MASK) <<
IDPF_CTLQ_FLAG_HOST_ID_S);
if (msg->data_len) {
struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
desc->datalen |= cpu_to_le16(msg->data_len);
desc->flags |= cpu_to_le16(IDPF_CTLQ_FLAG_BUF);
desc->flags |= cpu_to_le16(IDPF_CTLQ_FLAG_RD);
/* Update the address values in the desc with the pa
* value for respective buffer
*/
desc->params.indirect.addr_high =
cpu_to_le32(upper_32_bits(buff->pa));
desc->params.indirect.addr_low =
cpu_to_le32(lower_32_bits(buff->pa));
memcpy(&desc->params, msg->ctx.indirect.context,
IDPF_INDIRECT_CTX_SIZE);
} else {
memcpy(&desc->params, msg->ctx.direct,
IDPF_DIRECT_CTX_SIZE);
}
/* Store buffer info */
cq->bi.tx_msg[cq->next_to_use] = msg;
(cq->next_to_use)++;
if (cq->next_to_use == cq->ring_size)
cq->next_to_use = 0;
}
/* Force memory write to complete before letting hardware
* know that there are new descriptors to fetch.
*/
dma_wmb();
wr32(hw, cq->reg.tail, cq->next_to_use);
err_unlock:
mutex_unlock(&cq->cq_lock);
return err;
}
/**
* idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
* requested queue
* @cq: pointer to the specific Control queue
* @clean_count: (input|output) number of descriptors to clean as input, and
* number of descriptors actually cleaned as output
* @msg_status: (output) pointer to msg pointer array to be populated; needs
* to be allocated by caller
*
* Returns an array of message pointers associated with the cleaned
* descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
* descriptors. The status will be returned for each; any messages that failed
* to send will have a non-zero status. The caller is expected to free original
* ctlq_msgs and free or reuse the DMA buffers.
*/
int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
struct idpf_ctlq_msg *msg_status[])
{
struct idpf_ctlq_desc *desc;
u16 i, num_to_clean;
u16 ntc, desc_err;
if (*clean_count == 0)
return 0;
if (*clean_count > cq->ring_size)
return -EBADR;
mutex_lock(&cq->cq_lock);
ntc = cq->next_to_clean;
num_to_clean = *clean_count;
for (i = 0; i < num_to_clean; i++) {
/* Fetch next descriptor and check if marked as done */
desc = IDPF_CTLQ_DESC(cq, ntc);
if (!(le16_to_cpu(desc->flags) & IDPF_CTLQ_FLAG_DD))
break;
/* strip off FW internal code */
desc_err = le16_to_cpu(desc->ret_val) & 0xff;
msg_status[i] = cq->bi.tx_msg[ntc];
msg_status[i]->status = desc_err;
cq->bi.tx_msg[ntc] = NULL;
/* Zero out any stale data */
memset(desc, 0, sizeof(*desc));
ntc++;
if (ntc == cq->ring_size)
ntc = 0;
}
cq->next_to_clean = ntc;
mutex_unlock(&cq->cq_lock);
/* Return number of descriptors actually cleaned */
*clean_count = i;
return 0;
}
/**
* idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
* @hw: pointer to hw struct
* @cq: pointer to control queue handle
* @buff_count: (input|output) input is number of buffers caller is trying to
* return; output is number of buffers that were not posted
* @buffs: array of pointers to dma mem structs to be given to hardware
*
* Caller uses this function to return DMA buffers to the descriptor ring after
* consuming them; buff_count will be the number of buffers.
*
* Note: this function needs to be called after a receive call even
* if there are no DMA buffers to be returned, i.e. buff_count = 0,
* buffs = NULL to support direct commands
*/
int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
u16 *buff_count, struct idpf_dma_mem **buffs)
{
struct idpf_ctlq_desc *desc;
u16 ntp = cq->next_to_post;
bool buffs_avail = false;
u16 tbp = ntp + 1;
int i = 0;
if (*buff_count > cq->ring_size)
return -EBADR;
if (*buff_count > 0)
buffs_avail = true;
mutex_lock(&cq->cq_lock);
if (tbp >= cq->ring_size)
tbp = 0;
if (tbp == cq->next_to_clean)
/* Nothing to do */
goto post_buffs_out;
/* Post buffers for as many as provided or up until the last one used */
while (ntp != cq->next_to_clean) {
desc = IDPF_CTLQ_DESC(cq, ntp);
if (cq->bi.rx_buff[ntp])
goto fill_desc;
if (!buffs_avail) {
/* If the caller hasn't given us any buffers or
* there are none left, search the ring itself
* for an available buffer to move to this
* entry starting at the next entry in the ring
*/
tbp = ntp + 1;
/* Wrap ring if necessary */
if (tbp >= cq->ring_size)
tbp = 0;
while (tbp != cq->next_to_clean) {
if (cq->bi.rx_buff[tbp]) {
cq->bi.rx_buff[ntp] =
cq->bi.rx_buff[tbp];
cq->bi.rx_buff[tbp] = NULL;
/* Found a buffer, no need to
* search anymore
*/
break;
}
/* Wrap ring if necessary */
tbp++;
if (tbp >= cq->ring_size)
tbp = 0;
}
if (tbp == cq->next_to_clean)
goto post_buffs_out;
} else {
/* Give back pointer to DMA buffer */
cq->bi.rx_buff[ntp] = buffs[i];
i++;
if (i >= *buff_count)
buffs_avail = false;
}
fill_desc:
desc->flags =
cpu_to_le16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
/* Post buffers to descriptor */
desc->datalen = cpu_to_le16(cq->bi.rx_buff[ntp]->size);
desc->params.indirect.addr_high =
cpu_to_le32(upper_32_bits(cq->bi.rx_buff[ntp]->pa));
desc->params.indirect.addr_low =
cpu_to_le32(lower_32_bits(cq->bi.rx_buff[ntp]->pa));
ntp++;
if (ntp == cq->ring_size)
ntp = 0;
}
post_buffs_out:
/* Only update tail if buffers were actually posted */
if (cq->next_to_post != ntp) {
if (ntp)
/* Update next_to_post to ntp - 1 since current ntp
* will not have a buffer
*/
cq->next_to_post = ntp - 1;
else
/* Wrap to end of end ring since current ntp is 0 */
cq->next_to_post = cq->ring_size - 1;
wr32(hw, cq->reg.tail, cq->next_to_post);
}
mutex_unlock(&cq->cq_lock);
/* return the number of buffers that were not posted */
*buff_count = *buff_count - i;
return 0;
}
/**
* idpf_ctlq_recv - receive control queue message call back
* @cq: pointer to control queue handle to receive on
* @num_q_msg: (input|output) input number of messages that should be received;
* output number of messages actually received
* @q_msg: (output) array of received control queue messages on this q;
* needs to be pre-allocated by caller for as many messages as requested
*
* Called by interrupt handler or polling mechanism. Caller is expected
* to free buffers
*/
int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
struct idpf_ctlq_msg *q_msg)
{
u16 num_to_clean, ntc, flags;
struct idpf_ctlq_desc *desc;
int err = 0;
u16 i;
if (*num_q_msg == 0)
return 0;
else if (*num_q_msg > cq->ring_size)
return -EBADR;
/* take the lock before we start messing with the ring */
mutex_lock(&cq->cq_lock);
ntc = cq->next_to_clean;
num_to_clean = *num_q_msg;
for (i = 0; i < num_to_clean; i++) {
/* Fetch next descriptor and check if marked as done */
desc = IDPF_CTLQ_DESC(cq, ntc);
flags = le16_to_cpu(desc->flags);
if (!(flags & IDPF_CTLQ_FLAG_DD))
break;
q_msg[i].vmvf_type = (flags &
(IDPF_CTLQ_FLAG_FTYPE_VM |
IDPF_CTLQ_FLAG_FTYPE_PF)) >>
IDPF_CTLQ_FLAG_FTYPE_S;
if (flags & IDPF_CTLQ_FLAG_ERR)
err = -EBADMSG;
q_msg[i].cookie.mbx.chnl_opcode =
le32_to_cpu(desc->v_opcode_dtype);
q_msg[i].cookie.mbx.chnl_retval =
le32_to_cpu(desc->v_retval);
q_msg[i].opcode = le16_to_cpu(desc->opcode);
q_msg[i].data_len = le16_to_cpu(desc->datalen);
q_msg[i].status = le16_to_cpu(desc->ret_val);
if (desc->datalen) {
memcpy(q_msg[i].ctx.indirect.context,
&desc->params.indirect, IDPF_INDIRECT_CTX_SIZE);
/* Assign pointer to dma buffer to ctlq_msg array
* to be given to upper layer
*/
q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
/* Zero out pointer to DMA buffer info;
* will be repopulated by post buffers API
*/
cq->bi.rx_buff[ntc] = NULL;
} else {
memcpy(q_msg[i].ctx.direct, desc->params.raw,
IDPF_DIRECT_CTX_SIZE);
}
/* Zero out stale data in descriptor */
memset(desc, 0, sizeof(struct idpf_ctlq_desc));
ntc++;
if (ntc == cq->ring_size)
ntc = 0;
}
cq->next_to_clean = ntc;
mutex_unlock(&cq->cq_lock);
*num_q_msg = i;
if (*num_q_msg == 0)
err = -ENOMSG;
return err;
}

130
drivers/net/ethernet/intel/idpf/idpf_controlq.h Normal file
View File

@ -0,0 +1,130 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_CONTROLQ_H_
#define _IDPF_CONTROLQ_H_
#include <linux/slab.h>
#include "idpf_controlq_api.h"
/* Maximum buffer length for all control queue types */
#define IDPF_CTLQ_MAX_BUF_LEN 4096
#define IDPF_CTLQ_DESC(R, i) \
(&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
#define IDPF_CTLQ_DESC_UNUSED(R) \
((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
(R)->next_to_clean - (R)->next_to_use - 1))
/* Control Queue default settings */
#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
struct idpf_ctlq_desc {
/* Control queue descriptor flags */
__le16 flags;
/* Control queue message opcode */
__le16 opcode;
__le16 datalen; /* 0 for direct commands */
union {
__le16 ret_val;
__le16 pfid_vfid;
#define IDPF_CTLQ_DESC_VF_ID_S 0
#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
#define IDPF_CTLQ_DESC_PF_ID_S 11
#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
};
/* Virtchnl message opcode and virtchnl descriptor type
* v_opcode=[27:0], v_dtype=[31:28]
*/
__le32 v_opcode_dtype;
/* Virtchnl return value */
__le32 v_retval;
union {
struct {
__le32 param0;
__le32 param1;
__le32 param2;
__le32 param3;
} direct;
struct {
__le32 param0;
__le16 sw_cookie;
/* Virtchnl flags */
__le16 v_flags;
__le32 addr_high;
__le32 addr_low;
} indirect;
u8 raw[16];
} params;
};
/* Flags sub-structure
* |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
* |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
*/
/* command flags and offsets */
#define IDPF_CTLQ_FLAG_DD_S 0
#define IDPF_CTLQ_FLAG_CMP_S 1
#define IDPF_CTLQ_FLAG_ERR_S 2
#define IDPF_CTLQ_FLAG_FTYPE_S 6
#define IDPF_CTLQ_FLAG_RD_S 10
#define IDPF_CTLQ_FLAG_VFC_S 11
#define IDPF_CTLQ_FLAG_BUF_S 12
#define IDPF_CTLQ_FLAG_HOST_ID_S 13
#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
/* Host ID is a special field that has 3b and not a 1b flag */
#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
struct idpf_mbxq_desc {
u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
u32 chnl_opcode; /* avoid confusion with desc->opcode */
u32 chnl_retval; /* ditto for desc->retval */
u32 pf_vf_id; /* used by CP when sending to PF */
};
/* Define the driver hardware struct to replace other control structs as needed
* Align to ctlq_hw_info
*/
struct idpf_hw {
void __iomem *hw_addr;
resource_size_t hw_addr_len;
struct idpf_adapter *back;
/* control queue - send and receive */
struct idpf_ctlq_info *asq;
struct idpf_ctlq_info *arq;
/* pci info */
u16 device_id;
u16 vendor_id;
u16 subsystem_device_id;
u16 subsystem_vendor_id;
u8 revision_id;
bool adapter_stopped;
struct list_head cq_list_head;
};
int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
struct idpf_ctlq_info *cq);
void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
/* prototype for functions used for dynamic memory allocation */
void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
u64 size);
void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
#endif /* _IDPF_CONTROLQ_H_ */

169
drivers/net/ethernet/intel/idpf/idpf_controlq_api.h Normal file
View File

@ -0,0 +1,169 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_CONTROLQ_API_H_
#define _IDPF_CONTROLQ_API_H_
#include "idpf_mem.h"
struct idpf_hw;
/* Used for queue init, response and events */
enum idpf_ctlq_type {
IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
IDPF_CTLQ_TYPE_CONFIG_TX = 2,
IDPF_CTLQ_TYPE_CONFIG_RX = 3,
IDPF_CTLQ_TYPE_EVENT_RX = 4,
IDPF_CTLQ_TYPE_RDMA_TX = 5,
IDPF_CTLQ_TYPE_RDMA_RX = 6,
IDPF_CTLQ_TYPE_RDMA_COMPL = 7
};
/* Generic Control Queue Structures */
struct idpf_ctlq_reg {
/* used for queue tracking */
u32 head;
u32 tail;
/* Below applies only to default mb (if present) */
u32 len;
u32 bah;
u32 bal;
u32 len_mask;
u32 len_ena_mask;
u32 head_mask;
};
/* Generic queue msg structure */
struct idpf_ctlq_msg {
u8 vmvf_type; /* represents the source of the message on recv */
#define IDPF_VMVF_TYPE_VF 0
#define IDPF_VMVF_TYPE_VM 1
#define IDPF_VMVF_TYPE_PF 2
u8 host_id;
/* 3b field used only when sending a message to CP - to be used in
* combination with target func_id to route the message
*/
#define IDPF_HOST_ID_MASK 0x7
u16 opcode;
u16 data_len; /* data_len = 0 when no payload is attached */
union {
u16 func_id; /* when sending a message */
u16 status; /* when receiving a message */
};
union {
struct {
u32 chnl_opcode;
u32 chnl_retval;
} mbx;
} cookie;
union {
#define IDPF_DIRECT_CTX_SIZE 16
#define IDPF_INDIRECT_CTX_SIZE 8
/* 16 bytes of context can be provided or 8 bytes of context
* plus the address of a DMA buffer
*/
u8 direct[IDPF_DIRECT_CTX_SIZE];
struct {
u8 context[IDPF_INDIRECT_CTX_SIZE];
struct idpf_dma_mem *payload;
} indirect;
} ctx;
};
/* Generic queue info structures */
/* MB, CONFIG and EVENT q do not have extended info */
struct idpf_ctlq_create_info {
enum idpf_ctlq_type type;
int id; /* absolute queue offset passed as input
* -1 for default mailbox if present
*/
u16 len; /* Queue length passed as input */
u16 buf_size; /* buffer size passed as input */
u64 base_address; /* output, HPA of the Queue start */
struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
int ext_info_size;
void *ext_info; /* Specific to q type */
};
/* Control Queue information */
struct idpf_ctlq_info {
struct list_head cq_list;
enum idpf_ctlq_type cq_type;
int q_id;
struct mutex cq_lock; /* control queue lock */
/* used for interrupt processing */
u16 next_to_use;
u16 next_to_clean;
u16 next_to_post; /* starting descriptor to post buffers
* to after recev
*/
struct idpf_dma_mem desc_ring; /* descriptor ring memory
* idpf_dma_mem is defined in OSdep.h
*/
union {
struct idpf_dma_mem **rx_buff;
struct idpf_ctlq_msg **tx_msg;
} bi;
u16 buf_size; /* queue buffer size */
u16 ring_size; /* Number of descriptors */
struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
};
/**
* enum idpf_mbx_opc - PF/VF mailbox commands
* @idpf_mbq_opc_send_msg_to_cp: used by PF or VF to send a message to its CP
*/
enum idpf_mbx_opc {
idpf_mbq_opc_send_msg_to_cp = 0x0801,
};
/* API supported for control queue management */
/* Will init all required q including default mb. "q_info" is an array of
* create_info structs equal to the number of control queues to be created.
*/
int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
struct idpf_ctlq_create_info *q_info);
/* Allocate and initialize a single control queue, which will be added to the
* control queue list; returns a handle to the created control queue
*/
int idpf_ctlq_add(struct idpf_hw *hw,
struct idpf_ctlq_create_info *qinfo,
struct idpf_ctlq_info **cq);
/* Deinitialize and deallocate a single control queue */
void idpf_ctlq_remove(struct idpf_hw *hw,
struct idpf_ctlq_info *cq);
/* Sends messages to HW and will also free the buffer*/
int idpf_ctlq_send(struct idpf_hw *hw,
struct idpf_ctlq_info *cq,
u16 num_q_msg,
struct idpf_ctlq_msg q_msg[]);
/* Receives messages and called by interrupt handler/polling
* initiated by app/process. Also caller is supposed to free the buffers
*/
int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
struct idpf_ctlq_msg *q_msg);
/* Reclaims send descriptors on HW write back */
int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
struct idpf_ctlq_msg *msg_status[]);
/* Indicate RX buffers are done being processed */
int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
struct idpf_ctlq_info *cq,
u16 *buff_count,
struct idpf_dma_mem **buffs);
/* Will destroy all q including the default mb */
void idpf_ctlq_deinit(struct idpf_hw *hw);
#endif /* _IDPF_CONTROLQ_API_H_ */

171
drivers/net/ethernet/intel/idpf/idpf_controlq_setup.c Normal file
View File

@ -0,0 +1,171 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
#include "idpf_controlq.h"
/**
* idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
*/
static int idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
struct idpf_ctlq_info *cq)
{
size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
if (!cq->desc_ring.va)
return -ENOMEM;
return 0;
}
/**
* idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
*
* Allocate the buffer head for all control queues, and if it's a receive
* queue, allocate DMA buffers
*/
static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
struct idpf_ctlq_info *cq)
{
int i;
/* Do not allocate DMA buffers for transmit queues */
if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
return 0;
/* We'll be allocating the buffer info memory first, then we can
* allocate the mapped buffers for the event processing
*/
cq->bi.rx_buff = kcalloc(cq->ring_size, sizeof(struct idpf_dma_mem *),
GFP_KERNEL);
if (!cq->bi.rx_buff)
return -ENOMEM;
/* allocate the mapped buffers (except for the last one) */
for (i = 0; i < cq->ring_size - 1; i++) {
struct idpf_dma_mem *bi;
int num = 1; /* number of idpf_dma_mem to be allocated */
cq->bi.rx_buff[i] = kcalloc(num, sizeof(struct idpf_dma_mem),
GFP_KERNEL);
if (!cq->bi.rx_buff[i])
goto unwind_alloc_cq_bufs;
bi = cq->bi.rx_buff[i];
bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
if (!bi->va) {
/* unwind will not free the failed entry */
kfree(cq->bi.rx_buff[i]);
goto unwind_alloc_cq_bufs;
}
}
return 0;
unwind_alloc_cq_bufs:
/* don't try to free the one that failed... */
i--;
for (; i >= 0; i--) {
idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
kfree(cq->bi.rx_buff[i]);
}
kfree(cq->bi.rx_buff);
return -ENOMEM;
}
/**
* idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
*
* This assumes the posted send buffers have already been cleaned
* and de-allocated
*/
static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
struct idpf_ctlq_info *cq)
{
idpf_free_dma_mem(hw, &cq->desc_ring);
}
/**
* idpf_ctlq_free_bufs - Free CQ buffer info elements
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
*
* Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
* queues. The upper layers are expected to manage freeing of TX DMA buffers
*/
static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
{
void *bi;
if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
int i;
/* free DMA buffers for rx queues*/
for (i = 0; i < cq->ring_size; i++) {
if (cq->bi.rx_buff[i]) {
idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
kfree(cq->bi.rx_buff[i]);
}
}
bi = (void *)cq->bi.rx_buff;
} else {
bi = (void *)cq->bi.tx_msg;
}
/* free the buffer header */
kfree(bi);
}
/**
* idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
* @hw: pointer to hw struct
* @cq: pointer to the specific Control queue
*
* Free the memory used by the ring, buffers and other related structures
*/
void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
{
/* free ring buffers and the ring itself */
idpf_ctlq_free_bufs(hw, cq);
idpf_ctlq_free_desc_ring(hw, cq);
}
/**
* idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
* @hw: pointer to hw struct
* @cq: pointer to control queue struct
*
* Do *NOT* hold cq_lock when calling this as the memory allocation routines
* called are not going to be atomic context safe
*/
int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
{
int err;
/* allocate the ring memory */
err = idpf_ctlq_alloc_desc_ring(hw, cq);
if (err)
return err;
/* allocate buffers in the rings */
err = idpf_ctlq_alloc_bufs(hw, cq);
if (err)
goto idpf_init_cq_free_ring;
/* success! */
return 0;
idpf_init_cq_free_ring:
idpf_free_dma_mem(hw, &cq->desc_ring);
return err;
}

165
drivers/net/ethernet/intel/idpf/idpf_dev.c Normal file
View File

@ -0,0 +1,165 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
#include "idpf.h"
#include "idpf_lan_pf_regs.h"
#define IDPF_PF_ITR_IDX_SPACING 0x4
/**
* idpf_ctlq_reg_init - initialize default mailbox registers
* @cq: pointer to the array of create control queues
*/
static void idpf_ctlq_reg_init(struct idpf_ctlq_create_info *cq)
{
int i;
for (i = 0; i < IDPF_NUM_DFLT_MBX_Q; i++) {
struct idpf_ctlq_create_info *ccq = cq + i;
switch (ccq->type) {
case IDPF_CTLQ_TYPE_MAILBOX_TX:
/* set head and tail registers in our local struct */
ccq->reg.head = PF_FW_ATQH;
ccq->reg.tail = PF_FW_ATQT;
ccq->reg.len = PF_FW_ATQLEN;
ccq->reg.bah = PF_FW_ATQBAH;
ccq->reg.bal = PF_FW_ATQBAL;
ccq->reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
ccq->reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
ccq->reg.head_mask = PF_FW_ATQH_ATQH_M;
break;
case IDPF_CTLQ_TYPE_MAILBOX_RX:
/* set head and tail registers in our local struct */
ccq->reg.head = PF_FW_ARQH;
ccq->reg.tail = PF_FW_ARQT;
ccq->reg.len = PF_FW_ARQLEN;
ccq->reg.bah = PF_FW_ARQBAH;
ccq->reg.bal = PF_FW_ARQBAL;
ccq->reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
ccq->reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
ccq->reg.head_mask = PF_FW_ARQH_ARQH_M;
break;
default:
break;
}
}
}
/**
* idpf_mb_intr_reg_init - Initialize mailbox interrupt register
* @adapter: adapter structure
*/
static void idpf_mb_intr_reg_init(struct idpf_adapter *adapter)
{
struct idpf_intr_reg *intr = &adapter->mb_vector.intr_reg;
u32 dyn_ctl = le32_to_cpu(adapter->caps.mailbox_dyn_ctl);
intr->dyn_ctl = idpf_get_reg_addr(adapter, dyn_ctl);
intr->dyn_ctl_intena_m = PF_GLINT_DYN_CTL_INTENA_M;
intr->dyn_ctl_itridx_m = PF_GLINT_DYN_CTL_ITR_INDX_M;
intr->icr_ena = idpf_get_reg_addr(adapter, PF_INT_DIR_OICR_ENA);
intr->icr_ena_ctlq_m = PF_INT_DIR_OICR_ENA_M;
}
/**
* idpf_intr_reg_init - Initialize interrupt registers
* @vport: virtual port structure
*/
static int idpf_intr_reg_init(struct idpf_vport *vport)
{
struct idpf_adapter *adapter = vport->adapter;
int num_vecs = vport->num_q_vectors;
struct idpf_vec_regs *reg_vals;
int num_regs, i, err = 0;
u32 rx_itr, tx_itr;
u16 total_vecs;
total_vecs = idpf_get_reserved_vecs(vport->adapter);
reg_vals = kcalloc(total_vecs, sizeof(struct idpf_vec_regs),
GFP_KERNEL);
if (!reg_vals)
return -ENOMEM;
num_regs = idpf_get_reg_intr_vecs(vport, reg_vals);
if (num_regs < num_vecs) {
err = -EINVAL;
goto free_reg_vals;
}
for (i = 0; i < num_vecs; i++) {
struct idpf_q_vector *q_vector = &vport->q_vectors[i];
u16 vec_id = vport->q_vector_idxs[i] - IDPF_MBX_Q_VEC;
struct idpf_intr_reg *intr = &q_vector->intr_reg;
u32 spacing;
intr->dyn_ctl = idpf_get_reg_addr(adapter,
reg_vals[vec_id].dyn_ctl_reg);
intr->dyn_ctl_intena_m = PF_GLINT_DYN_CTL_INTENA_M;
intr->dyn_ctl_itridx_s = PF_GLINT_DYN_CTL_ITR_INDX_S;
intr->dyn_ctl_intrvl_s = PF_GLINT_DYN_CTL_INTERVAL_S;
spacing = IDPF_ITR_IDX_SPACING(reg_vals[vec_id].itrn_index_spacing,
IDPF_PF_ITR_IDX_SPACING);
rx_itr = PF_GLINT_ITR_ADDR(VIRTCHNL2_ITR_IDX_0,
reg_vals[vec_id].itrn_reg,
spacing);
tx_itr = PF_GLINT_ITR_ADDR(VIRTCHNL2_ITR_IDX_1,
reg_vals[vec_id].itrn_reg,
spacing);
intr->rx_itr = idpf_get_reg_addr(adapter, rx_itr);
intr->tx_itr = idpf_get_reg_addr(adapter, tx_itr);
}
free_reg_vals:
kfree(reg_vals);
return err;
}
/**
* idpf_reset_reg_init - Initialize reset registers
* @adapter: Driver specific private structure
*/
static void idpf_reset_reg_init(struct idpf_adapter *adapter)
{
adapter->reset_reg.rstat = idpf_get_reg_addr(adapter, PFGEN_RSTAT);
adapter->reset_reg.rstat_m = PFGEN_RSTAT_PFR_STATE_M;
}
/**
* idpf_trigger_reset - trigger reset
* @adapter: Driver specific private structure
* @trig_cause: Reason to trigger a reset
*/
static void idpf_trigger_reset(struct idpf_adapter *adapter,
enum idpf_flags __always_unused trig_cause)
{
u32 reset_reg;
reset_reg = readl(idpf_get_reg_addr(adapter, PFGEN_CTRL));
writel(reset_reg | PFGEN_CTRL_PFSWR,
idpf_get_reg_addr(adapter, PFGEN_CTRL));
}
/**
* idpf_reg_ops_init - Initialize register API function pointers
* @adapter: Driver specific private structure
*/
static void idpf_reg_ops_init(struct idpf_adapter *adapter)
{
adapter->dev_ops.reg_ops.ctlq_reg_init = idpf_ctlq_reg_init;
adapter->dev_ops.reg_ops.intr_reg_init = idpf_intr_reg_init;
adapter->dev_ops.reg_ops.mb_intr_reg_init = idpf_mb_intr_reg_init;
adapter->dev_ops.reg_ops.reset_reg_init = idpf_reset_reg_init;
adapter->dev_ops.reg_ops.trigger_reset = idpf_trigger_reset;
}
/**
* idpf_dev_ops_init - Initialize device API function pointers
* @adapter: Driver specific private structure
*/
void idpf_dev_ops_init(struct idpf_adapter *adapter)
{
idpf_reg_ops_init(adapter);
}

10
drivers/net/ethernet/intel/idpf/idpf_devids.h Normal file
View File

@ -0,0 +1,10 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_DEVIDS_H_
#define _IDPF_DEVIDS_H_
#define IDPF_DEV_ID_PF 0x1452
#define IDPF_DEV_ID_VF 0x145C
#endif /* _IDPF_DEVIDS_H_ */

1369
drivers/net/ethernet/intel/idpf/idpf_ethtool.c Normal file
View File

File diff suppressed because it is too large Load Diff

124
drivers/net/ethernet/intel/idpf/idpf_lan_pf_regs.h Normal file
View File

@ -0,0 +1,124 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_LAN_PF_REGS_H_
#define _IDPF_LAN_PF_REGS_H_
/* Receive queues */
#define PF_QRX_BASE 0x00000000
#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
#define PF_QRX_BUFFQ_BASE 0x03000000
#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
/* Transmit queues */
#define PF_QTX_BASE 0x05000000
#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
/* Control(PF Mailbox) Queue */
#define PF_FW_BASE 0x08400000
#define PF_FW_ARQBAL (PF_FW_BASE)
#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
#define PF_FW_ARQLEN_ARQLEN_S 0
#define PF_FW_ARQLEN_ARQLEN_M GENMASK(12, 0)
#define PF_FW_ARQLEN_ARQVFE_S 28
#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
#define PF_FW_ARQLEN_ARQOVFL_S 29
#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
#define PF_FW_ARQLEN_ARQCRIT_S 30
#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
#define PF_FW_ARQLEN_ARQENABLE_S 31
#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
#define PF_FW_ARQH (PF_FW_BASE + 0xC)
#define PF_FW_ARQH_ARQH_S 0
#define PF_FW_ARQH_ARQH_M GENMASK(12, 0)
#define PF_FW_ARQT (PF_FW_BASE + 0x10)
#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
#define PF_FW_ATQLEN_ATQLEN_S 0
#define PF_FW_ATQLEN_ATQLEN_M GENMASK(9, 0)
#define PF_FW_ATQLEN_ATQVFE_S 28
#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
#define PF_FW_ATQLEN_ATQOVFL_S 29
#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
#define PF_FW_ATQLEN_ATQCRIT_S 30
#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
#define PF_FW_ATQLEN_ATQENABLE_S 31
#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
#define PF_FW_ATQH (PF_FW_BASE + 0x20)
#define PF_FW_ATQH_ATQH_S 0
#define PF_FW_ATQH_ATQH_M GENMASK(9, 0)
#define PF_FW_ATQT (PF_FW_BASE + 0x24)
/* Interrupts */
#define PF_GLINT_BASE 0x08900000
#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
#define PF_GLINT_DYN_CTL_INTENA_S 0
#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
#define PF_GLINT_DYN_CTL_ITR_INDX_M GENMASK(4, 3)
#define PF_GLINT_DYN_CTL_INTERVAL_S 5
#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
/* _ITR is ITR index, _INT is interrupt index, _itrn_indx_spacing is
* spacing b/w itrn registers of the same vector.
*/
#define PF_GLINT_ITR_ADDR(_ITR, _reg_start, _itrn_indx_spacing) \
((_reg_start) + ((_ITR) * (_itrn_indx_spacing)))
/* For PF, itrn_indx_spacing is 4 and itrn_reg_spacing is 0x1000 */
#define PF_GLINT_ITR(_ITR, _INT) \
(PF_GLINT_BASE + (((_ITR) + 1) * 4) + ((_INT) * 0x1000))
#define PF_GLINT_ITR_MAX_INDEX 2
#define PF_GLINT_ITR_INTERVAL_S 0
#define PF_GLINT_ITR_INTERVAL_M GENMASK(11, 0)
/* Generic registers */
#define PF_INT_DIR_OICR_ENA 0x08406000
#define PF_INT_DIR_OICR_ENA_S 0
#define PF_INT_DIR_OICR_ENA_M GENMASK(31, 0)
#define PF_INT_DIR_OICR 0x08406004
#define PF_INT_DIR_OICR_TSYN_EVNT 0
#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
#define PF_INT_DIR_OICR_CAUSE 0x08406008
#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
#define PF_INT_DIR_OICR_CAUSE_CAUSE_M GENMASK(31, 0)
#define PF_INT_PBA_CLEAR 0x0840600C
#define PF_FUNC_RID 0x08406010
#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
#define PF_FUNC_RID_FUNCTION_NUMBER_M GENMASK(2, 0)
#define PF_FUNC_RID_DEVICE_NUMBER_S 3
#define PF_FUNC_RID_DEVICE_NUMBER_M GENMASK(7, 3)
#define PF_FUNC_RID_BUS_NUMBER_S 8
#define PF_FUNC_RID_BUS_NUMBER_M GENMASK(15, 8)
/* Reset registers */
#define PFGEN_RTRIG 0x08407000
#define PFGEN_RTRIG_CORER_S 0
#define PFGEN_RTRIG_CORER_M BIT(0)
#define PFGEN_RTRIG_LINKR_S 1
#define PFGEN_RTRIG_LINKR_M BIT(1)
#define PFGEN_RTRIG_IMCR_S 2
#define PFGEN_RTRIG_IMCR_M BIT(2)
#define PFGEN_RSTAT 0x08407008 /* PFR Status */
#define PFGEN_RSTAT_PFR_STATE_S 0
#define PFGEN_RSTAT_PFR_STATE_M GENMASK(1, 0)
#define PFGEN_CTRL 0x0840700C
#define PFGEN_CTRL_PFSWR BIT(0)
#endif

293
drivers/net/ethernet/intel/idpf/idpf_lan_txrx.h Normal file
View File

@ -0,0 +1,293 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_LAN_TXRX_H_
#define _IDPF_LAN_TXRX_H_
enum idpf_rss_hash {
IDPF_HASH_INVALID = 0,
/* Values 1 - 28 are reserved for future use */
IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
IDPF_HASH_NONF_IPV4_UDP,
IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
IDPF_HASH_NONF_IPV4_TCP,
IDPF_HASH_NONF_IPV4_SCTP,
IDPF_HASH_NONF_IPV4_OTHER,
IDPF_HASH_FRAG_IPV4,
/* Values 37-38 are reserved */
IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
IDPF_HASH_NONF_IPV6_UDP,
IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
IDPF_HASH_NONF_IPV6_TCP,
IDPF_HASH_NONF_IPV6_SCTP,
IDPF_HASH_NONF_IPV6_OTHER,
IDPF_HASH_FRAG_IPV6,
IDPF_HASH_NONF_RSVD47,
IDPF_HASH_NONF_FCOE_OX,
IDPF_HASH_NONF_FCOE_RX,
IDPF_HASH_NONF_FCOE_OTHER,
/* Values 51-62 are reserved */
IDPF_HASH_L2_PAYLOAD = 63,
IDPF_HASH_MAX
};
/* Supported RSS offloads */
#define IDPF_DEFAULT_RSS_HASH \
(BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
BIT_ULL(IDPF_HASH_L2_PAYLOAD))
#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
/* For idpf_splitq_base_tx_compl_desc */
#define IDPF_TXD_COMPLQ_GEN_S 15
#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
#define IDPF_TXD_COMPLQ_COMPL_TYPE_M GENMASK_ULL(13, 11)
#define IDPF_TXD_COMPLQ_QID_S 0
#define IDPF_TXD_COMPLQ_QID_M GENMASK_ULL(9, 0)
/* For base mode TX descriptors */
#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
(0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
(0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
(0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
(0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
#define IDPF_TXD_CTX_QW1_MSS_S 50
#define IDPF_TXD_CTX_QW1_MSS_M GENMASK_ULL(63, 50)
#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
#define IDPF_TXD_CTX_QW1_TSO_LEN_M GENMASK_ULL(47, 30)
#define IDPF_TXD_CTX_QW1_CMD_S 4
#define IDPF_TXD_CTX_QW1_CMD_M GENMASK_ULL(15, 4)
#define IDPF_TXD_CTX_QW1_DTYPE_S 0
#define IDPF_TXD_CTX_QW1_DTYPE_M GENMASK_ULL(3, 0)
#define IDPF_TXD_QW1_L2TAG1_S 48
#define IDPF_TXD_QW1_L2TAG1_M GENMASK_ULL(63, 48)
#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
#define IDPF_TXD_QW1_TX_BUF_SZ_M GENMASK_ULL(47, 34)
#define IDPF_TXD_QW1_OFFSET_S 16
#define IDPF_TXD_QW1_OFFSET_M GENMASK_ULL(33, 16)
#define IDPF_TXD_QW1_CMD_S 4
#define IDPF_TXD_QW1_CMD_M GENMASK_ULL(15, 4)
#define IDPF_TXD_QW1_DTYPE_S 0
#define IDPF_TXD_QW1_DTYPE_M GENMASK_ULL(3, 0)
/* TX Completion Descriptor Completion Types */
#define IDPF_TXD_COMPLT_ITR_FLUSH 0
/* Descriptor completion type 1 is reserved */
#define IDPF_TXD_COMPLT_RS 2
/* Descriptor completion type 3 is reserved */
#define IDPF_TXD_COMPLT_RE 4
#define IDPF_TXD_COMPLT_SW_MARKER 5
enum idpf_tx_desc_dtype_value {
IDPF_TX_DESC_DTYPE_DATA = 0,
IDPF_TX_DESC_DTYPE_CTX = 1,
/* DTYPE 2 is reserved
* DTYPE 3 is free for future use
* DTYPE 4 is reserved
*/
IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
/* DTYPE 6 is reserved */
IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
/* DTYPE 8, 9 are free for future use
* DTYPE 10 is reserved
* DTYPE 11 is free for future use
*/
IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
/* DTYPE 13, 14 are free for future use */
/* DESC_DONE - HW has completed write-back of descriptor */
IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
};
enum idpf_tx_ctx_desc_cmd_bits {
IDPF_TX_CTX_DESC_TSO = 0x01,
IDPF_TX_CTX_DESC_TSYN = 0x02,
IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
IDPF_TX_CTX_DESC_RSVD = 0x08,
IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
IDPF_TX_CTX_DESC_RSVD1 = 0xF00
};
enum idpf_tx_desc_len_fields {
/* Note: These are predefined bit offsets */
IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
};
enum idpf_tx_base_desc_cmd_bits {
IDPF_TX_DESC_CMD_EOP = BIT(0),
IDPF_TX_DESC_CMD_RS = BIT(1),
/* only on VFs else RSVD */
IDPF_TX_DESC_CMD_ICRC = BIT(2),
IDPF_TX_DESC_CMD_IL2TAG1 = BIT(3),
IDPF_TX_DESC_CMD_RSVD1 = BIT(4),
IDPF_TX_DESC_CMD_IIPT_IPV6 = BIT(5),
IDPF_TX_DESC_CMD_IIPT_IPV4 = BIT(6),
IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = GENMASK(6, 5),
IDPF_TX_DESC_CMD_RSVD2 = BIT(7),
IDPF_TX_DESC_CMD_L4T_EOFT_TCP = BIT(8),
IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = BIT(9),
IDPF_TX_DESC_CMD_L4T_EOFT_UDP = GENMASK(9, 8),
IDPF_TX_DESC_CMD_RSVD3 = BIT(10),
IDPF_TX_DESC_CMD_RSVD4 = BIT(11),
};
/* Transmit descriptors */
/* splitq tx buf, singleq tx buf and singleq compl desc */
struct idpf_base_tx_desc {
__le64 buf_addr; /* Address of descriptor's data buf */
__le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
}; /* read used with buffer queues */
struct idpf_splitq_tx_compl_desc {
/* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
__le16 qid_comptype_gen;
union {
__le16 q_head; /* Queue head */
__le16 compl_tag; /* Completion tag */
} q_head_compl_tag;
u8 ts[3];
u8 rsvd; /* Reserved */
}; /* writeback used with completion queues */
/* Context descriptors */
struct idpf_base_tx_ctx_desc {
struct {
__le32 tunneling_params;
__le16 l2tag2;
__le16 rsvd1;
} qw0;
__le64 qw1; /* type_cmd_tlen_mss/rt_hint */
};
/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
enum idpf_tx_flex_desc_cmd_bits {
IDPF_TX_FLEX_DESC_CMD_EOP = BIT(0),
IDPF_TX_FLEX_DESC_CMD_RS = BIT(1),
IDPF_TX_FLEX_DESC_CMD_RE = BIT(2),
IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = BIT(3),
IDPF_TX_FLEX_DESC_CMD_DUMMY = BIT(4),
IDPF_TX_FLEX_DESC_CMD_CS_EN = BIT(5),
IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = BIT(6),
IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = BIT(7),
};
struct idpf_flex_tx_desc {
__le64 buf_addr; /* Packet buffer address */
struct {
#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
#define IDPF_FLEX_TXD_QW1_DTYPE_M GENMASK(4, 0)
#define IDPF_FLEX_TXD_QW1_CMD_S 5
#define IDPF_FLEX_TXD_QW1_CMD_M GENMASK(15, 5)
__le16 cmd_dtype;
/* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
struct {
__le16 l2tag1;
__le16 l2tag2;
} l2tags;
__le16 buf_size;
} qw1;
};
struct idpf_flex_tx_sched_desc {
__le64 buf_addr; /* Packet buffer address */
/* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
struct {
u8 cmd_dtype;
#define IDPF_TXD_FLEX_FLOW_DTYPE_M GENMASK(4, 0)
#define IDPF_TXD_FLEX_FLOW_CMD_EOP BIT(5)
#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN BIT(6)
#define IDPF_TXD_FLEX_FLOW_CMD_RE BIT(7)
/* [23:23] Horizon Overflow bit, [22:0] timestamp */
u8 ts[3];
#define IDPF_TXD_FLOW_SCH_HORIZON_OVERFLOW_M BIT(7)
__le16 compl_tag;
__le16 rxr_bufsize;
#define IDPF_TXD_FLEX_FLOW_RXR BIT(14)
#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M GENMASK(13, 0)
} qw1;
};
/* Common cmd fields for all flex context descriptors
* Note: these defines already account for the 5 bit dtype in the cmd_dtype
* field
*/
enum idpf_tx_flex_ctx_desc_cmd_bits {
IDPF_TX_FLEX_CTX_DESC_CMD_TSO = BIT(5),
IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = BIT(6),
IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = BIT(7),
IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = BIT(9),
IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = BIT(10),
IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = GENMASK(10, 9),
};
/* Standard flex descriptor TSO context quad word */
struct idpf_flex_tx_tso_ctx_qw {
__le32 flex_tlen;
#define IDPF_TXD_FLEX_CTX_TLEN_M GENMASK(17, 0)
#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
__le16 mss_rt;
#define IDPF_TXD_FLEX_CTX_MSS_RT_M GENMASK(13, 0)
u8 hdr_len;
u8 flex;
};
struct idpf_flex_tx_ctx_desc {
/* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
struct {
struct idpf_flex_tx_tso_ctx_qw qw0;
struct {
__le16 cmd_dtype;
u8 flex[6];
} qw1;
} tso;
};
#endif /* _IDPF_LAN_TXRX_H_ */

128
drivers/net/ethernet/intel/idpf/idpf_lan_vf_regs.h Normal file
View File

@ -0,0 +1,128 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_LAN_VF_REGS_H_
#define _IDPF_LAN_VF_REGS_H_
/* Reset */
#define VFGEN_RSTAT 0x00008800
#define VFGEN_RSTAT_VFR_STATE_S 0
#define VFGEN_RSTAT_VFR_STATE_M GENMASK(1, 0)
/* Control(VF Mailbox) Queue */
#define VF_BASE 0x00006000
#define VF_ATQBAL (VF_BASE + 0x1C00)
#define VF_ATQBAH (VF_BASE + 0x1800)
#define VF_ATQLEN (VF_BASE + 0x0800)
#define VF_ATQLEN_ATQLEN_S 0
#define VF_ATQLEN_ATQLEN_M GENMASK(9, 0)
#define VF_ATQLEN_ATQVFE_S 28
#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
#define VF_ATQLEN_ATQOVFL_S 29
#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
#define VF_ATQLEN_ATQCRIT_S 30
#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
#define VF_ATQLEN_ATQENABLE_S 31
#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
#define VF_ATQH (VF_BASE + 0x0400)
#define VF_ATQH_ATQH_S 0
#define VF_ATQH_ATQH_M GENMASK(9, 0)
#define VF_ATQT (VF_BASE + 0x2400)
#define VF_ARQBAL (VF_BASE + 0x0C00)
#define VF_ARQBAH (VF_BASE)
#define VF_ARQLEN (VF_BASE + 0x2000)
#define VF_ARQLEN_ARQLEN_S 0
#define VF_ARQLEN_ARQLEN_M GENMASK(9, 0)
#define VF_ARQLEN_ARQVFE_S 28
#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
#define VF_ARQLEN_ARQOVFL_S 29
#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
#define VF_ARQLEN_ARQCRIT_S 30
#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
#define VF_ARQLEN_ARQENABLE_S 31
#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
#define VF_ARQH (VF_BASE + 0x1400)
#define VF_ARQH_ARQH_S 0
#define VF_ARQH_ARQH_M GENMASK(12, 0)
#define VF_ARQT (VF_BASE + 0x1000)
/* Transmit queues */
#define VF_QTX_TAIL_BASE 0x00000000
#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
#define VF_QTX_TAIL_EXT_BASE 0x00040000
#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
/* Receive queues */
#define VF_QRX_TAIL_BASE 0x00002000
#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
#define VF_QRX_TAIL_EXT_BASE 0x00050000
#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
#define VF_QRXB_TAIL_BASE 0x00060000
#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
/* Interrupts */
#define VF_INT_DYN_CTL0 0x00005C00
#define VF_INT_DYN_CTL0_INTENA_S 0
#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
#define VF_INT_DYN_CTL0_ITR_INDX_S 3
#define VF_INT_DYN_CTL0_ITR_INDX_M GENMASK(4, 3)
#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
#define VF_INT_DYN_CTLN_INTENA_S 0
#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
#define VF_INT_DYN_CTLN_CLEARPBA_S 1
#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
#define VF_INT_DYN_CTLN_ITR_INDX_S 3
#define VF_INT_DYN_CTLN_ITR_INDX_M GENMASK(4, 3)
#define VF_INT_DYN_CTLN_INTERVAL_S 5
#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
/* _ITR is ITR index, _INT is interrupt index, _itrn_indx_spacing is spacing
* b/w itrn registers of the same vector
*/
#define VF_INT_ITR0(_ITR) (0x00004C00 + ((_ITR) * 4))
#define VF_INT_ITRN_ADDR(_ITR, _reg_start, _itrn_indx_spacing) \
((_reg_start) + ((_ITR) * (_itrn_indx_spacing)))
/* For VF with 16 vector support, itrn_reg_spacing is 0x4, itrn_indx_spacing
* is 0x40 and base register offset is 0x00002800
*/
#define VF_INT_ITRN(_INT, _ITR) \
(0x00002800 + ((_INT) * 4) + ((_ITR) * 0x40))
/* For VF with 64 vector support, itrn_reg_spacing is 0x4, itrn_indx_spacing
* is 0x100 and base register offset is 0x00002C00
*/
#define VF_INT_ITRN_64(_INT, _ITR) \
(0x00002C00 + ((_INT) * 4) + ((_ITR) * 0x100))
/* For VF with 2k vector support, itrn_reg_spacing is 0x4, itrn_indx_spacing
* is 0x2000 and base register offset is 0x00072000
*/
#define VF_INT_ITRN_2K(_INT, _ITR) \
(0x00072000 + ((_INT) * 4) + ((_ITR) * 0x2000))
#define VF_INT_ITRN_MAX_INDEX 2
#define VF_INT_ITRN_INTERVAL_S 0
#define VF_INT_ITRN_INTERVAL_M GENMASK(11, 0)
#define VF_INT_PBA_CLEAR 0x00008900
#define VF_INT_ICR0_ENA1 0x00005000
#define VF_INT_ICR0_ENA1_ADMINQ_S 30
#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
#define VF_INT_ICR0_ENA1_RSVD_S 31
#define VF_INT_ICR01 0x00004800
#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
#define VF_QF_HENA_MAX_INDX 1
#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
#define VF_QF_HKEY_MAX_INDX 12
#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
#define VF_QF_HLUT_MAX_INDX 15
#endif

2379
drivers/net/ethernet/intel/idpf/idpf_lib.c Normal file
View File

File diff suppressed because it is too large Load Diff

279
drivers/net/ethernet/intel/idpf/idpf_main.c Normal file
View File

@ -0,0 +1,279 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
#include "idpf.h"
#include "idpf_devids.h"
#define DRV_SUMMARY "Intel(R) Infrastructure Data Path Function Linux Driver"
MODULE_DESCRIPTION(DRV_SUMMARY);
MODULE_LICENSE("GPL");
/**
* idpf_remove - Device removal routine
* @pdev: PCI device information struct
*/
static void idpf_remove(struct pci_dev *pdev)
{
struct idpf_adapter *adapter = pci_get_drvdata(pdev);
int i;
set_bit(IDPF_REMOVE_IN_PROG, adapter->flags);
/* Wait until vc_event_task is done to consider if any hard reset is
* in progress else we may go ahead and release the resources but the
* thread doing the hard reset might continue the init path and
* end up in bad state.
*/
cancel_delayed_work_sync(&adapter->vc_event_task);
if (adapter->num_vfs)
idpf_sriov_configure(pdev, 0);
idpf_vc_core_deinit(adapter);
/* Be a good citizen and leave the device clean on exit */
adapter->dev_ops.reg_ops.trigger_reset(adapter, IDPF_HR_FUNC_RESET);
idpf_deinit_dflt_mbx(adapter);
if (!adapter->netdevs)
goto destroy_wqs;
/* There are some cases where it's possible to still have netdevs
* registered with the stack at this point, e.g. if the driver detected
* a HW reset and rmmod is called before it fully recovers. Unregister
* any stale netdevs here.
*/
for (i = 0; i < adapter->max_vports; i++) {
if (!adapter->netdevs[i])
continue;
if (adapter->netdevs[i]->reg_state != NETREG_UNINITIALIZED)
unregister_netdev(adapter->netdevs[i]);
free_netdev(adapter->netdevs[i]);
adapter->netdevs[i] = NULL;
}
destroy_wqs:
destroy_workqueue(adapter->init_wq);
destroy_workqueue(adapter->serv_wq);
destroy_workqueue(adapter->mbx_wq);
destroy_workqueue(adapter->stats_wq);
destroy_workqueue(adapter->vc_event_wq);
for (i = 0; i < adapter->max_vports; i++) {
kfree(adapter->vport_config[i]);
adapter->vport_config[i] = NULL;
}
kfree(adapter->vport_config);
adapter->vport_config = NULL;
kfree(adapter->netdevs);
adapter->netdevs = NULL;
mutex_destroy(&adapter->vport_ctrl_lock);
mutex_destroy(&adapter->vector_lock);
mutex_destroy(&adapter->queue_lock);
mutex_destroy(&adapter->vc_buf_lock);
pci_set_drvdata(pdev, NULL);
kfree(adapter);
}
/**
* idpf_shutdown - PCI callback for shutting down device
* @pdev: PCI device information struct
*/
static void idpf_shutdown(struct pci_dev *pdev)
{
idpf_remove(pdev);
if (system_state == SYSTEM_POWER_OFF)
pci_set_power_state(pdev, PCI_D3hot);
}
/**
* idpf_cfg_hw - Initialize HW struct
* @adapter: adapter to setup hw struct for
*
* Returns 0 on success, negative on failure
*/
static int idpf_cfg_hw(struct idpf_adapter *adapter)
{
struct pci_dev *pdev = adapter->pdev;
struct idpf_hw *hw = &adapter->hw;
hw->hw_addr = pcim_iomap_table(pdev)[0];
if (!hw->hw_addr) {
pci_err(pdev, "failed to allocate PCI iomap table\n");
return -ENOMEM;
}
hw->back = adapter;
return 0;
}
/**
* idpf_probe - Device initialization routine
* @pdev: PCI device information struct
* @ent: entry in idpf_pci_tbl
*
* Returns 0 on success, negative on failure
*/
static int idpf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
struct device *dev = &pdev->dev;
struct idpf_adapter *adapter;
int err;
adapter = kzalloc(sizeof(*adapter), GFP_KERNEL);
if (!adapter)
return -ENOMEM;
adapter->req_tx_splitq = true;
adapter->req_rx_splitq = true;
switch (ent->device) {
case IDPF_DEV_ID_PF:
idpf_dev_ops_init(adapter);
break;
case IDPF_DEV_ID_VF:
idpf_vf_dev_ops_init(adapter);
adapter->crc_enable = true;
break;
default:
err = -ENODEV;
dev_err(&pdev->dev, "Unexpected dev ID 0x%x in idpf probe\n",
ent->device);
goto err_free;
}
adapter->pdev = pdev;
err = pcim_enable_device(pdev);
if (err)
goto err_free;
err = pcim_iomap_regions(pdev, BIT(0), pci_name(pdev));
if (err) {
pci_err(pdev, "pcim_iomap_regions failed %pe\n", ERR_PTR(err));
goto err_free;
}
/* set up for high or low dma */
err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
if (err) {
pci_err(pdev, "DMA configuration failed: %pe\n", ERR_PTR(err));
goto err_free;
}
pci_set_master(pdev);
pci_set_drvdata(pdev, adapter);
adapter->init_wq = alloc_workqueue("%s-%s-init", 0, 0,
dev_driver_string(dev),
dev_name(dev));
if (!adapter->init_wq) {
dev_err(dev, "Failed to allocate init workqueue\n");
err = -ENOMEM;
goto err_free;
}
adapter->serv_wq = alloc_workqueue("%s-%s-service", 0, 0,
dev_driver_string(dev),
dev_name(dev));
if (!adapter->serv_wq) {
dev_err(dev, "Failed to allocate service workqueue\n");
err = -ENOMEM;
goto err_serv_wq_alloc;
}
adapter->mbx_wq = alloc_workqueue("%s-%s-mbx", 0, 0,
dev_driver_string(dev),
dev_name(dev));
if (!adapter->mbx_wq) {
dev_err(dev, "Failed to allocate mailbox workqueue\n");
err = -ENOMEM;
goto err_mbx_wq_alloc;
}
adapter->stats_wq = alloc_workqueue("%s-%s-stats", 0, 0,
dev_driver_string(dev),
dev_name(dev));
if (!adapter->stats_wq) {
dev_err(dev, "Failed to allocate workqueue\n");
err = -ENOMEM;
goto err_stats_wq_alloc;
}
adapter->vc_event_wq = alloc_workqueue("%s-%s-vc_event", 0, 0,
dev_driver_string(dev),
dev_name(dev));
if (!adapter->vc_event_wq) {
dev_err(dev, "Failed to allocate virtchnl event workqueue\n");
err = -ENOMEM;
goto err_vc_event_wq_alloc;
}
/* setup msglvl */
adapter->msg_enable = netif_msg_init(-1, IDPF_AVAIL_NETIF_M);
err = idpf_cfg_hw(adapter);
if (err) {
dev_err(dev, "Failed to configure HW structure for adapter: %d\n",
err);
goto err_cfg_hw;
}
mutex_init(&adapter->vport_ctrl_lock);
mutex_init(&adapter->vector_lock);
mutex_init(&adapter->queue_lock);
mutex_init(&adapter->vc_buf_lock);
init_waitqueue_head(&adapter->vchnl_wq);
INIT_DELAYED_WORK(&adapter->init_task, idpf_init_task);
INIT_DELAYED_WORK(&adapter->serv_task, idpf_service_task);
INIT_DELAYED_WORK(&adapter->mbx_task, idpf_mbx_task);
INIT_DELAYED_WORK(&adapter->stats_task, idpf_statistics_task);
INIT_DELAYED_WORK(&adapter->vc_event_task, idpf_vc_event_task);
adapter->dev_ops.reg_ops.reset_reg_init(adapter);
set_bit(IDPF_HR_DRV_LOAD, adapter->flags);
queue_delayed_work(adapter->vc_event_wq, &adapter->vc_event_task,
msecs_to_jiffies(10 * (pdev->devfn & 0x07)));
return 0;
err_cfg_hw:
destroy_workqueue(adapter->vc_event_wq);
err_vc_event_wq_alloc:
destroy_workqueue(adapter->stats_wq);
err_stats_wq_alloc:
destroy_workqueue(adapter->mbx_wq);
err_mbx_wq_alloc:
destroy_workqueue(adapter->serv_wq);
err_serv_wq_alloc:
destroy_workqueue(adapter->init_wq);
err_free:
kfree(adapter);
return err;
}
/* idpf_pci_tbl - PCI Dev idpf ID Table
*/
static const struct pci_device_id idpf_pci_tbl[] = {
{ PCI_VDEVICE(INTEL, IDPF_DEV_ID_PF)},
{ PCI_VDEVICE(INTEL, IDPF_DEV_ID_VF)},
{ /* Sentinel */ }
};
MODULE_DEVICE_TABLE(pci, idpf_pci_tbl);
static struct pci_driver idpf_driver = {
.name = KBUILD_MODNAME,
.id_table = idpf_pci_tbl,
.probe = idpf_probe,
.sriov_configure = idpf_sriov_configure,
.remove = idpf_remove,
.shutdown = idpf_shutdown,
};
module_pci_driver(idpf_driver);

20
drivers/net/ethernet/intel/idpf/idpf_mem.h Normal file
View File

@ -0,0 +1,20 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _IDPF_MEM_H_
#define _IDPF_MEM_H_
#include <linux/io.h>
struct idpf_dma_mem {
void *va;
dma_addr_t pa;
size_t size;
};
#define wr32(a, reg, value) writel((value), ((a)->hw_addr + (reg)))
#define rd32(a, reg) readl((a)->hw_addr + (reg))
#define wr64(a, reg, value) writeq((value), ((a)->hw_addr + (reg)))
#define rd64(a, reg) readq((a)->hw_addr + (reg))
#endif /* _IDPF_MEM_H_ */

1183
drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c Normal file
View File

File diff suppressed because it is too large Load Diff

4289
drivers/net/ethernet/intel/idpf/idpf_txrx.c Normal file
View File

File diff suppressed because it is too large Load Diff

1023
drivers/net/ethernet/intel/idpf/idpf_txrx.h Normal file
View File

File diff suppressed because it is too large Load Diff

163
drivers/net/ethernet/intel/idpf/idpf_vf_dev.c Normal file
View File

@ -0,0 +1,163 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
#include "idpf.h"
#include "idpf_lan_vf_regs.h"
#define IDPF_VF_ITR_IDX_SPACING 0x40
/**
* idpf_vf_ctlq_reg_init - initialize default mailbox registers
* @cq: pointer to the array of create control queues
*/
static void idpf_vf_ctlq_reg_init(struct idpf_ctlq_create_info *cq)
{
int i;
for (i = 0; i < IDPF_NUM_DFLT_MBX_Q; i++) {
struct idpf_ctlq_create_info *ccq = cq + i;
switch (ccq->type) {
case IDPF_CTLQ_TYPE_MAILBOX_TX:
/* set head and tail registers in our local struct */
ccq->reg.head = VF_ATQH;
ccq->reg.tail = VF_ATQT;
ccq->reg.len = VF_ATQLEN;
ccq->reg.bah = VF_ATQBAH;
ccq->reg.bal = VF_ATQBAL;
ccq->reg.len_mask = VF_ATQLEN_ATQLEN_M;
ccq->reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
ccq->reg.head_mask = VF_ATQH_ATQH_M;
break;
case IDPF_CTLQ_TYPE_MAILBOX_RX:
/* set head and tail registers in our local struct */
ccq->reg.head = VF_ARQH;
ccq->reg.tail = VF_ARQT;
ccq->reg.len = VF_ARQLEN;
ccq->reg.bah = VF_ARQBAH;
ccq->reg.bal = VF_ARQBAL;
ccq->reg.len_mask = VF_ARQLEN_ARQLEN_M;
ccq->reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
ccq->reg.head_mask = VF_ARQH_ARQH_M;
break;
default:
break;
}
}
}
/**
* idpf_vf_mb_intr_reg_init - Initialize the mailbox register
* @adapter: adapter structure
*/
static void idpf_vf_mb_intr_reg_init(struct idpf_adapter *adapter)
{
struct idpf_intr_reg *intr = &adapter->mb_vector.intr_reg;
u32 dyn_ctl = le32_to_cpu(adapter->caps.mailbox_dyn_ctl);
intr->dyn_ctl = idpf_get_reg_addr(adapter, dyn_ctl);
intr->dyn_ctl_intena_m = VF_INT_DYN_CTL0_INTENA_M;
intr->dyn_ctl_itridx_m = VF_INT_DYN_CTL0_ITR_INDX_M;
intr->icr_ena = idpf_get_reg_addr(adapter, VF_INT_ICR0_ENA1);
intr->icr_ena_ctlq_m = VF_INT_ICR0_ENA1_ADMINQ_M;
}
/**
* idpf_vf_intr_reg_init - Initialize interrupt registers
* @vport: virtual port structure
*/
static int idpf_vf_intr_reg_init(struct idpf_vport *vport)
{
struct idpf_adapter *adapter = vport->adapter;
int num_vecs = vport->num_q_vectors;
struct idpf_vec_regs *reg_vals;
int num_regs, i, err = 0;
u32 rx_itr, tx_itr;
u16 total_vecs;
total_vecs = idpf_get_reserved_vecs(vport->adapter);
reg_vals = kcalloc(total_vecs, sizeof(struct idpf_vec_regs),
GFP_KERNEL);
if (!reg_vals)
return -ENOMEM;
num_regs = idpf_get_reg_intr_vecs(vport, reg_vals);
if (num_regs < num_vecs) {
err = -EINVAL;
goto free_reg_vals;
}
for (i = 0; i < num_vecs; i++) {
struct idpf_q_vector *q_vector = &vport->q_vectors[i];
u16 vec_id = vport->q_vector_idxs[i] - IDPF_MBX_Q_VEC;
struct idpf_intr_reg *intr = &q_vector->intr_reg;
u32 spacing;
intr->dyn_ctl = idpf_get_reg_addr(adapter,
reg_vals[vec_id].dyn_ctl_reg);
intr->dyn_ctl_intena_m = VF_INT_DYN_CTLN_INTENA_M;
intr->dyn_ctl_itridx_s = VF_INT_DYN_CTLN_ITR_INDX_S;
spacing = IDPF_ITR_IDX_SPACING(reg_vals[vec_id].itrn_index_spacing,
IDPF_VF_ITR_IDX_SPACING);
rx_itr = VF_INT_ITRN_ADDR(VIRTCHNL2_ITR_IDX_0,
reg_vals[vec_id].itrn_reg,
spacing);
tx_itr = VF_INT_ITRN_ADDR(VIRTCHNL2_ITR_IDX_1,
reg_vals[vec_id].itrn_reg,
spacing);
intr->rx_itr = idpf_get_reg_addr(adapter, rx_itr);
intr->tx_itr = idpf_get_reg_addr(adapter, tx_itr);
}
free_reg_vals:
kfree(reg_vals);
return err;
}
/**
* idpf_vf_reset_reg_init - Initialize reset registers
* @adapter: Driver specific private structure
*/
static void idpf_vf_reset_reg_init(struct idpf_adapter *adapter)
{
adapter->reset_reg.rstat = idpf_get_reg_addr(adapter, VFGEN_RSTAT);
adapter->reset_reg.rstat_m = VFGEN_RSTAT_VFR_STATE_M;
}
/**
* idpf_vf_trigger_reset - trigger reset
* @adapter: Driver specific private structure
* @trig_cause: Reason to trigger a reset
*/
static void idpf_vf_trigger_reset(struct idpf_adapter *adapter,
enum idpf_flags trig_cause)
{
/* Do not send VIRTCHNL2_OP_RESET_VF message on driver unload */
if (trig_cause == IDPF_HR_FUNC_RESET &&
!test_bit(IDPF_REMOVE_IN_PROG, adapter->flags))
idpf_send_mb_msg(adapter, VIRTCHNL2_OP_RESET_VF, 0, NULL);
}
/**
* idpf_vf_reg_ops_init - Initialize register API function pointers
* @adapter: Driver specific private structure
*/
static void idpf_vf_reg_ops_init(struct idpf_adapter *adapter)
{
adapter->dev_ops.reg_ops.ctlq_reg_init = idpf_vf_ctlq_reg_init;
adapter->dev_ops.reg_ops.intr_reg_init = idpf_vf_intr_reg_init;
adapter->dev_ops.reg_ops.mb_intr_reg_init = idpf_vf_mb_intr_reg_init;
adapter->dev_ops.reg_ops.reset_reg_init = idpf_vf_reset_reg_init;
adapter->dev_ops.reg_ops.trigger_reset = idpf_vf_trigger_reset;
}
/**
* idpf_vf_dev_ops_init - Initialize device API function pointers
* @adapter: Driver specific private structure
*/
void idpf_vf_dev_ops_init(struct idpf_adapter *adapter)
{
idpf_vf_reg_ops_init(adapter);
}

3791
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c Normal file
View File

File diff suppressed because it is too large Load Diff

1273
drivers/net/ethernet/intel/idpf/virtchnl2.h Normal file
View File

File diff suppressed because it is too large Load Diff

451
drivers/net/ethernet/intel/idpf/virtchnl2_lan_desc.h Normal file
View File

@ -0,0 +1,451 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright (C) 2023 Intel Corporation */
#ifndef _VIRTCHNL2_LAN_DESC_H_
#define _VIRTCHNL2_LAN_DESC_H_
#include <linux/bits.h>
/* This is an interface definition file where existing enums and their values
* must remain unchanged over time, so we specify explicit values for all enums.
*/
/* Transmit descriptor ID flags
*/
enum virtchnl2_tx_desc_ids {
VIRTCHNL2_TXDID_DATA = BIT(0),
VIRTCHNL2_TXDID_CTX = BIT(1),
/* TXDID bit 2 is reserved
* TXDID bit 3 is free for future use
* TXDID bit 4 is reserved
*/
VIRTCHNL2_TXDID_FLEX_TSO_CTX = BIT(5),
/* TXDID bit 6 is reserved */
VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 = BIT(7),
/* TXDID bits 8 and 9 are free for future use
* TXDID bit 10 is reserved
* TXDID bit 11 is free for future use
*/
VIRTCHNL2_TXDID_FLEX_FLOW_SCHED = BIT(12),
/* TXDID bits 13 and 14 are free for future use */
VIRTCHNL2_TXDID_DESC_DONE = BIT(15),
};
/* Receive descriptor IDs */
enum virtchnl2_rx_desc_ids {
VIRTCHNL2_RXDID_1_32B_BASE = 1,
/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
* differentiated based on queue model; e.g. single queue model can
* only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
* for DID 2.
*/
VIRTCHNL2_RXDID_2_FLEX_SPLITQ = 2,
VIRTCHNL2_RXDID_2_FLEX_SQ_NIC = VIRTCHNL2_RXDID_2_FLEX_SPLITQ,
/* 3 through 6 are reserved */
VIRTCHNL2_RXDID_7_HW_RSVD = 7,
/* 8 through 15 are free */
};
/* Receive descriptor ID bitmasks */
#define VIRTCHNL2_RXDID_M(bit) BIT_ULL(VIRTCHNL2_RXDID_##bit)
enum virtchnl2_rx_desc_id_bitmasks {
VIRTCHNL2_RXDID_1_32B_BASE_M = VIRTCHNL2_RXDID_M(1_32B_BASE),
VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M = VIRTCHNL2_RXDID_M(2_FLEX_SPLITQ),
VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M = VIRTCHNL2_RXDID_M(2_FLEX_SQ_NIC),
VIRTCHNL2_RXDID_7_HW_RSVD_M = VIRTCHNL2_RXDID_M(7_HW_RSVD),
};
/* For splitq virtchnl2_rx_flex_desc_adv_nic_3 desc members */
#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M GENMASK(3, 0)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M GENMASK(7, 6)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M GENMASK(9, 0)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_RAW_CSUM_INV_S 12
#define VIRTCHNL2_RX_FLEX_DESC_ADV_RAW_CSUM_INV_M \
BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RAW_CSUM_INV_S)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M GENMASK(15, 13)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M GENMASK(13, 0)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M GENMASK(9, 0)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M GENMASK(14, 12)
#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
/* Bitmasks for splitq virtchnl2_rx_flex_desc_adv_nic_3 */
enum virtchl2_rx_flex_desc_adv_status_error_0_qw1_bits {
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_M = BIT(0),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_M = BIT(1),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_M = BIT(2),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_M = BIT(3),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_M = BIT(4),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_M = BIT(5),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_M = BIT(6),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_M = BIT(7),
};
/* Bitmasks for splitq virtchnl2_rx_flex_desc_adv_nic_3 */
enum virtchnl2_rx_flex_desc_adv_status_error_0_qw0_bits {
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_M = BIT(0),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_M = BIT(1),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_M = BIT(2),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_M = BIT(3),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_M = BIT(4),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_M = BIT(5),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_M = BIT(6),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_M = BIT(7),
};
/* Bitmasks for splitq virtchnl2_rx_flex_desc_adv_nic_3 */
enum virtchnl2_rx_flex_desc_adv_status_error_1_bits {
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_M = GENMASK(1, 0),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_M = BIT(2),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_M = BIT(3),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_M = BIT(4),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_M = BIT(5),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_M = BIT(6),
VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_M = BIT(7),
};
/* For singleq (flex) virtchnl2_rx_flex_desc fields
* For virtchnl2_rx_flex_desc.ptype_flex_flags0 member
*/
#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M GENMASK(9, 0)
/* For virtchnl2_rx_flex_desc.pkt_len member */
#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M GENMASK(13, 0)
/* Bitmasks for singleq (flex) virtchnl2_rx_flex_desc */
enum virtchnl2_rx_flex_desc_status_error_0_bits {
VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_M = BIT(0),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_M = BIT(1),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_M = BIT(2),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_M = BIT(3),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_M = BIT(4),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_M = BIT(5),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_M = BIT(6),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_M = BIT(7),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_M = BIT(8),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_M = BIT(9),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_M = BIT(10),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_M = BIT(11),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_M = BIT(12),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_M = BIT(13),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_M = BIT(14),
VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_M = BIT(15),
};
/* Bitmasks for singleq (flex) virtchnl2_rx_flex_desc */
enum virtchnl2_rx_flex_desc_status_error_1_bits {
VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_M = GENMASK(3, 0),
VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_M = BIT(4),
VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_M = BIT(5),
/* [10:6] reserved */
VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_M = BIT(11),
VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_M = BIT(12),
VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_M = BIT(13),
VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_M = BIT(14),
VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_M = BIT(15),
};
/* For virtchnl2_rx_flex_desc.ts_low member */
#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M GENMASK_ULL(51, 38)
#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M GENMASK_ULL(37, 30)
#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M GENMASK_ULL(26, 19)
#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M GENMASK_ULL(18, 0)
/* Bitmasks for singleq (base) virtchnl2_rx_base_desc */
enum virtchnl2_rx_base_desc_status_bits {
VIRTCHNL2_RX_BASE_DESC_STATUS_DD_M = BIT(0),
VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_M = BIT(1),
VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_M = BIT(2),
VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_M = BIT(3),
VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_M = BIT(4),
VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_M = GENMASK(7, 5),
VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_M = BIT(8),
VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_M = GENMASK(10, 9),
VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_M = BIT(11),
VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_M = GENMASK(13, 12),
VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_M = BIT(14),
VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_M = BIT(15),
VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_M = GENMASK(17, 16),
VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_M = BIT(18),
};
/* Bitmasks for singleq (base) virtchnl2_rx_base_desc */
enum virtchnl2_rx_base_desc_error_bits {
VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_M = BIT(0),
VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_M = BIT(1),
VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_M = BIT(2),
VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_M = GENMASK(5, 3),
VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_M = BIT(3),
VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_M = BIT(4),
VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_M = BIT(5),
VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_M = BIT(6),
VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_M = BIT(7),
};
/* Bitmasks for singleq (base) virtchnl2_rx_base_desc */
#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH_M GENMASK(13, 12)
/**
* struct virtchnl2_splitq_rx_buf_desc - SplitQ RX buffer descriptor format
* @qword0: RX buffer struct.
* @qword0.buf_id: Buffer identifier.
* @qword0.rsvd0: Reserved.
* @qword0.rsvd1: Reserved.
* @pkt_addr: Packet buffer address.
* @hdr_addr: Header buffer address.
* @rsvd2: Rerserved.
*
* Receive Descriptors
* SplitQ buffer
* | 16| 0|
* ----------------------------------------------------------------
* | RSV | Buffer ID |
* ----------------------------------------------------------------
* | Rx packet buffer address |
* ----------------------------------------------------------------
* | Rx header buffer address |
* ----------------------------------------------------------------
* | RSV |
* ----------------------------------------------------------------
* | 0|
*/
struct virtchnl2_splitq_rx_buf_desc {
struct {
__le16 buf_id;
__le16 rsvd0;
__le32 rsvd1;
} qword0;
__le64 pkt_addr;
__le64 hdr_addr;
__le64 rsvd2;
};
/**
* struct virtchnl2_singleq_rx_buf_desc - SingleQ RX buffer descriptor format.
* @pkt_addr: Packet buffer address.
* @hdr_addr: Header buffer address.
* @rsvd1: Reserved.
* @rsvd2: Reserved.
*
* SingleQ buffer
* | 0|
* ----------------------------------------------------------------
* | Rx packet buffer address |
* ----------------------------------------------------------------
* | Rx header buffer address |
* ----------------------------------------------------------------
* | RSV |
* ----------------------------------------------------------------
* | RSV |
* ----------------------------------------------------------------
* | 0|
*/
struct virtchnl2_singleq_rx_buf_desc {
__le64 pkt_addr;
__le64 hdr_addr;
__le64 rsvd1;
__le64 rsvd2;
};
/**
* struct virtchnl2_singleq_base_rx_desc - RX descriptor writeback format.
* @qword0: First quad word struct.
* @qword0.lo_dword: Lower dual word struct.
* @qword0.lo_dword.mirroring_status: Mirrored packet status.
* @qword0.lo_dword.l2tag1: Stripped L2 tag from the received packet.
* @qword0.hi_dword: High dual word union.
* @qword0.hi_dword.rss: RSS hash.
* @qword0.hi_dword.fd_id: Flow director filter id.
* @qword1: Second quad word struct.
* @qword1.status_error_ptype_len: Status/error/PTYPE/length.
* @qword2: Third quad word struct.
* @qword2.ext_status: Extended status.
* @qword2.rsvd: Reserved.
* @qword2.l2tag2_1: Extracted L2 tag 2 from the packet.
* @qword2.l2tag2_2: Reserved.
* @qword3: Fourth quad word struct.
* @qword3.reserved: Reserved.
* @qword3.fd_id: Flow director filter id.
*
* Profile ID 0x1, SingleQ, base writeback format
*/
struct virtchnl2_singleq_base_rx_desc {
struct {
struct {
__le16 mirroring_status;
__le16 l2tag1;
} lo_dword;
union {
__le32 rss;
__le32 fd_id;
} hi_dword;
} qword0;
struct {
__le64 status_error_ptype_len;
} qword1;
struct {
__le16 ext_status;
__le16 rsvd;
__le16 l2tag2_1;
__le16 l2tag2_2;
} qword2;
struct {
__le32 reserved;
__le32 fd_id;
} qword3;
};
/**
* struct virtchnl2_rx_flex_desc_nic - RX descriptor writeback format.
*
* @rxdid: Descriptor builder profile id.
* @mir_id_umb_cast: umb_cast=[7:6], mirror=[5:0]
* @ptype_flex_flags0: ff0=[15:10], ptype=[9:0]
* @pkt_len: Packet length, [15:14] are reserved.
* @hdr_len_sph_flex_flags1: ff1/ext=[15:12], sph=[11], header=[10:0].
* @status_error0: Status/Error section 0.
* @l2tag1: Stripped L2 tag from the received packet
* @rss_hash: RSS hash.
* @status_error1: Status/Error section 1.
* @flexi_flags2: Flexible flags section 2.
* @ts_low: Lower word of timestamp value.
* @l2tag2_1st: First L2TAG2.
* @l2tag2_2nd: Second L2TAG2.
* @flow_id: Flow id.
* @flex_ts: Timestamp and flexible flow id union.
* @flex_ts.ts_high: Timestamp higher word of the timestamp value.
* @flex_ts.flex.rsvd: Reserved.
* @flex_ts.flex.flow_id_ipv6: IPv6 flow id.
*
* Profile ID 0x2, SingleQ, flex writeback format
*/
struct virtchnl2_rx_flex_desc_nic {
/* Qword 0 */
u8 rxdid;
u8 mir_id_umb_cast;
__le16 ptype_flex_flags0;
__le16 pkt_len;
__le16 hdr_len_sph_flex_flags1;
/* Qword 1 */
__le16 status_error0;
__le16 l2tag1;
__le32 rss_hash;
/* Qword 2 */
__le16 status_error1;
u8 flexi_flags2;
u8 ts_low;
__le16 l2tag2_1st;
__le16 l2tag2_2nd;
/* Qword 3 */
__le32 flow_id;
union {
struct {
__le16 rsvd;
__le16 flow_id_ipv6;
} flex;
__le32 ts_high;
} flex_ts;
};
/**
* struct virtchnl2_rx_flex_desc_adv_nic_3 - RX descriptor writeback format.
* @rxdid_ucast: ucast=[7:6], rsvd=[5:4], profile_id=[3:0].
* @status_err0_qw0: Status/Error section 0 in quad word 0.
* @ptype_err_fflags0: ff0=[15:12], udp_len_err=[11], ip_hdr_err=[10],
* ptype=[9:0].
* @pktlen_gen_bufq_id: bufq_id=[15] only in splitq, gen=[14] only in splitq,
* plen=[13:0].
* @hdrlen_flags: miss_prepend=[15], trunc_mirr=[14], int_udp_0=[13],
* ext_udp0=[12], sph=[11] only in splitq, rsc=[10]
* only in splitq, header=[9:0].
* @status_err0_qw1: Status/Error section 0 in quad word 1.
* @status_err1: Status/Error section 1.
* @fflags1: Flexible flags section 1.
* @ts_low: Lower word of timestamp value.
* @buf_id: Buffer identifier. Only in splitq mode.
* @misc: Union.
* @misc.raw_cs: Raw checksum.
* @misc.l2tag1: Stripped L2 tag from the received packet
* @misc.rscseglen:
* @hash1: Lower bits of Rx hash value.
* @ff2_mirrid_hash2: Union.
* @ff2_mirrid_hash2.fflags2: Flexible flags section 2.
* @ff2_mirrid_hash2.mirrorid: Mirror id.
* @ff2_mirrid_hash2.rscseglen: RSC segment length.
* @hash3: Upper bits of Rx hash value.
* @l2tag2: Extracted L2 tag 2 from the packet.
* @fmd4: Flexible metadata container 4.
* @l2tag1: Stripped L2 tag from the received packet
* @fmd6: Flexible metadata container 6.
* @ts_high: Timestamp higher word of the timestamp value.
*
* Profile ID 0x2, SplitQ, flex writeback format
*
* Flex-field 0: BufferID
* Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
* Flex-field 2: Hash[15:0]
* Flex-flags 2: Hash[23:16]
* Flex-field 3: L2TAG2
* Flex-field 5: L2TAG1
* Flex-field 7: Timestamp (upper 32 bits)
*/
struct virtchnl2_rx_flex_desc_adv_nic_3 {
/* Qword 0 */
u8 rxdid_ucast;
u8 status_err0_qw0;
__le16 ptype_err_fflags0;
__le16 pktlen_gen_bufq_id;
__le16 hdrlen_flags;
/* Qword 1 */
u8 status_err0_qw1;
u8 status_err1;
u8 fflags1;
u8 ts_low;
__le16 buf_id;
union {
__le16 raw_cs;
__le16 l2tag1;
__le16 rscseglen;
} misc;
/* Qword 2 */
__le16 hash1;
union {
u8 fflags2;
u8 mirrorid;
u8 hash2;
} ff2_mirrid_hash2;
u8 hash3;
__le16 l2tag2;
__le16 fmd4;
/* Qword 3 */
__le16 l2tag1;
__le16 fmd6;
__le32 ts_high;
};
/* Common union for accessing descriptor format structs */
union virtchnl2_rx_desc {
struct virtchnl2_singleq_base_rx_desc base_wb;
struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
};
#endif /* _VIRTCHNL_LAN_DESC_H_ */