iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
linux/drivers/infiniband/hw/mlx5/gsi.c

490 lines
12 KiB
C
Raw Normal View History

IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
/*
* Copyright (c) 2016, Mellanox Technologies. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
* General Public License (GPL) Version 2, available from the file
* COPYING in the main directory of this source tree, or the
* OpenIB.org BSD license below:
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above
* copyright notice, this list of conditions and the following
* disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials
* provided with the distribution.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include "mlx5_ib.h"
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
struct mlx5_ib_gsi_wr {
struct ib_cqe cqe;
struct ib_wc wc;
bool completed:1;
};
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
static bool mlx5_ib_deth_sqpn_cap(struct mlx5_ib_dev *dev)
{
return MLX5_CAP_GEN(dev->mdev, set_deth_sqpn);
}
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
/* Call with gsi->lock locked */
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
static void generate_completions(struct mlx5_ib_qp *mqp)
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
{
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
struct ib_cq *gsi_cq = mqp->ibqp.send_cq;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
struct mlx5_ib_gsi_wr *wr;
u32 index;
IB/mlx5: Fix iteration overrun in GSI qps Number of outstanding_pi may overflow and as a result may indicate that there are no elements in the queue. The effect of doing this is that the MAD layer will get stuck waiting for completions. The MAD layer will think that the QP is full - because it didn't receive these completions. This fix changes it so the outstanding_pi number is increased with 32-bit wraparound and is not limited to max_send_wr so that the difference between outstanding_pi and outstanding_ci will really indicate the number of outstanding completions. Cc: Stable <stable@vger.kernel.org> Fixes: ea6dc2036224 ('IB/mlx5: Reorder GSI completions') Signed-off-by: Slava Shwartsman <slavash@mellanox.com> Signed-off-by: Leon Romanovsky <leon@kernel.org> Reviewed-by: Haggai Eran <haggaie@mellanox.com> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-07-03 15:28:19 +03:00
for (index = gsi->outstanding_ci; index != gsi->outstanding_pi;
index++) {
wr = &gsi->outstanding_wrs[index % gsi->cap.max_send_wr];
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
if (!wr->completed)
break;
RDMA/mlx5: Delete not needed GSI QP signal QP type GSI QP doesn't need signal QP type because it is initialized statically to zero, which is IB_SIGNAL_ALL_WR also wr->send_flags isn't set too. This means that the GSI QP signal QP type can be removed. Link: https://lore.kernel.org/r/20200926102450.2966017-5-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:44 +03:00
WARN_ON_ONCE(mlx5_ib_generate_wc(gsi_cq, &wr->wc));
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
wr->completed = false;
}
gsi->outstanding_ci = index;
}
static void handle_single_completion(struct ib_cq *cq, struct ib_wc *wc)
{
struct mlx5_ib_gsi_qp *gsi = cq->cq_context;
struct mlx5_ib_gsi_wr *wr =
container_of(wc->wr_cqe, struct mlx5_ib_gsi_wr, cqe);
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_qp *mqp = container_of(gsi, struct mlx5_ib_qp, gsi);
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
u64 wr_id;
unsigned long flags;
spin_lock_irqsave(&gsi->lock, flags);
wr->completed = true;
wr_id = wr->wc.wr_id;
wr->wc = *wc;
wr->wc.wr_id = wr_id;
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
wr->wc.qp = &mqp->ibqp;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
generate_completions(mqp);
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
spin_unlock_irqrestore(&gsi->lock, flags);
}
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
int mlx5_ib_create_gsi(struct ib_pd *pd, struct mlx5_ib_qp *mqp,
struct ib_qp_init_attr *attr)
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
struct mlx5_ib_gsi_qp *gsi;
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
struct ib_qp_init_attr hw_init_attr = *attr;
const u8 port_num = attr->port_num;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
int num_qps = 0;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
int ret;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
if (mlx5_ib_deth_sqpn_cap(dev)) {
if (MLX5_CAP_GEN(dev->mdev,
port_type) == MLX5_CAP_PORT_TYPE_IB)
num_qps = pd->device->attrs.max_pkeys;
else if (dev->lag_active)
net/mlx5: Lag, expose number of lag ports Downstream patches will add support for hardware lag with more than 2 ports. Add a way for users to query the number of lag ports. Signed-off-by: Mark Bloch <mbloch@nvidia.com> Reviewed-by: Maor Gottlieb <maorg@nvidia.com> Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
2022-03-01 15:42:01 +00:00
num_qps = dev->lag_ports;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
}
RDMA/mlx5: Embed GSI QP into general mlx5_ib QP The GSI QPs have different create flow from the regular QPs, but it is not really needed. Update the code to use mlx5_ib_qp as a storage class for all outside of GSI calls. Link: https://lore.kernel.org/r/20200926102450.2966017-2-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:41 +03:00
gsi = &mqp->gsi;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
gsi->tx_qps = kcalloc(num_qps, sizeof(*gsi->tx_qps), GFP_KERNEL);
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
if (!gsi->tx_qps)
return -ENOMEM;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
gsi->outstanding_wrs =
kcalloc(attr->cap.max_send_wr, sizeof(*gsi->outstanding_wrs),
GFP_KERNEL);
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
if (!gsi->outstanding_wrs) {
ret = -ENOMEM;
goto err_free_tx;
}
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
if (dev->devr.ports[port_num - 1].gsi) {
mlx5_ib_warn(dev, "GSI QP already exists on port %d\n",
port_num);
ret = -EBUSY;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
goto err_free_wrs;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
}
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
gsi->num_qps = num_qps;
spin_lock_init(&gsi->lock);
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
gsi->cap = attr->cap;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
gsi->port_num = port_num;
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
gsi->cq = ib_alloc_cq(pd->device, gsi, attr->cap.max_send_wr, 0,
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
IB_POLL_SOFTIRQ);
if (IS_ERR(gsi->cq)) {
mlx5_ib_warn(dev, "unable to create send CQ for GSI QP. error %ld\n",
PTR_ERR(gsi->cq));
ret = PTR_ERR(gsi->cq);
goto err_free_wrs;
}
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
hw_init_attr.qp_type = MLX5_IB_QPT_HW_GSI;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
hw_init_attr.send_cq = gsi->cq;
IB/mlx5: Eliminate GSI RX QP's send buffers Now that the transmission of GSI MADs is done with the special transmission QPs, eliminate the send buffers in the GSI receive QP. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:11 +02:00
if (num_qps) {
hw_init_attr.cap.max_send_wr = 0;
hw_init_attr.cap.max_send_sge = 0;
hw_init_attr.cap.max_inline_data = 0;
}
RDMA/mlx5: Don't call to restrack recursively The restrack is going to manage memory of all IB objects and must be called before object is created. GSI QP in the mlx5_ib separated between creating dummy interface and HW object beneath. This was achieved by double call to ib_create_qp(). In order to skip such reentry call to internal driver create_qp code. Link: https://lore.kernel.org/r/20200922091106.2152715-3-leon@kernel.org Reviewed-by: Mark Zhang <markz@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-22 12:11:03 +03:00
RDMA/mlx5: Rework custom driver QP type creation Starting from commit 2b1f747071c5 ("RDMA/core: Allow drivers to disable restrack DB") the restrack is able to handle non-standard QP types either. That change allows us to rewrite custom QP calls to their IB/core counterparts, so we will use general QP creation flow even for the driver QP types. Link: https://lore.kernel.org/r/51682ab82298748941f38bd23ee3bf77ef1cab7b.1627040189.git.leonro@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-07-23 14:39:48 +03:00
gsi->rx_qp = ib_create_qp(pd, &hw_init_attr);
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
if (IS_ERR(gsi->rx_qp)) {
mlx5_ib_warn(dev, "unable to create hardware GSI QP. error %ld\n",
PTR_ERR(gsi->rx_qp));
ret = PTR_ERR(gsi->rx_qp);
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
goto err_destroy_cq;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
}
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
dev->devr.ports[attr->port_num - 1].gsi = gsi;
return 0;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
err_destroy_cq:
ib_free_cq(gsi->cq);
err_free_wrs:
kfree(gsi->outstanding_wrs);
err_free_tx:
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
kfree(gsi->tx_qps);
RDMA/mlx5: Change GSI QP to have same creation flow like other QPs There is no reason to have separate create flow for the GSI QP, while general create_qp routine has all needed checks and ability to allocate and free the proper struct mlx5_ib_qp. Link: https://lore.kernel.org/r/20200926102450.2966017-4-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:43 +03:00
return ret;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
}
RDMA/mlx5: Embed GSI QP into general mlx5_ib QP The GSI QPs have different create flow from the regular QPs, but it is not really needed. Update the code to use mlx5_ib_qp as a storage class for all outside of GSI calls. Link: https://lore.kernel.org/r/20200926102450.2966017-2-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:41 +03:00
int mlx5_ib_destroy_gsi(struct mlx5_ib_qp *mqp)
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
{
RDMA/mlx5: Embed GSI QP into general mlx5_ib QP The GSI QPs have different create flow from the regular QPs, but it is not really needed. Update the code to use mlx5_ib_qp as a storage class for all outside of GSI calls. Link: https://lore.kernel.org/r/20200926102450.2966017-2-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:41 +03:00
struct mlx5_ib_dev *dev = to_mdev(mqp->ibqp.device);
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
const int port_num = gsi->port_num;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
int qp_index;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
int ret;
RDMA/mlx5: Rework custom driver QP type creation Starting from commit 2b1f747071c5 ("RDMA/core: Allow drivers to disable restrack DB") the restrack is able to handle non-standard QP types either. That change allows us to rewrite custom QP calls to their IB/core counterparts, so we will use general QP creation flow even for the driver QP types. Link: https://lore.kernel.org/r/51682ab82298748941f38bd23ee3bf77ef1cab7b.1627040189.git.leonro@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-07-23 14:39:48 +03:00
ret = ib_destroy_qp(gsi->rx_qp);
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
if (ret) {
mlx5_ib_warn(dev, "unable to destroy hardware GSI QP. error %d\n",
ret);
return ret;
}
dev->devr.ports[port_num - 1].gsi = NULL;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
gsi->rx_qp = NULL;
for (qp_index = 0; qp_index < gsi->num_qps; ++qp_index) {
if (!gsi->tx_qps[qp_index])
continue;
WARN_ON_ONCE(ib_destroy_qp(gsi->tx_qps[qp_index]));
gsi->tx_qps[qp_index] = NULL;
}
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
ib_free_cq(gsi->cq);
kfree(gsi->outstanding_wrs);
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
kfree(gsi->tx_qps);
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
return 0;
}
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
static struct ib_qp *create_gsi_ud_qp(struct mlx5_ib_gsi_qp *gsi)
{
struct ib_pd *pd = gsi->rx_qp->pd;
struct ib_qp_init_attr init_attr = {
.event_handler = gsi->rx_qp->event_handler,
.qp_context = gsi->rx_qp->qp_context,
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
.send_cq = gsi->cq,
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
.recv_cq = gsi->rx_qp->recv_cq,
.cap = {
.max_send_wr = gsi->cap.max_send_wr,
.max_send_sge = gsi->cap.max_send_sge,
.max_inline_data = gsi->cap.max_inline_data,
},
.qp_type = IB_QPT_UD,
IB/mlx5: Align usage of QP1 create flags with rest of mlx5 defines There is little value in keeping separate function for one flag, provide it directly like any other mlx5 define. Link: https://lore.kernel.org/r/20191020064400.8344-2-leon@kernel.org Signed-off-by: Michael Guralnik <michaelgur@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-20 09:43:59 +03:00
.create_flags = MLX5_IB_QP_CREATE_SQPN_QP1,
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
};
return ib_create_qp(pd, &init_attr);
}
static int modify_to_rts(struct mlx5_ib_gsi_qp *gsi, struct ib_qp *qp,
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
u16 pkey_index)
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
{
struct mlx5_ib_dev *dev = to_mdev(qp->device);
struct ib_qp_attr attr;
int mask;
int ret;
mask = IB_QP_STATE | IB_QP_PKEY_INDEX | IB_QP_QKEY | IB_QP_PORT;
attr.qp_state = IB_QPS_INIT;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
attr.pkey_index = pkey_index;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
attr.qkey = IB_QP1_QKEY;
attr.port_num = gsi->port_num;
ret = ib_modify_qp(qp, &attr, mask);
if (ret) {
mlx5_ib_err(dev, "could not change QP%d state to INIT: %d\n",
qp->qp_num, ret);
return ret;
}
attr.qp_state = IB_QPS_RTR;
ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
if (ret) {
mlx5_ib_err(dev, "could not change QP%d state to RTR: %d\n",
qp->qp_num, ret);
return ret;
}
attr.qp_state = IB_QPS_RTS;
attr.sq_psn = 0;
ret = ib_modify_qp(qp, &attr, IB_QP_STATE | IB_QP_SQ_PSN);
if (ret) {
mlx5_ib_err(dev, "could not change QP%d state to RTS: %d\n",
qp->qp_num, ret);
return ret;
}
return 0;
}
static void setup_qp(struct mlx5_ib_gsi_qp *gsi, u16 qp_index)
{
struct ib_device *device = gsi->rx_qp->device;
struct mlx5_ib_dev *dev = to_mdev(device);
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
int pkey_index = qp_index;
struct mlx5_ib_qp *mqp;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
struct ib_qp *qp;
unsigned long flags;
u16 pkey;
int ret;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
if (MLX5_CAP_GEN(dev->mdev, port_type) != MLX5_CAP_PORT_TYPE_IB)
pkey_index = 0;
ret = ib_query_pkey(device, gsi->port_num, pkey_index, &pkey);
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
if (ret) {
mlx5_ib_warn(dev, "unable to read P_Key at port %d, index %d\n",
gsi->port_num, qp_index);
return;
}
if (!pkey) {
mlx5_ib_dbg(dev, "invalid P_Key at port %d, index %d. Skipping.\n",
gsi->port_num, qp_index);
return;
}
spin_lock_irqsave(&gsi->lock, flags);
qp = gsi->tx_qps[qp_index];
spin_unlock_irqrestore(&gsi->lock, flags);
if (qp) {
mlx5_ib_dbg(dev, "already existing GSI TX QP at port %d, index %d. Skipping\n",
gsi->port_num, qp_index);
return;
}
qp = create_gsi_ud_qp(gsi);
if (IS_ERR(qp)) {
mlx5_ib_warn(dev, "unable to create hardware UD QP for GSI: %ld\n",
PTR_ERR(qp));
return;
}
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
mqp = to_mqp(qp);
if (dev->lag_active)
mqp->gsi_lag_port = qp_index + 1;
ret = modify_to_rts(gsi, qp, pkey_index);
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
if (ret)
goto err_destroy_qp;
spin_lock_irqsave(&gsi->lock, flags);
WARN_ON_ONCE(gsi->tx_qps[qp_index]);
gsi->tx_qps[qp_index] = qp;
spin_unlock_irqrestore(&gsi->lock, flags);
return;
err_destroy_qp:
WARN_ON_ONCE(qp);
}
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
int mlx5_ib_gsi_modify_qp(struct ib_qp *qp, struct ib_qp_attr *attr,
int attr_mask)
{
struct mlx5_ib_dev *dev = to_mdev(qp->device);
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_qp *mqp = to_mqp(qp);
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
RDMA/mlx5: Delete device resource mutex that didn't protect anything The dev->devr.mutex was intended to protect GSI QP pointer change in the struct mlx5_ib_port_resources when it is accessed from the pkey_change_work. However that pointer isn't changed during the runtime and once IB/core adds MAD, it stays stable. Link: https://lore.kernel.org/r/6e338c561033df20d92e1371fc6a7a0d93aad945.1627040189.git.leonro@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-07-23 14:39:47 +03:00
u16 qp_index;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
int ret;
mlx5_ib_dbg(dev, "modifying GSI QP to state %d\n", attr->qp_state);
ret = ib_modify_qp(gsi->rx_qp, attr, attr_mask);
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
if (ret) {
mlx5_ib_warn(dev, "unable to modify GSI rx QP: %d\n", ret);
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
return ret;
IB/mlx5: Create multiple transmission GSI QPs In order to send GSI MADs on different P_Keys, mlx5 needs different QPs to be created, each with a different P_Key set when the QP is modified to the INIT state. Create QPs for each non-zero P_Key in the P_Key table. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:06 +02:00
}
RDMA/mlx5: Delete device resource mutex that didn't protect anything The dev->devr.mutex was intended to protect GSI QP pointer change in the struct mlx5_ib_port_resources when it is accessed from the pkey_change_work. However that pointer isn't changed during the runtime and once IB/core adds MAD, it stays stable. Link: https://lore.kernel.org/r/6e338c561033df20d92e1371fc6a7a0d93aad945.1627040189.git.leonro@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-07-23 14:39:47 +03:00
if (to_mqp(gsi->rx_qp)->state != IB_QPS_RTS)
return 0;
for (qp_index = 0; qp_index < gsi->num_qps; ++qp_index)
setup_qp(gsi, qp_index);
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
return 0;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
}
int mlx5_ib_gsi_query_qp(struct ib_qp *qp, struct ib_qp_attr *qp_attr,
int qp_attr_mask,
struct ib_qp_init_attr *qp_init_attr)
{
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_qp *mqp = to_mqp(qp);
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
int ret;
ret = ib_query_qp(gsi->rx_qp, qp_attr, qp_attr_mask, qp_init_attr);
qp_init_attr->cap = gsi->cap;
return ret;
}
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
/* Call with gsi->lock locked */
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
static int mlx5_ib_add_outstanding_wr(struct mlx5_ib_qp *mqp,
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
struct ib_ud_wr *wr, struct ib_wc *wc)
{
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
struct mlx5_ib_dev *dev = to_mdev(gsi->rx_qp->device);
struct mlx5_ib_gsi_wr *gsi_wr;
if (gsi->outstanding_pi == gsi->outstanding_ci + gsi->cap.max_send_wr) {
mlx5_ib_warn(dev, "no available GSI work request.\n");
return -ENOMEM;
}
IB/mlx5: Fix iteration overrun in GSI qps Number of outstanding_pi may overflow and as a result may indicate that there are no elements in the queue. The effect of doing this is that the MAD layer will get stuck waiting for completions. The MAD layer will think that the QP is full - because it didn't receive these completions. This fix changes it so the outstanding_pi number is increased with 32-bit wraparound and is not limited to max_send_wr so that the difference between outstanding_pi and outstanding_ci will really indicate the number of outstanding completions. Cc: Stable <stable@vger.kernel.org> Fixes: ea6dc2036224 ('IB/mlx5: Reorder GSI completions') Signed-off-by: Slava Shwartsman <slavash@mellanox.com> Signed-off-by: Leon Romanovsky <leon@kernel.org> Reviewed-by: Haggai Eran <haggaie@mellanox.com> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-07-03 15:28:19 +03:00
gsi_wr = &gsi->outstanding_wrs[gsi->outstanding_pi %
gsi->cap.max_send_wr];
gsi->outstanding_pi++;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
if (!wc) {
memset(&gsi_wr->wc, 0, sizeof(gsi_wr->wc));
gsi_wr->wc.pkey_index = wr->pkey_index;
gsi_wr->wc.wr_id = wr->wr.wr_id;
} else {
gsi_wr->wc = *wc;
gsi_wr->completed = true;
}
gsi_wr->cqe.done = &handle_single_completion;
wr->wr.wr_cqe = &gsi_wr->cqe;
return 0;
}
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
/* Call with gsi->lock locked */
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
static int mlx5_ib_gsi_silent_drop(struct mlx5_ib_qp *mqp, struct ib_ud_wr *wr)
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
{
struct ib_wc wc = {
{ .wr_id = wr->wr.wr_id },
.status = IB_WC_SUCCESS,
.opcode = IB_WC_SEND,
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
.qp = &mqp->ibqp,
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
};
int ret;
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
ret = mlx5_ib_add_outstanding_wr(mqp, wr, &wc);
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
if (ret)
return ret;
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
generate_completions(mqp);
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
return 0;
}
/* Call with gsi->lock locked */
static struct ib_qp *get_tx_qp(struct mlx5_ib_gsi_qp *gsi, struct ib_ud_wr *wr)
{
struct mlx5_ib_dev *dev = to_mdev(gsi->rx_qp->device);
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
struct mlx5_ib_ah *ah = to_mah(wr->ah);
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
int qp_index = wr->pkey_index;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
if (!gsi->num_qps)
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
return gsi->rx_qp;
RDMA/mlx5: Set lag tx affinity according to slave The patch sets the lag tx affinity of the data QPs and the GSI QPs according to the LAG xmit slave. For GSI QPs, in case the link layer is Ethenet (RoCE) we create two GSI QPs, one for each physical port. When the driver selects the GSI QP, it will consider the port affinity result. For connected QPs, the driver sets the affinity of the xmit slave. The above, ensures that RC QP and it's corresponding GSI QP will transmit from the same physical port. Link: https://lore.kernel.org/r/20200430192146.12863-17-maorg@mellanox.com Signed-off-by: Maor Gottlieb <maorg@mellanox.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-04-30 22:21:46 +03:00
if (dev->lag_active && ah->xmit_port)
qp_index = ah->xmit_port - 1;
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
if (qp_index >= gsi->num_qps)
return NULL;
return gsi->tx_qps[qp_index];
}
RDMA, core and ULPs: Declare ib_post_send() and ib_post_recv() arguments const Since neither ib_post_send() nor ib_post_recv() modify the data structure their second argument points at, declare that argument const. This change makes it necessary to declare the 'bad_wr' argument const too and also to modify all ULPs that call ib_post_send(), ib_post_recv() or ib_post_srq_recv(). This patch does not change any functionality but makes it possible for the compiler to verify whether the ib_post_(send|recv|srq_recv) really do not modify the posted work request. To make this possible, only one cast had to be introduce that casts away constness, namely in rpcrdma_post_recvs(). The only way I can think of to avoid that cast is to introduce an additional loop in that function or to change the data type of bad_wr from struct ib_recv_wr ** into int (an index that refers to an element in the work request list). However, both approaches would require even more extensive changes than this patch. Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Reviewed-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-18 09:25:32 -07:00
int mlx5_ib_gsi_post_send(struct ib_qp *qp, const struct ib_send_wr *wr,
const struct ib_send_wr **bad_wr)
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
{
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_qp *mqp = to_mqp(qp);
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
struct ib_qp *tx_qp;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
unsigned long flags;
int ret;
for (; wr; wr = wr->next) {
struct ib_ud_wr cur_wr = *ud_wr(wr);
cur_wr.wr.next = NULL;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
spin_lock_irqsave(&gsi->lock, flags);
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
tx_qp = get_tx_qp(gsi, &cur_wr);
if (!tx_qp) {
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
ret = mlx5_ib_gsi_silent_drop(mqp, &cur_wr);
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
if (ret)
goto err;
spin_unlock_irqrestore(&gsi->lock, flags);
continue;
}
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
ret = mlx5_ib_add_outstanding_wr(mqp, &cur_wr, NULL);
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
if (ret)
goto err;
IB/mlx5: Pick the right GSI transmission QP for sending Pick the QP to use according to the wr.ud.pkey_index field in the work request. If the QP doesn't exist, it means the P_Key is zero and the packet would have been dropped, so just generate a completion and move on. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:10 +02:00
ret = ib_post_send(tx_qp, &cur_wr.wr, bad_wr);
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
if (ret) {
/* Undo the effect of adding the outstanding wr */
IB/mlx5: Fix outstanding_pi index for GSI qps Commit b0ffeb537f3a ("IB/mlx5: Fix iteration overrun in GSI qps") changed the way outstanding WRs are tracked for the GSI QP. But the fix did not cover the case when a call to ib_post_send() fails and updates index to track outstanding. Since the prior commmit outstanding_pi should not be bounded otherwise the loop generate_completions() will fail. Fixes: b0ffeb537f3a ("IB/mlx5: Fix iteration overrun in GSI qps") Link: https://lore.kernel.org/r/1576195889-23527-1-git-send-email-psajeepa@purestorage.com Signed-off-by: Prabhath Sajeepa <psajeepa@purestorage.com> Acked-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-12-12 17:11:29 -07:00
gsi->outstanding_pi--;
IB/mlx5: Reorder GSI completions The emulated GSI QP's send completions are generated by multiple hardware QPs, so their completions could arrive out of order with respect to the order their work request were submitted. Reorder the completions by keeping a list of the posted work request and their completions. A newly received completion from the hardware updates the list and marks its work request as completed. However, the completions are only reported to the client according to the list order. In order to support that, create a new private CQ to handle the hardware completions. Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:09 +02:00
goto err;
}
spin_unlock_irqrestore(&gsi->lock, flags);
}
return 0;
err:
spin_unlock_irqrestore(&gsi->lock, flags);
*bad_wr = wr;
return ret;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
}
RDMA, core and ULPs: Declare ib_post_send() and ib_post_recv() arguments const Since neither ib_post_send() nor ib_post_recv() modify the data structure their second argument points at, declare that argument const. This change makes it necessary to declare the 'bad_wr' argument const too and also to modify all ULPs that call ib_post_send(), ib_post_recv() or ib_post_srq_recv(). This patch does not change any functionality but makes it possible for the compiler to verify whether the ib_post_(send|recv|srq_recv) really do not modify the posted work request. To make this possible, only one cast had to be introduce that casts away constness, namely in rpcrdma_post_recvs(). The only way I can think of to avoid that cast is to introduce an additional loop in that function or to change the data type of bad_wr from struct ib_recv_wr ** into int (an index that refers to an element in the work request list). However, both approaches would require even more extensive changes than this patch. Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Reviewed-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-18 09:25:32 -07:00
int mlx5_ib_gsi_post_recv(struct ib_qp *qp, const struct ib_recv_wr *wr,
const struct ib_recv_wr **bad_wr)
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
{
RDMA/mlx5: Reuse existing fields in parent QP storage object Remove duplication of mlx5_ib_qp and mlx5_ib_gsi_qp fields. This change returns the memory footprint of mlx5_ib QP to be as it was before embedding GSI QP. Link: https://lore.kernel.org/r/20200926102450.2966017-3-leon@kernel.org Reviewed-by: Maor Gottlieb <maorg@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2020-09-26 13:24:42 +03:00
struct mlx5_ib_qp *mqp = to_mqp(qp);
struct mlx5_ib_gsi_qp *gsi = &mqp->gsi;
IB/mlx5: Add GSI QP wrapper mlx5 creates special GSI QPs that has limited ability to control the P_Key of transmitted packets. The sent P_Key is taken from the QP object, similarly to what happens with regular UD QPs. Create a software wrapper around GSI QPs that with the following patches will be able to emulate the functionality of a GSI QP including control of the P_Key per work request. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:05 +02:00
return ib_post_recv(gsi->rx_qp, wr, bad_wr);
}
IB/mlx5: Create GSI transmission QPs when P_Key table is changed Whenever the P_Key table is changed, we create the required GSI transmission QPs on-demand. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:07 +02:00
void mlx5_ib_gsi_pkey_change(struct mlx5_ib_gsi_qp *gsi)
{
RDMA/mlx5: Delete device resource mutex that didn't protect anything The dev->devr.mutex was intended to protect GSI QP pointer change in the struct mlx5_ib_port_resources when it is accessed from the pkey_change_work. However that pointer isn't changed during the runtime and once IB/core adds MAD, it stays stable. Link: https://lore.kernel.org/r/6e338c561033df20d92e1371fc6a7a0d93aad945.1627040189.git.leonro@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-07-23 14:39:47 +03:00
u16 qp_index;
IB/mlx5: Create GSI transmission QPs when P_Key table is changed Whenever the P_Key table is changed, we create the required GSI transmission QPs on-demand. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:07 +02:00
RDMA/mlx5: Delete device resource mutex that didn't protect anything The dev->devr.mutex was intended to protect GSI QP pointer change in the struct mlx5_ib_port_resources when it is accessed from the pkey_change_work. However that pointer isn't changed during the runtime and once IB/core adds MAD, it stays stable. Link: https://lore.kernel.org/r/6e338c561033df20d92e1371fc6a7a0d93aad945.1627040189.git.leonro@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-07-23 14:39:47 +03:00
for (qp_index = 0; qp_index < gsi->num_qps; ++qp_index)
setup_qp(gsi, qp_index);
IB/mlx5: Create GSI transmission QPs when P_Key table is changed Whenever the P_Key table is changed, we create the required GSI transmission QPs on-demand. Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Haggai Eran <haggaie@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2016-02-29 15:45:07 +02:00
}
Reference in New Issue Copy Permalink