Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NVMe/NVMeoF: Export nvfs register and unregister functions as GPL #10

Closed
Closed
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
1b17ae6
UBUNTU: [Packaging] Initialize linux-nvidia-6.5
ianmay81 Oct 3, 2023
b16faed
Revert "UBUNTU: SAUCE: modpost: support arbitrary symbol length in mo…
ianmay81 Oct 4, 2023
8635d59
UBUNTU: [Packaging] update variants
ianmay81 Oct 4, 2023
2a2f308
UBUNTU: [Packaging] update Ubuntu.md
ianmay81 Oct 4, 2023
8912883
UBUNTU: Start new release
ianmay81 Oct 4, 2023
855fe66
UBUNTU: [Config] nvidia-6.5: update annotations
ianmay81 Oct 4, 2023
189b65c
UBUNTU: Ubuntu-nvidia-6.5-6.5.0-1001.1
ianmay81 Oct 4, 2023
3aa0d62
UBUNTU: [Packaging] nvidia-6.5: disable rust support
ianmay81 Oct 12, 2023
1d06550
UBUNTU: Start new release
ianmay81 Oct 12, 2023
8b723cd
UBUNTU: link-to-tracker: update tracking bug
ianmay81 Oct 12, 2023
ce68dae
UBUNTU: [Config] nvidia-6.5: update annotations
ianmay81 Oct 12, 2023
f400f4e
UBUNTU: Ubuntu-nvidia-6.5-6.5.0-1004.4
ianmay81 Oct 12, 2023
508976f
UBUNTU: Start new release
piso77 Dec 11, 2023
a10b173
UBUNTU: rename debian.nvidia-6.6 to debian.nvidia
piso77 Dec 11, 2023
21c0669
UBUNTU: link-to-tracker: update tracking bug
piso77 Dec 11, 2023
c6f7452
UBUNTU: [Packaging] update variants
piso77 Dec 11, 2023
651fa11
UBUNTU: [Packaging] update update.conf
piso77 Dec 11, 2023
3e778f9
UBUNTU: [Packaging] move to gcc-13 by default
Jul 17, 2023
b50a190
UBUNTU: rebase on Ubuntu-6.6.0-14.14
piso77 Dec 11, 2023
d5fa6f0
UBUNTU: [Config] updateconfigs following Ubuntu-6.6.0-14.14 rebase
piso77 Dec 11, 2023
1c13bcf
UBUNTU: Ubuntu-nvidia-6.6.0-1001.1
piso77 Dec 11, 2023
27028de
UBUNTU: [Packaging] move to linux 6.8
Feb 27, 2024
3b22c0b
UBUNTU: update dropped.txt
Feb 27, 2024
6a39482
UBUNTU: Start new release
Feb 27, 2024
685541f
UBUNTU: link-to-tracker: update tracking bug
Feb 27, 2024
6f62cdc
UBUNTU: debian.nvidia/dkms-versions -- update from kernel-versions (m…
Feb 27, 2024
2a28d36
UBUNTU: [Packaging] add Rust build dependencies
Feb 27, 2024
81783f7
UBUNTU: [Config] update annotations after rebase to v6.8
Feb 27, 2024
4798782
UBUNTU: [Packaging] clean ABI check files
Feb 27, 2024
a7dcaa1
UBUNTU: Ubuntu-nvidia-6.8.0-1001.1
Feb 27, 2024
ed361c3
UBUNTU: Start new release
ianmay81 Mar 18, 2024
6bc833e
UBUNTU: link-to-tracker: update tracking bug
ianmay81 Mar 18, 2024
d6d3ea6
UBUNTU: [Config] nvidia: update annotations
ianmay81 Mar 18, 2024
1883882
UBUNTU: Ubuntu-nvidia-6.8.0-1002.2
ianmay81 Mar 18, 2024
07444b1
UBUNTU: [Packaging] dkms-versions standalone provides support
nvidia-bfigg Jan 5, 2024
9fb2f19
UBUNTU: [Packaging] add versioning to dkms standalone rprovides
ianmay81 Mar 21, 2024
4759dbc
NVIDIA: [Config]: Grouping AAEON config options together, under a com…
nvidia-bfigg Apr 4, 2024
f4b6311
NVIDIA: [Config]: Disable the NOUVEAU driver which is not used with -…
nvidia-bfigg Apr 4, 2024
b9ef9ff
NVIDIA: [Config]: Adding CORESIGHT and ARM64_ERRATUM configs to annot…
nvidia-bfigg Apr 5, 2024
db2f8e9
UBUNTU: [Config] update nvidia specific annotations with notes
ianmay81 Apr 10, 2024
d559487
UBUNTU: [Config] update annotations with updateconfigs
ianmay81 Apr 10, 2024
cce0e65
UBUNTU: [Packaging] remove tools host package
ianmay81 Apr 10, 2024
e17a18a
NVIDIA: SAUCE: Patch NFS driver to support GDS with 6.8 Kernel
sourabgupta3 Oct 30, 2023
b1da2dc
NVIDIA: SAUCE: NVMe/MVMEeOF: Patch NVMe/NVMeOF driver to support GDS …
sourabgupta3 Oct 30, 2023
99ece0b
NVIDIA: [Config] Add nvidia-fs build dependencies
nvidia-bfigg Mar 29, 2024
e36d7b1
UBUNTU: [Packaging] drop getabis data
ianmay81 Apr 15, 2024
97d9c91
UBUNTU: [Packaging] Replace fs/cifs with fs/smb in inclusion list
ianmay81 Apr 15, 2024
8718e41
UBUNTU: [Packaging] remove bindgen-0.56
ianmay81 Apr 15, 2024
0a4c8e3
UBUNTU: Start new release
ianmay81 Apr 15, 2024
5ba7fb7
UBUNTU: [Packaging] debian.nvidia/dkms-versions -- update from kernel…
ianmay81 Apr 10, 2024
9c45592
UBUNTU: link-to-tracker: update tracking bug
ianmay81 Apr 22, 2024
196705c
UBUNTU: Ubuntu-nvidia-6.8.0-1006.6
ianmay81 Apr 22, 2024
6a7cd7f
gpio: tegra186: Fix tegra186_gpio_is_accessible() check
Apr 24, 2024
b739174
arm64/mm: make set_ptes() robust when OAs cross 48-bit boundary
Jan 29, 2024
83683cb
arm/pgtable: define PFN_PTE_SHIFT
davidhildenbrand Jan 29, 2024
eca43c2
nios2/pgtable: define PFN_PTE_SHIFT
davidhildenbrand Jan 29, 2024
294c511
powerpc/pgtable: define PFN_PTE_SHIFT
davidhildenbrand Jan 29, 2024
a94b720
riscv/pgtable: define PFN_PTE_SHIFT
davidhildenbrand Jan 29, 2024
93b3718
s390/pgtable: define PFN_PTE_SHIFT
davidhildenbrand Jan 29, 2024
412c516
sparc/pgtable: define PFN_PTE_SHIFT
davidhildenbrand Jan 29, 2024
471e039
mm/pgtable: make pte_next_pfn() independent of set_ptes()
davidhildenbrand Jan 29, 2024
0a1dfdb
arm/mm: use pte_next_pfn() in set_ptes()
davidhildenbrand Jan 29, 2024
2b99f03
powerpc/mm: use pte_next_pfn() in set_ptes()
davidhildenbrand Jan 29, 2024
2b7d40c
mm/memory: factor out copying the actual PTE in copy_present_pte()
davidhildenbrand Jan 29, 2024
cd4ff8d
mm/memory: pass PTE to copy_present_pte()
davidhildenbrand Jan 29, 2024
b66bbc5
mm/memory: optimize fork() with PTE-mapped THP
davidhildenbrand Jan 29, 2024
f7acae8
mm/memory: ignore dirty/accessed/soft-dirty bits in folio_pte_batch()
davidhildenbrand Jan 29, 2024
1ee3a79
mm/memory: ignore writable bit in folio_pte_batch()
davidhildenbrand Jan 29, 2024
d549f56
mm: clarify the spec for set_ptes()
Feb 15, 2024
1c507af
mm: thp: batch-collapse PMD with set_ptes()
Feb 15, 2024
93e1fe8
mm: introduce pte_advance_pfn() and use for pte_next_pfn()
Feb 15, 2024
e83382a
arm64/mm: convert pte_next_pfn() to pte_advance_pfn()
Feb 15, 2024
7e321a1
x86/mm: convert pte_next_pfn() to pte_advance_pfn()
Feb 15, 2024
40eedae
mm: tidy up pte_next_pfn() definition
Feb 15, 2024
c66a4f2
arm64/mm: convert READ_ONCE(*ptep) to ptep_get(ptep)
Feb 15, 2024
69ca841
arm64/mm: convert set_pte_at() to set_ptes(..., 1)
Feb 15, 2024
0cf2dbe
arm64/mm: convert ptep_clear() to ptep_get_and_clear()
Feb 15, 2024
14b21c0
arm64/mm: new ptep layer to manage contig bit
Feb 15, 2024
bb072bc
arm64/mm: dplit __flush_tlb_range() to elide trailing DSB
Feb 15, 2024
21a8200
NVIDIA: [Config] arm64: ARM64_CONTPTE=y
Mar 27, 2024
eb0862c
arm64/mm: wire up PTE_CONT for user mappings
Feb 15, 2024
7ccca5c
arm64/mm: implement new wrprotect_ptes() batch API
Feb 15, 2024
0ddab60
arm64/mm: implement new [get_and_]clear_full_ptes() batch APIs
Feb 15, 2024
b9beb7a
mm: add pte_batch_hint() to reduce scanning in folio_pte_batch()
Feb 15, 2024
2d583c7
arm64/mm: implement pte_batch_hint()
Feb 15, 2024
266113f
arm64/mm: __always_inline to improve fork() perf
Feb 15, 2024
d95b60d
arm64/mm: automatically fold contpte mappings
Feb 15, 2024
b333d9d
arm64/mm: export contpte symbols only to GPL users
Feb 26, 2024
846cdeb
arm64/mm: improve comment in contpte_ptep_get_lockless()
Feb 26, 2024
89dbe2f
NVIDIA: [Packaging] update nvidia-fs driver to latest version
nvidia-bfigg May 23, 2024
0eda9f4
UBUNTU: [Packaging] blacklist coresight_etm4x
ianmay81 Apr 25, 2024
6f18a98
tpm_tis_spi: Account for SPI header when allocating TPM SPI xfer buffer
nvmochs May 22, 2024
5e556d2
UBUNTU: [Packaging] update Ubuntu.md
jacobmartin0 May 30, 2024
1835bae
UBUNTU: Start new release
jacobmartin0 May 30, 2024
5b3a63f
UBUNTU: [Packaging] enable perf python module
Mar 14, 2024
d9f6493
UBUNTU: [Packaging] add Real-time Linux Analysis tool (rtla) to linux…
Mar 26, 2024
8181674
UBUNTU: [Packaging] update dependencies for rtla
Apr 9, 2024
9002939
UBUNTU: link-to-tracker: update tracking bug
jacobmartin0 May 30, 2024
4e6dbea
UBUNTU: [Packaging] debian.nvidia/dkms-versions -- update from kernel…
jacobmartin0 May 30, 2024
3928124
UBUNTU: Ubuntu-nvidia-6.8.0-1007.7
jacobmartin0 May 30, 2024
8c379e3
NVMe/NVMeoF: Export nvfs register and unregister functions as GPL
sourabgupta3 Jun 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
NVIDIA: SAUCE: Patch NFS driver to support GDS with 6.8 Kernel
BugLink: https://bugs.launchpad.net/bugs/2059814

With this change, the NFS driver would be enabled to support GPUDirectStorage(GDS).
The change is around frwr_map and frwr_unmap in the NFS driver, where the IO request
is first intercepted to check for GDS pages and if it is a GDS page then the
request is served by GDS driver component called nvidia-fs,
else the request would be served by the standard NFS driver code.

Signed-off-by: Sourab Gupta <sougupta@nvidia.com>
Acked-by: Brad Figg <bfigg@nvidia.com>
Acked-by: Ian May <ian.may@canonical.com>
Signed-off-by: Ian May <ian.may@canonical.com>
sourabgupta3 authored and jacobmartin0 committed May 30, 2024
commit e17a18a348d95d5d37f87e82fafebf405c6331dd
2 changes: 2 additions & 0 deletions net/sunrpc/xprtrdma/Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# SPDX-License-Identifier: GPL-2.0
ccflags-y += -DCONFIG_NVFS
obj-$(CONFIG_SUNRPC_XPRT_RDMA) += rpcrdma.o

rpcrdma-y := transport.o rpc_rdma.o verbs.o frwr_ops.o \
svc_rdma.o svc_rdma_backchannel.o svc_rdma_transport.o \
svc_rdma_sendto.o svc_rdma_recvfrom.o svc_rdma_rw.o \
svc_rdma_pcl.o module.o
rpcrdma-y += nvfs_rpc_rdma.o
rpcrdma-$(CONFIG_SUNRPC_BACKCHANNEL) += backchannel.o
35 changes: 31 additions & 4 deletions net/sunrpc/xprtrdma/frwr_ops.c
Original file line number Diff line number Diff line change
@@ -44,6 +44,11 @@

#include "xprt_rdma.h"
#include <trace/events/rpcrdma.h>
#ifdef CONFIG_NVFS
#define NVFS_FRWR
#include "nvfs.h"
#include "nvfs_rpc_rdma.h"
#endif

static void frwr_cid_init(struct rpcrdma_ep *ep,
struct rpcrdma_mr *mr)
@@ -58,6 +63,13 @@ static void frwr_mr_unmap(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr *mr)
{
if (mr->mr_device) {
trace_xprtrdma_mr_unmap(mr);
#ifdef CONFIG_NVFS
if (rpcrdma_nvfs_unmap_data(mr->mr_device->dma_device,
mr->mr_sg, mr->mr_nents, mr->mr_dir))
pr_debug("rpcrdma_nvfs_unmap_data device %s mr->mr_sg: %p , nents: %d\n",
mr->mr_device->name, mr->mr_sg, mr->mr_nents);
else
#endif
ib_dma_unmap_sg(mr->mr_device, mr->mr_sg, mr->mr_nents,
mr->mr_dir);
mr->mr_device = NULL;
@@ -286,6 +298,9 @@ struct rpcrdma_mr_seg *frwr_map(struct rpcrdma_xprt *r_xprt,
int nsegs, bool writing, __be32 xid,
struct rpcrdma_mr *mr)
{
#ifdef CONFIG_NVFS
bool is_nvfs_io = false;
#endif
struct rpcrdma_ep *ep = r_xprt->rx_ep;
struct ib_reg_wr *reg_wr;
int i, n, dma_nents;
@@ -308,11 +323,23 @@ struct rpcrdma_mr_seg *frwr_map(struct rpcrdma_xprt *r_xprt,
}
mr->mr_dir = rpcrdma_data_dir(writing);
mr->mr_nents = i;

dma_nents = ib_dma_map_sg(ep->re_id->device, mr->mr_sg, mr->mr_nents,
mr->mr_dir);
if (!dma_nents)
#ifdef CONFIG_NVFS
dma_nents = rpcrdma_nvfs_map_data(ep->re_id->device->dma_device,
mr->mr_sg, i, mr->mr_dir,
&is_nvfs_io);
if (dma_nents == -EIO) {
goto out_dmamap_err;
} else if (is_nvfs_io) {
pr_debug("rpcrdma_nvfs_map_data device %s mr->mr_sg: %p , nents: %d\n",
ep->re_id->device->name, mr->mr_sg, mr->mr_nents);
} else
#endif
{
dma_nents = ib_dma_map_sg(ep->re_id->device, mr->mr_sg, mr->mr_nents,
mr->mr_dir);
if (!dma_nents)
goto out_dmamap_err;
}
mr->mr_device = ep->re_id->device;

ibmr = mr->mr_ibmr;
113 changes: 113 additions & 0 deletions net/sunrpc/xprtrdma/nvfs.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms and conditions of the GNU General Public License,
* version 2, as published by the Free Software Foundation.
*
* This program is distributed in the hope it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*/

#ifndef NVFS_H
#define NVFS_H

#include <linux/types.h>
#include <linux/delay.h>
#include <linux/blkdev.h>
#include <linux/cpumask.h>
#include <linux/scatterlist.h>
#include <linux/percpu-defs.h>
#include <linux/dma-direction.h>

#define REGSTR2(x) x##_register_nvfs_dma_ops
#define REGSTR(x) REGSTR2(x)

#define UNREGSTR2(x) x##_unregister_nvfs_dma_ops
#define UNREGSTR(x) UNREGSTR2(x)

#define REGISTER_FUNC REGSTR(MODULE_PREFIX)
#define UNREGISTER_FUNC UNREGSTR(MODULE_PREFIX)

#define NVFS_IO_ERR -1
#define NVFS_CPU_REQ -2

#define NVFS_HOLD_TIME_MS 1000

extern struct nvfs_dma_rw_ops *nvfs_ops;

extern atomic_t nvfs_shutdown;

DECLARE_PER_CPU(long, nvfs_n_ops);

static inline long nvfs_count_ops(void)
{
int i;
long sum = 0;

for_each_possible_cpu(i)
sum += per_cpu(nvfs_n_ops, i);
return sum;
}

static inline bool nvfs_get_ops(void)
{
if (nvfs_ops && !atomic_read(&nvfs_shutdown)) {
this_cpu_inc(nvfs_n_ops);
return true;
}
return false;
}

static inline void nvfs_put_ops(void)
{
this_cpu_dec(nvfs_n_ops);
}

struct nvfs_dma_rw_ops {
unsigned long long ft_bmap; // feature bitmap

int (*nvfs_blk_rq_map_sg)(struct request_queue *q,
struct request *req,
struct scatterlist *sglist);

int (*nvfs_dma_map_sg_attrs)(struct device *device,
struct scatterlist *sglist,
int nents,
enum dma_data_direction dma_dir,
unsigned long attrs);

int (*nvfs_dma_unmap_sg)(struct device *device,
struct scatterlist *sglist,
int nents,
enum dma_data_direction dma_dir);

bool (*nvfs_is_gpu_page)(struct page *page);

unsigned int (*nvfs_gpu_index)(struct page *page);

unsigned int (*nvfs_device_priority)(struct device *dev, unsigned int gpu_index);
};

// feature list for dma_ops, values indicate bit pos
enum ft_bits {
nvfs_ft_prep_sglist = 1ULL << 0,
nvfs_ft_map_sglist = 1ULL << 1,
nvfs_ft_is_gpu_page = 1ULL << 2,
nvfs_ft_device_priority = 1ULL << 3,
};

// check features for use in registration with vendor drivers
#define NVIDIA_FS_CHECK_FT_SGLIST_PREP(ops) ((ops)->ft_bmap & nvfs_ft_prep_sglist)
#define NVIDIA_FS_CHECK_FT_SGLIST_DMA(ops) ((ops)->ft_bmap & nvfs_ft_map_sglist)
#define NVIDIA_FS_CHECK_FT_GPU_PAGE(ops) ((ops)->ft_bmap & nvfs_ft_is_gpu_page)
#define NVIDIA_FS_CHECK_FT_DEVICE_PRIORITY(ops) ((ops)->ft_bmap & nvfs_ft_device_priority)

int REGISTER_FUNC(struct nvfs_dma_rw_ops *ops);

void UNREGISTER_FUNC(void);

#endif /* NVFS_H */
51 changes: 51 additions & 0 deletions net/sunrpc/xprtrdma/nvfs_rpc_rdma.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms and conditions of the GNU General Public License,
* version 2, as published by the Free Software Foundation.
*
* This program is distributed in the hope it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*/

#ifdef CONFIG_NVFS
#define MODULE_PREFIX rpcrdma
#include "nvfs.h"

struct nvfs_dma_rw_ops *nvfs_ops;

atomic_t nvfs_shutdown = ATOMIC_INIT(1);

DEFINE_PER_CPU(long, nvfs_n_ops);

// must have for compatibility
#define NVIDIA_FS_COMPAT_FT(ops) \
((NVIDIA_FS_CHECK_FT_SGLIST_PREP(ops)) && (NVIDIA_FS_CHECK_FT_SGLIST_DMA(ops)))

// protected via nvfs_module_mutex
int REGISTER_FUNC(struct nvfs_dma_rw_ops *ops)
{
if (NVIDIA_FS_COMPAT_FT(ops)) {
nvfs_ops = ops;
atomic_set(&nvfs_shutdown, 0);
return 0;
}
return -EOPNOTSUPP;
}
EXPORT_SYMBOL(REGISTER_FUNC);

// protected via nvfs_module_mutex
void UNREGISTER_FUNC(void)
{
(void)atomic_cmpxchg(&nvfs_shutdown, 0, 1);
do {
msleep(NVFS_HOLD_TIME_MS);
} while (nvfs_count_ops());
nvfs_ops = NULL;
}
EXPORT_SYMBOL(UNREGISTER_FUNC);
#endif
65 changes: 65 additions & 0 deletions net/sunrpc/xprtrdma/nvfs_rpc_rdma.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms and conditions of the GNU General Public License,
* version 2, as published by the Free Software Foundation.
*
* This program is distributed in the hope it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*/

#ifndef NVFS_RPCRDMA_H
#define NVFS_RPCRDMA_H

#ifdef NVFS_FRWR
static int rpcrdma_nvfs_map_data(struct device *dev, struct scatterlist *sg,
int nents, enum dma_data_direction dma_dir,
bool *is_nvfs_io)
{
int count;

*is_nvfs_io = false;
count = 0;
if (nvfs_get_ops()) {
count = nvfs_ops->nvfs_dma_map_sg_attrs(dev,
sg,
nents,
dma_dir,
DMA_ATTR_NO_WARN);

if (unlikely(count == NVFS_IO_ERR)) {
nvfs_put_ops();
return -EIO;
}

if (unlikely(count == NVFS_CPU_REQ)) {
nvfs_put_ops();
return 0;
}
*is_nvfs_io = true;
}
return count;
}
#endif

static bool rpcrdma_nvfs_unmap_data(struct device *dev, struct scatterlist *sg,
int nents, enum dma_data_direction dma_dir)
{
int count;

if (nvfs_ops != NULL) {
count = nvfs_ops->nvfs_dma_unmap_sg(dev, sg, nents,
dma_dir);
if (count > 0) {
nvfs_put_ops();
return true;
}
}
return false;
}

#endif /* NVFS_RPCRDMA_H */