1137036 Commits

Author SHA1 Message Date
Ben Skeggs
e3f324956a drm/nouveau/fb/gp102-: unlock VPR right after devinit
Under memory load, instmem allocations could end up in the regions of
VRAM that are inaccessible right after boot, and be corrupted after a
suspend/resume cycle as a result of being restored before booting the
mem unlock firmware.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
5728d06419 drm/nouveau/fb: handle sysmem flush page from common code
- also executes pre-DEVINIT, so early boot is able to DMA sysmem

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
0e44c21708 drm/nouveau/flcn: new code to load+boot simple HS FWs (VPR scrubber)
Adds the start of common interfaces to load and boot the HS binaries
provided by NVIDIA that enable the usage of GR.

ACR already handles most of this, but it's very much tied into ACR's
init process, and there's other code that could benefit from reusing
a lot of this stuff too (ie. VBIOS DEVINIT/PreOS, VPR scrubber).

The VPR scrubber code is fairly independent, and a good first target.

- adds better debug output to fw loading process, to ease bring-up/debug

v2:
- whitespace, 0->false

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
f15cde64b6 drm/nouveau/flcn: rework falcon reset
Mostly preparation to fit in Ampere changes, but should result in reset
sequences a lot closer to RM's, and perhaps help out with the issues we
sometimes see reported in this area.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
c7c0aac742 drm/nouveau/sec2: switch to newer style interrupt handler
Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
3b330f0875 drm/nouveau/sec2: unload RTOS before tearing down WPR
Reset regs won't be available on Ampere while SEC2 RTOS is running, and
we're apparently supposed to be doing this on earlier GPUs too.

v2:
- fixed some excessive indentation

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
a9d90860c4 drm/nouveau/pmu/gm20b,gp10b: boot RTOS from PMU init
Cleanup before falcon changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:58 +10:00
Ben Skeggs
ccdc043123 drm/nouveau/pmu: move init() falcon reset to non-nvfw code
Cleanup before falcon changes.

- fixes (attempt at?) reset of pmu while rtos is running, on gm20b

v2:
- remove extra whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:57 +10:00
Ben Skeggs
b7f44ef732 drm/nouveau/pmu: move preinit() falcon reset to devinit
Cleanup before falcon changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:57 +10:00
Ben Skeggs
74f9dcb0df drm/nouveau/gsp: add funcs
Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:57 +10:00
Ben Skeggs
7f4f35ea5b drm/nouveau/fifo/ga100-: initial support
- replaces the hacked-up version that existed solely to support TTM

v2. remove earlier hack preventing use of non-stall intr for fences

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
2022-11-09 10:44:50 +10:00
Ben Skeggs
05d271c32e drm/nouveau/ce/ga100-: initial support
- replaces the hacked-up version that existed solely to support TTM
- noop until the next commit, adding proper support for ampere host

v2. fixup for ga103 early merge

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
2022-11-09 10:44:50 +10:00
Ben Skeggs
06db7fded6 drm/nouveau/fifo: add new channel classes
Exposes a bunch of the new features that became possible as a result
of the earlier commits.  DRM will build on this in the future to add
support for features such as SCG ("async compute") and multi-device
rendering, as part of the work necessary to be able to write a half-
decent vulkan driver - finally.

For the moment, this just crudely ports DRM to the API changes.

- channel class interfaces now the same for all HW classes
- channel group class exposed (SCG)
- channel runqueue selector exposed (SCG)
- channel sub-device id control exposed (multi-device rendering)
- channel names in logging will reflect creating process, not fd owner
- explicit USERD allocation required by VOLTA_CHANNEL_GPFIFO_A and newer
- drm is smarter about determining the appropriate channel class to use

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:50 +10:00
Ben Skeggs
7ac2933281 drm/nouveau/fifo: add new engine object handling
Simplifies the GPU-specific code, completing the switch to newer HALs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
8ab849d6dd drm/nouveau/fifo: add new engine context handling
Builds on the context tracking that was added earlier.

- marks engine context PTEs as 'priv' where possible

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
3647c53bd7 drm/nouveau/fifo: add RAMFC info to nvkm_chan_func
- adds support for specifying SUBDEVICE_ID for channel
- rounds non-power-of-two GPFIFO sizes down, rather than up

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
fbe9f4337c drm/nouveau/fifo: add USERD info to nvkm_chan_func
And use it to cleanup multiple implementations of almost the same thing.

- prepares for non-polled / client-provided USERD
- only zeroes relevant "registers", rather than entire USERD

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
d3e7a4392c drm/nouveau/fifo: add RAMIN info to nvkm_chan_func
Currently provided by {chan,dma,gpfifo}*.c, and those are going away.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
b084fff210 drm/nouveau/fifo: add common runlist control
- less dependence on waiting for runlist updates, on GPUs that allow it
- supports runqueue selector in RAMRL entries
- completes switch to common runl/cgrp/chan topology info

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
4d60100a23 drm/nouveau/fifo: add common channel recovery
That sure was fun to untangle.

- handled per-runlist, rather than globally
- more straight-forward process in general
- various potential SW/HW races have been fixed
- fixes lockdep issues that were present in >=gk104's prior implementation
- volta recovery now actually stands a chance of working
- volta/turing waiting for PBDMA idle before engine reset
- turing using hw-provided TSG info for CTXSW_TIMEOUT

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
0b1bb1296f drm/nouveau/fifo: kill channel on NV_PPBDMA_INTR_1_CTXNOTVALID
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
520db0405e drm/nouveau/fifo: kill channel on a selection of PBDMA errors
A bunch of these can be handled in such a way that the channel can
continue, however, any of these are a pretty decent sign something
has gone horribly wrong, and the safest option is to disable the
channel.

This is a bit of a hack, we will want to handle these individually
and dump relevant debug info for each at some point.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
acff941535 drm/nouveau/fifo: add chan/cgrp preempt()
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:49 +10:00
Ben Skeggs
67059b9fb8 drm/nouveau/fifo: add chan start()/stop()
- nvkm_chan_error() built on top, stops channel and sends 'killed' event
- removes an odd double-bashing of channel enable regs on kepler and up
- pokes doorbell on turing and up, after enabling channel

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
62742b5ef3 drm/nouveau/fifo: add chan bind()/unbind()
- stops programming (non-existent) runl id field on bind(), from maxwell

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
3a6bc9c242 drm/nouveau/fifo: add runlist block()/allow()
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
4a492fd5d2 drm/nouveau/fifo: add runlist wait()
- adds g8x/turing registers, which were missing before
- switches fermi to polled wait, like later hw (see: 4f2fc25c0f8bc...)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
f48dd29361 drm/nouveau/fifo: add new engine context tracking
Channel groups have somewhat more complicated requirements than what we
currently support.  An engine context is shared between all channels in
a channel group, VEID/subctx support (later) brings per-VEID components,
and we need to track an individual channel's engine context pointers.

This commit adds the structures and refcounting to support the above,
wrapping the prior implementation for the moment.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
c358f53871 drm/nouveau/fifo: add new channel lookup interfaces
- supports per-runlist CHIDs
- channel group lock held across reference, rather than global lock

v2:
- remove unnecessary parenthesis

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
e43c872c03 drm/nouveau/fifo: merge mmu fault handlers together
After updating GF100 implementation from the GK104/TU102 ones, and using
the new runlist/engine topology info, all three handlers become (almost)
identical.

- there's a temporary kludge to call through to the HW-specific recovery
- engine fault mapping info determined at load time, not on every fault

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
923f1ff527 drm/nouveau/fifo: move PBDMA intr to runq
- merges gf100/gk104- NV_PFIFO_INTR_0_PBDMA and NV_PPBDMA_INTR_0 code

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
87c8602431 drm/nouveau/fifo: move PBDMA init to runq
- bumps pbdma timeout to value RM uses on newer HW
- bumps fb timeout to max from boot default
- one/both of these greatly improves stability on // piglit runs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
324176e7c8 drm/nouveau/fifo: program NV_PFIFO_FB_TIMEOUT on init
NVGPU and RM both program this value.

Fixes a bunch of random hangs running parallel piglit.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
965c41d911 drm/nouveau/fifo: tidy global PBDMA init
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
2022-11-09 10:44:48 +10:00
Ben Skeggs
d67f3b9646 drm/nouveau/fifo: tidy up non-stall intr handling
- removes a layer of indirection in the intr handling
- prevents non-stall ctrl racing with unknown intrs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
2fc71a0566 drm/nouveau/fifo: use explicit intr interfaces
More control, and shallower call-chain to get to the point.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
0fc72ee9d8 drm/nouveau/fifo: use runlist engine info to lookup engine classes
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
468fae7bb0 drm/nouveau/fifo: add cgrp, have all channels be part of one
Engine context tracking will move to nvkm_cgrp in later commits, so we
create SW-only channel groups on HW without support for them.

- switches to nvkm_chid for TSG/channel ID allocation

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
eb39c61348 drm/nouveau/fifo: expose per-runlist CHID information
DRM uses this to setup fence-related items.

- nouveau_chan.runlist will always be "0" for the moment, not an issue
  as GPUs prior to ampere have system-wide channel IDs,

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
6de125383a drm/nouveau/fifo: expose runlist topology info on all chipsets
Previously only available from Kepler onwards.

- also fixes the info() queries causing fifo init()/fini() unnecessarily

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
d94470e9d1 drm/nouveau/fifo: add common runlist/engine topology
Creates an nvkm_runl for each runlist on the GPU, and an nvkm_engn for
each engine that is reachable from a runlist.

- basically what gk104- already does, but extended to all chips
- adds per-runlist CHID allocators (Ampere)
- splits g98/gt2xx out from g84 (different target engines)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
1c488ba96e drm/nouveau/fifo: add runq
Creates an nvkm_runq for each PBDMA, these will be associated with the
relevant runlist(s) later.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
800ac1f8d7 drm/nouveau/fifo: add chid allocator
We need to be able to allocate TSG IDs as well as channel IDs, also,
Ampere has per-runlist channel IDs.

- holds per-ID private data, which will be used for/to protect lookup
- holds an nvkm_event which will be used for events tied to IDs
- not used yet beyond setup, and switching use of "fifo->nr - 1" for
  channel ID mask to "chid->mask"

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
9be9c606c4 drm/nouveau/fifo: merge gk104_fifo_func into nvkm_host_func
This makes it easier to transition everything.

- a couple of function renames for collisions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
fd67738a28 drm/nouveau/fifo: pre-move some blocks of code around
- will make subsequent patches more obvious
- no code changes

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:47 +10:00
Ben Skeggs
f5e4568991 drm/nouveau/fifo: unify handling of channel classes
Adds the basic skeleton for common channel (group) interfaces.

- common behaviour between <gk104 and >=gk104 impl's
- separates priv/user channel objects
- passthrough to existing object for now, kludges removed later

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:46 +10:00
Ben Skeggs
8c18138c76 drm/nouveau/fifo: add chid_nr()
- reads channel count from GPU from gm200 onwards
- removes gm20b/gp10b (they become identical to gm200/gp100)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:46 +10:00
Ben Skeggs
973b32443b drm/nouveau/imem: allow bar2 mapping of user allocations
Will be used to init client-allocated USERD to default values.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:46 +10:00
Ben Skeggs
e442f1e453 drm/nouveau/flcn: show falcon user in debug output
Displays both owner/user of the falcon (when they differ), and takes
both subdevs' debug levels into account when deciding whether to log
the message.

- runlist debugging will use one of the alternate macros added here

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:46 +10:00
Ben Skeggs
8478cd5a74 drm/nouveau/nvkm: add locking to subdev/engine init paths
This wasn't really needed before; the main place this could race is with
channel recovery, but (through potentially fragile means) shouldn't have
been possible.

However, a number of upcoming patches benefit from having better control
over subdev init, necessitating some improvements here.

- allows subdev/engine oneinit() without init() (host/fifo patches)
- merges engine use locking/tracking into subdev, and extends it to fix
  some issues that will arise with future usage patterns (acr patches)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
2022-11-09 10:44:46 +10:00