iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
linux/drivers/gpu/drm/i915/gvt/gtt.h

311 lines
8.7 KiB
C
Raw Normal View History

drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
/*
* Copyright(c) 2011-2016 Intel Corporation. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
* Authors:
* Zhi Wang <zhi.a.wang@intel.com>
* Zhenyu Wang <zhenyuw@linux.intel.com>
* Xiao Zheng <xiao.zheng@intel.com>
*
* Contributors:
* Min He <min.he@intel.com>
* Bing Niu <bing.niu@intel.com>
*
*/
#ifndef _GVT_GTT_H_
#define _GVT_GTT_H_
drm/i915/gvt: Use I915_GTT_PAGE_SIZE As there is already an I915_GTT_PAGE_SIZE marco in i915, let GVT-g use it as well. Also this patch re-names some GTT marcos with additional prefix. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-10-10 13:51:32 +08:00
#define I915_GTT_PAGE_SHIFT 12
#define I915_GTT_PAGE_MASK (~(I915_GTT_PAGE_SIZE - 1))
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct intel_vgpu_mm;
#define INTEL_GVT_GTT_HASH_BITS 8
#define INTEL_GVT_INVALID_ADDR (~0UL)
struct intel_gvt_gtt_entry {
u64 val64;
int type;
};
struct intel_gvt_gtt_pte_ops {
drm/i915/gvt: Add carefully checking in GTT walker paths When debugging the gtt code, found the intel_vgpu_gma_to_gpa() can translate any given GMA though the GMA is not valid. This because the GTT ops suppress the possible errors, which may result in an invalid PT entry is retrieved by upper caller. This patch changed the prototype of pte ops to propagate status to callers. Then we make sure the GTT walker stop as early as when a error is detected to prevent undefined behavior. Signed-off-by: Changbin Du <changbin.du@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2017-08-02 15:06:37 +08:00
int (*get_entry)(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index,
bool hypervisor_access,
unsigned long gpa,
struct intel_vgpu *vgpu);
int (*set_entry)(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index,
bool hypervisor_access,
unsigned long gpa,
struct intel_vgpu *vgpu);
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
bool (*test_present)(struct intel_gvt_gtt_entry *e);
void (*clear_present)(struct intel_gvt_gtt_entry *e);
bool (*test_pse)(struct intel_gvt_gtt_entry *e);
void (*set_pfn)(struct intel_gvt_gtt_entry *e, unsigned long pfn);
unsigned long (*get_pfn)(struct intel_gvt_gtt_entry *e);
};
struct intel_gvt_gtt_gma_ops {
unsigned long (*gma_to_ggtt_pte_index)(unsigned long gma);
unsigned long (*gma_to_pte_index)(unsigned long gma);
unsigned long (*gma_to_pde_index)(unsigned long gma);
unsigned long (*gma_to_l3_pdp_index)(unsigned long gma);
unsigned long (*gma_to_l4_pdp_index)(unsigned long gma);
unsigned long (*gma_to_pml4_index)(unsigned long gma);
};
struct intel_gvt_gtt {
struct intel_gvt_gtt_pte_ops *pte_ops;
struct intel_gvt_gtt_gma_ops *gma_ops;
int (*mm_alloc_page_table)(struct intel_vgpu_mm *mm);
void (*mm_free_page_table)(struct intel_vgpu_mm *mm);
struct list_head oos_page_use_list_head;
struct list_head oos_page_free_list_head;
struct list_head mm_lru_list_head;
drm/i915/gvt: reset the GGTT entry when vGPU created The GGTT space is partitioned between vGPUs, it could be reused by next vGPU after previous one is release, the stale entries need point to scratch page when vGPU created. v2: Reset logic move to vGPU create. v3: Correct the commit msg. v4: Move the reset function to vGPU init gtt function, as result it's no need explicitly in vGPU reset logic as vGPU init gtt called during reset. Signed-off-by: Ping Gao <ping.a.gao@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-12-08 10:14:48 +08:00
drm/i915/gvt: Let the caller choose if a shadow page should be put into hash table As we want to re-use intel_vgpu_shadow_page in buidling scrach page table and we don't want to put scrach page table page into hash table, a new param is introduced to give the caller a choice to decide if a shadow page should be put into hash table. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-10-10 14:34:11 +08:00
struct page *scratch_page;
unsigned long scratch_mfn;
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
};
enum {
INTEL_GVT_MM_GGTT = 0,
INTEL_GVT_MM_PPGTT,
};
drm/i915/gvt: implement scratch page table tree for shadow PPGTT All the unused entries in the page table tree(PML4E->PDPE->PDE->PTE) should point to scratch page table/scratch page to avoid page walk error due to the page prefetching. When removing an entry in shadow PPGTT, it need map to scratch page also, the older implementation use single scratch page to assign to all level entries, it doesn't align the page walk behavior when removed entry is in PML, PDP, PD. To avoid potential page walk error this patch implement a scratch page tree to replace the single scratch page. v2: more details in commit message address Kevin's comments. Signed-off-by: Ping Gao <ping.a.gao@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-11-04 13:47:35 +08:00
typedef enum {
GTT_TYPE_INVALID = -1,
GTT_TYPE_GGTT_PTE,
GTT_TYPE_PPGTT_PTE_4K_ENTRY,
GTT_TYPE_PPGTT_PTE_2M_ENTRY,
GTT_TYPE_PPGTT_PTE_1G_ENTRY,
GTT_TYPE_PPGTT_PTE_ENTRY,
GTT_TYPE_PPGTT_PDE_ENTRY,
GTT_TYPE_PPGTT_PDP_ENTRY,
GTT_TYPE_PPGTT_PML4_ENTRY,
GTT_TYPE_PPGTT_ROOT_ENTRY,
GTT_TYPE_PPGTT_ROOT_L3_ENTRY,
GTT_TYPE_PPGTT_ROOT_L4_ENTRY,
GTT_TYPE_PPGTT_ENTRY,
GTT_TYPE_PPGTT_PTE_PT,
GTT_TYPE_PPGTT_PDE_PT,
GTT_TYPE_PPGTT_PDP_PT,
GTT_TYPE_PPGTT_PML4_PT,
GTT_TYPE_MAX,
} intel_gvt_gtt_type_t;
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct intel_vgpu_mm {
int type;
bool initialized;
bool shadowed;
int page_table_entry_type;
u32 page_table_entry_size;
u32 page_table_entry_cnt;
void *virtual_page_table;
void *shadow_page_table;
int page_table_level;
bool has_shadow_page_table;
u32 pde_base_index;
struct list_head list;
struct kref ref;
atomic_t pincount;
struct list_head lru_list;
struct intel_vgpu *vgpu;
};
drm/i915/gvt: Add carefully checking in GTT walker paths When debugging the gtt code, found the intel_vgpu_gma_to_gpa() can translate any given GMA though the GMA is not valid. This because the GTT ops suppress the possible errors, which may result in an invalid PT entry is retrieved by upper caller. This patch changed the prototype of pte ops to propagate status to callers. Then we make sure the GTT walker stop as early as when a error is detected to prevent undefined behavior. Signed-off-by: Changbin Du <changbin.du@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2017-08-02 15:06:37 +08:00
extern int intel_vgpu_mm_get_entry(
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct intel_vgpu_mm *mm,
void *page_table, struct intel_gvt_gtt_entry *e,
unsigned long index);
drm/i915/gvt: Add carefully checking in GTT walker paths When debugging the gtt code, found the intel_vgpu_gma_to_gpa() can translate any given GMA though the GMA is not valid. This because the GTT ops suppress the possible errors, which may result in an invalid PT entry is retrieved by upper caller. This patch changed the prototype of pte ops to propagate status to callers. Then we make sure the GTT walker stop as early as when a error is detected to prevent undefined behavior. Signed-off-by: Changbin Du <changbin.du@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2017-08-02 15:06:37 +08:00
extern int intel_vgpu_mm_set_entry(
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct intel_vgpu_mm *mm,
void *page_table, struct intel_gvt_gtt_entry *e,
unsigned long index);
#define ggtt_get_guest_entry(mm, e, index) \
intel_vgpu_mm_get_entry(mm, mm->virtual_page_table, e, index)
#define ggtt_set_guest_entry(mm, e, index) \
intel_vgpu_mm_set_entry(mm, mm->virtual_page_table, e, index)
#define ggtt_get_shadow_entry(mm, e, index) \
intel_vgpu_mm_get_entry(mm, mm->shadow_page_table, e, index)
#define ggtt_set_shadow_entry(mm, e, index) \
intel_vgpu_mm_set_entry(mm, mm->shadow_page_table, e, index)
#define ppgtt_get_guest_root_entry(mm, e, index) \
intel_vgpu_mm_get_entry(mm, mm->virtual_page_table, e, index)
#define ppgtt_set_guest_root_entry(mm, e, index) \
intel_vgpu_mm_set_entry(mm, mm->virtual_page_table, e, index)
#define ppgtt_get_shadow_root_entry(mm, e, index) \
intel_vgpu_mm_get_entry(mm, mm->shadow_page_table, e, index)
#define ppgtt_set_shadow_root_entry(mm, e, index) \
intel_vgpu_mm_set_entry(mm, mm->shadow_page_table, e, index)
extern struct intel_vgpu_mm *intel_vgpu_create_mm(struct intel_vgpu *vgpu,
int mm_type, void *virtual_page_table, int page_table_level,
u32 pde_base_index);
extern void intel_vgpu_destroy_mm(struct kref *mm_ref);
struct intel_vgpu_guest_page;
drm/i915/gvt: implement scratch page table tree for shadow PPGTT All the unused entries in the page table tree(PML4E->PDPE->PDE->PTE) should point to scratch page table/scratch page to avoid page walk error due to the page prefetching. When removing an entry in shadow PPGTT, it need map to scratch page also, the older implementation use single scratch page to assign to all level entries, it doesn't align the page walk behavior when removed entry is in PML, PDP, PD. To avoid potential page walk error this patch implement a scratch page tree to replace the single scratch page. v2: more details in commit message address Kevin's comments. Signed-off-by: Ping Gao <ping.a.gao@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-11-04 13:47:35 +08:00
struct intel_vgpu_scratch_pt {
struct page *page;
unsigned long page_mfn;
};
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct intel_vgpu_gtt {
struct intel_vgpu_mm *ggtt_mm;
unsigned long active_ppgtt_mm_bitmap;
struct list_head mm_list_head;
DECLARE_HASHTABLE(shadow_page_hash_table, INTEL_GVT_GTT_HASH_BITS);
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
DECLARE_HASHTABLE(tracked_guest_page_hash_table, INTEL_GVT_GTT_HASH_BITS);
atomic_t n_tracked_guest_page;
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct list_head oos_page_list_head;
struct list_head post_shadow_list_head;
drm/i915/gvt: implement scratch page table tree for shadow PPGTT All the unused entries in the page table tree(PML4E->PDPE->PDE->PTE) should point to scratch page table/scratch page to avoid page walk error due to the page prefetching. When removing an entry in shadow PPGTT, it need map to scratch page also, the older implementation use single scratch page to assign to all level entries, it doesn't align the page walk behavior when removed entry is in PML, PDP, PD. To avoid potential page walk error this patch implement a scratch page tree to replace the single scratch page. v2: more details in commit message address Kevin's comments. Signed-off-by: Ping Gao <ping.a.gao@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-11-04 13:47:35 +08:00
struct intel_vgpu_scratch_pt scratch_pt[GTT_TYPE_MAX];
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
};
extern int intel_vgpu_init_gtt(struct intel_vgpu *vgpu);
extern void intel_vgpu_clean_gtt(struct intel_vgpu *vgpu);
drm/i915/gvt: reset the GGTT entry when vGPU created The GGTT space is partitioned between vGPUs, it could be reused by next vGPU after previous one is release, the stale entries need point to scratch page when vGPU created. v2: Reset logic move to vGPU create. v3: Correct the commit msg. v4: Move the reset function to vGPU init gtt function, as result it's no need explicitly in vGPU reset logic as vGPU init gtt called during reset. Signed-off-by: Ping Gao <ping.a.gao@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-12-08 10:14:48 +08:00
void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu);
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
extern int intel_gvt_init_gtt(struct intel_gvt *gvt);
drm/i915/gvt: Refine the intel_vgpu_reset_gtt reset function When doing the VGPU reset, we don't need to do the gtt/ppgtt reset. This will make the GVT to do the ppgtt shadow every time for a workload and caused really bad performance after a VGPU reset. This patch will make sure ppgtt clean only happen at device module level reset to fix this. Signed-off-by: Chuanxiao Dong <chuanxiao.dong@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2017-08-04 13:08:59 +08:00
void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu);
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
extern void intel_gvt_clean_gtt(struct intel_gvt *gvt);
extern struct intel_vgpu_mm *intel_gvt_find_ppgtt_mm(struct intel_vgpu *vgpu,
int page_table_level, void *root_entry);
struct intel_vgpu_oos_page;
struct intel_vgpu_shadow_page {
void *vaddr;
struct page *page;
int type;
struct hlist_node node;
unsigned long mfn;
};
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
struct intel_vgpu_page_track {
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct hlist_node node;
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
bool tracked;
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
unsigned long gfn;
int (*handler)(void *, u64, void *, int);
void *data;
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
};
struct intel_vgpu_guest_page {
struct intel_vgpu_page_track track;
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
unsigned long write_cnt;
struct intel_vgpu_oos_page *oos_page;
};
struct intel_vgpu_oos_page {
struct intel_vgpu_guest_page *guest_page;
struct list_head list;
struct list_head vm_list;
int id;
drm/i915/gvt: Use I915_GTT_PAGE_SIZE As there is already an I915_GTT_PAGE_SIZE marco in i915, let GVT-g use it as well. Also this patch re-names some GTT marcos with additional prefix. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-10-10 13:51:32 +08:00
unsigned char mem[I915_GTT_PAGE_SIZE];
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
};
#define GTT_ENTRY_NUM_IN_ONE_PAGE 512
struct intel_vgpu_ppgtt_spt {
struct intel_vgpu_shadow_page shadow_page;
struct intel_vgpu_guest_page guest_page;
int guest_page_type;
atomic_t refcount;
struct intel_vgpu *vgpu;
DECLARE_BITMAP(post_shadow_bitmap, GTT_ENTRY_NUM_IN_ONE_PAGE);
struct list_head post_shadow_list;
};
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
int intel_vgpu_init_page_track(struct intel_vgpu *vgpu,
struct intel_vgpu_page_track *t,
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
unsigned long gfn,
int (*handler)(void *gp, u64, void *, int),
void *data);
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
void intel_vgpu_clean_page_track(struct intel_vgpu *vgpu,
struct intel_vgpu_page_track *t);
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
drm/i915/gvt: Factor intel_vgpu_page_track As the data structure of "intel_vgpu_guest_page" will become much heavier in future, it's better to factor out the guest memory page track mechnisim as early as possible. Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>
2017-09-29 02:47:55 +08:00
struct intel_vgpu_page_track *intel_vgpu_find_tracked_page(
drm/i915/gvt: vGPU graphics memory virtualization The vGPU graphics memory emulation framework is responsible for graphics memory table virtualization. Under virtualization environment, a VM will populate the page table entry with guest page frame number(GPFN/GFN), while HW needs a page table filled with MFN(Machine frame number). The relationship between GFN and MFN(Machine frame number) is managed by hypervisor, while GEN HW doesn't have such knowledge to translate a GFN. To solve this gap, shadow GGTT/PPGTT page table is introdcued. For GGTT, the GFN inside the guest GGTT page table entry will be translated into MFN and written into physical GTT MMIO registers when guest write virtual GTT MMIO registers. For PPGTT, a shadow PPGTT page table will be created and write-protected translated from guest PPGTT page table. And the shadow page table root pointers will be written into the shadow context after a guest workload is shadowed. vGPU graphics memory emulation framework consists: - Per-GEN HW platform page table entry bits extract/de-extract routines. - GTT MMIO register emulation handlers, which will call hypercall to do GFN->MFN translation when guest write GTT MMIO register - PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync Signed-off-by: Zhi Wang <zhi.a.wang@intel.com> Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
2016-03-28 23:23:16 +08:00
struct intel_vgpu *vgpu, unsigned long gfn);
int intel_vgpu_sync_oos_pages(struct intel_vgpu *vgpu);
int intel_vgpu_flush_post_shadow(struct intel_vgpu *vgpu);
static inline void intel_gvt_mm_reference(struct intel_vgpu_mm *mm)
{
kref_get(&mm->ref);
}
static inline void intel_gvt_mm_unreference(struct intel_vgpu_mm *mm)
{
kref_put(&mm->ref, intel_vgpu_destroy_mm);
}
int intel_vgpu_pin_mm(struct intel_vgpu_mm *mm);
void intel_vgpu_unpin_mm(struct intel_vgpu_mm *mm);
unsigned long intel_vgpu_gma_to_gpa(struct intel_vgpu_mm *mm,
unsigned long gma);
struct intel_vgpu_mm *intel_vgpu_find_ppgtt_mm(struct intel_vgpu *vgpu,
int page_table_level, void *root_entry);
int intel_vgpu_g2v_create_ppgtt_mm(struct intel_vgpu *vgpu,
int page_table_level);
int intel_vgpu_g2v_destroy_ppgtt_mm(struct intel_vgpu *vgpu,
int page_table_level);
int intel_vgpu_emulate_gtt_mmio_read(struct intel_vgpu *vgpu,
unsigned int off, void *p_data, unsigned int bytes);
int intel_vgpu_emulate_gtt_mmio_write(struct intel_vgpu *vgpu,
unsigned int off, void *p_data, unsigned int bytes);
#endif /* _GVT_GTT_H_ */
Reference in New Issue Copy Permalink