summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/i915/i915_gem_execbuffer.c
AgeCommit message (Collapse)Author
2017-07-27drm/i915: Force CPU synchronisation even if userspace requests ASYNCChris Wilson
The goal here was to minimise doing any thing or any check inside the kernel that was not strictly required. For a userspace that assumes complete control over the cache domains, the kernel is usually using outdated information and may trigger clflushes where none were required. However, swapping is a situation where userspace has no knowledge of the domain transfer, and will leave the object in the CPU cache. The kernel must flush this out to the backing storage prior to use with the GPU. As we use an asynchronous task tracked by an implicit fence for this, we also need to cancel the ASYNC flag on the object so that the object will wait for the clflush to complete before being executed. This also absolves userspace of the responsibility imposed by commit 77ae9957897d ("drm/i915: Enable userspace to opt-out of implicit fencing") that its needed to ensure that the object was out of the CPU cache prior to use on the GPU. Fixes: 77ae9957897d ("drm/i915: Enable userspace to opt-out of implicit fencing") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101571 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Link: https://patchwork.freedesktop.org/patch/msgid/20170721145037.25105-5-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> (cherry picked from commit 0f46daa1a273779a0b73d768a788ca3f04238f9c) Cc: stable@vger.kernel.org Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-07-27drm/i915: Only skip updating execobject.offset after errorChris Wilson
I was being overly paranoid in not updating the execobject.offset after performing the fallback copy where we set reloc.presumed_offset to -1. The thinking was to ensure that a subsequent NORELOC execbuf would be forced to process the invalid relocations. However this is overkill so long as we *only* update the execobject.offset following a successful update of the relocation value witin the batch. If we have to repeat the execbuf due to a later interruption, then we may skip the relocations on the second pass (honouring NORELOC) since the execobject.offset match the actual offsets (even though reloc.presumed_offset is garbage). Subsequent calls to execbuf with NORELOC should themselves ensure that the reloc.presumed_offset have been corrected in case of future migration. Reporting back the actual execobject.offset, even when reloc.presumed_offset is garbage, ensures that reuse of those objects use the latest information to avoid relocations. Fixes: 2889caa92321 ("drm/i915: Eliminate lots of iterations over the execobjects array") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101635 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20170721145037.25105-4-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> (cherry picked from commit 1f727d9e725a408ef58d159c20fb2e51818ff153) Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-07-27drm/i915: Only mark the execobject as pinned on successChris Wilson
If we fail to acquire a fence (for old school fenced GPU access) then we unwind the vma reservation, including its pin. However, we were making the execobject as holding the pin before erring out, leading to a double unpin: [ 3193.991802] kernel BUG at drivers/gpu/drm/i915/i915_vma.h:287! [ 3193.998131] invalid opcode: 0000 [#1] PREEMPT SMP [ 3194.002816] Modules linked in: snd_hda_intel i915 vgem snd_hda_codec_analog snd_hda_codec_generic coretemp snd_hda_codec snd_hwdep snd_hda_core snd_pcm lpc_ich mei_me e1000e mei prime_numbers ptp pps_core [last unloaded: i915] [ 3194.022841] CPU: 0 PID: 8123 Comm: kms_flip Tainted: G U 4.13.0-rc1-CI-CI_DRM_471+ #1 [ 3194.031765] Hardware name: Dell Inc. OptiPlex 755 /0PU052, BIOS A04 11/05/2007 [ 3194.040343] task: ffff8800785d4c40 task.stack: ffffc90001768000 [ 3194.046339] RIP: 0010:eb_release_vmas.isra.6+0x119/0x180 [i915] [ 3194.052234] RSP: 0018:ffffc9000176ba80 EFLAGS: 00010246 [ 3194.057439] RAX: 00000000000003c0 RBX: ffff8800710fc2d8 RCX: ffff8800588e4f48 [ 3194.064546] RDX: ffffffff1fffffff RSI: 00000000ffffffff RDI: ffff8800588e00d0 [ 3194.071654] RBP: ffffc9000176bab0 R08: 0000000000000000 R09: 0000000000000000 [ 3194.078761] R10: 0000000000000040 R11: 0000000000000001 R12: ffff880060822f00 [ 3194.085867] R13: 0000000000000310 R14: 00000000000003b8 R15: ffffc9000176bbb0 [ 3194.092975] FS: 00007fd2b94aba40(0000) GS:ffff88007d200000(0000) knlGS:0000000000000000 [ 3194.101033] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3194.106754] CR2: 00007ffbec3ff000 CR3: 0000000074e67000 CR4: 00000000000006f0 [ 3194.113861] Call Trace: [ 3194.116321] eb_relocate_slow+0x67/0x4e0 [i915] [ 3194.120861] i915_gem_do_execbuffer+0x429/0x1260 [i915] [ 3194.126070] ? lock_acquire+0xb5/0x210 [ 3194.129803] ? __might_fault+0x39/0x90 [ 3194.133563] i915_gem_execbuffer2+0x9b/0x1b0 [i915] [ 3194.138447] ? i915_gem_execbuffer+0x2b0/0x2b0 [i915] [ 3194.143478] drm_ioctl_kernel+0x64/0xb0 [ 3194.147298] drm_ioctl+0x2cd/0x390 [ 3194.150710] ? i915_gem_execbuffer+0x2b0/0x2b0 [i915] [ 3194.155741] ? finish_task_switch+0xa5/0x210 [ 3194.159993] ? finish_task_switch+0x6a/0x210 [ 3194.164247] do_vfs_ioctl+0x90/0x670 [ 3194.167806] ? entry_SYSCALL_64_fastpath+0x5/0xb1 [ 3194.172492] ? __this_cpu_preempt_check+0x13/0x20 [ 3194.177176] ? trace_hardirqs_on_caller+0xe7/0x1c0 [ 3194.181946] SyS_ioctl+0x3c/0x70 [ 3194.185159] entry_SYSCALL_64_fastpath+0x1c/0xb1 [ 3194.189756] RIP: 0033:0x7fd2b76a8587 [ 3194.193314] RSP: 002b:00007fff074845b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 3194.200855] RAX: ffffffffffffffda RBX: ffffffff8146da43 RCX: 00007fd2b76a8587 [ 3194.207962] RDX: 00007fff074846e0 RSI: 0000000040406469 RDI: 0000000000000003 [ 3194.215068] RBP: ffffc9000176bf88 R08: 0000000000000000 R09: 0000000000000003 [ 3194.222175] R10: 00007fd2b796bb58 R11: 0000000000000246 R12: 00007fff07484880 [ 3194.229280] R13: 0000000000000003 R14: 0000000040406469 R15: 0000000000000000 [ 3194.236386] ? __this_cpu_preempt_check+0x13/0x20 [ 3194.241070] Code: 24 b0 00 00 00 48 85 c9 0f 84 6c ff ff ff 8b 41 20 85 c0 7e 73 83 e8 01 89 41 20 41 8b 84 24 e8 00 00 00 a8 0f 0f 85 5f ff ff ff <0f> 0b 48 83 c4 08 5b 41 5c 41 5d 41 5e 41 5f 5d f3 c3 49 8b 84 [ 3194.259943] RIP: eb_release_vmas.isra.6+0x119/0x180 [i915] RSP: ffffc9000176ba80 [ 3194.268047] ---[ end trace 1d7348c6575d8800 ]--- [ 3673.658819] softdog: Initiating panic [ 3673.662471] Kernel panic - not syncing: Software Watchdog Timer expired [ 3673.669066] Kernel Offset: disabled [ 3673.672541] Rebooting in 1 seconds.. Reported-by: Tomi Sarvela <tomi.p.sarvela@intel.com> Fixes: 2889caa92321 ("drm/i915: Eliminate lots of iterations over the execobjects array") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20170721145037.25105-3-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> (cherry picked from commit 1da7b54c46bcfe5484af0b27d8c9003b238031b0) Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-07-27drm/i915: Fix user ptr check size in eb_relocate_vma()Imre Deak
Fix the sizeof(ptr) vs. sizeof(*ptr) typo. Fixes: 2889caa92321 ("drm/i915: Eliminate lots of iterations over the execobjects array") Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170714151242.517-2-imre.deak@intel.com (cherry picked from commit edd9003f7f9dddd28fdd768e6e7569d996c769cb) Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-07-03drm/i915: Avoid undefined behaviour of "u32 >> 32"Chris Wilson
When computing a hash for looking up relocation target handles in an execbuf, we start with a large size for the hashtable and proceed to halve it until the allocation succeeds. The final attempt is with an order of 0 (i.e. a single element). This means that we then pass bits=0 to hash_32() which then computes "hash >> (32 - 0)" to lookup the single element. Right shifting a value by the width of the operand is undefined, so limit the smallest hash table we use to order 1. v2: Keep the retry allocation flag for the final pass Fixes: 4ff4b44cbb70 ("drm/i915: Store a direct lookup from object handle to vma") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170629150425.27508-1-chris@chris-wilson.co.uk (cherry picked from commit 4d470f7359c4bf22518baa30700ad45649371a22) Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2017-06-26drm/i915: Clear execbuf's vma backpointer upon releaseChris Wilson
commit 2889caa92321 ("drm/i915: Eliminate lots of iterations over the execobjects array") jiggled around the error handling and replace a test that we cleaned up properly after ourselves with an assertion. That assertion failed because in the release function (moments after the assertion) we were indeed forgetting to mark the vma as cleared. The consequence was when testing an invalid relocation address, we would try to release the vma twice (following the couple of attempts to verify the address) and on the second release notice that the first release was incomplete. Testcase: igt/gem_reloc_overflow/invalid-address Fixes: 2889caa92321 ("drm/i915: Eliminate lots of iterations over the execobjects array") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170622104722.2583-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> (cherry picked from commit 51d05e1b29676a0425749a1533b87e3ad3c6f176) Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-06-26drm/i915: Pass the right flags to i915_vma_move_to_active()Chris Wilson
i915_vma_move_to_active() takes the execobject flags and not a boolean! Instead of passing EXEC_OBJECT_WRITE we passed true [i.e. EXEC_OBJECT_NEEDS_FENCE] causing us to start tracking the vma->last_fence access and since we forgot to clear that on unbinding, we caused a use-after-free. [ 321.263854] BUG: KASAN: use-after-free in i915_gem_request_retire+0x1728/0x1740 [i915] [ 321.264001] Read of size 8 at addr ffff880100fc67d8 by task gem_exec_reloc/2868 [ 321.264181] CPU: 0 PID: 2868 Comm: gem_exec_reloc Not tainted 4.12.0-rc6-CI-Custom_2759+ #1 [ 321.264195] Hardware name: GIGABYTE GB-BXBT-1900/MZBAYAB-00, BIOS F6 02/17/2015 [ 321.264208] Call Trace: [ 321.264234] dump_stack+0x67/0x99 [ 321.264260] print_address_description+0x77/0x290 [ 321.264437] ? i915_gem_request_retire+0x1728/0x1740 [i915] [ 321.264459] kasan_report+0x269/0x350 [ 321.264487] __asan_report_load8_noabort+0x14/0x20 [ 321.264660] i915_gem_request_retire+0x1728/0x1740 [i915] [ 321.264841] ? intel_ring_context_pin+0x131/0x690 [i915] [ 321.265021] i915_gem_request_alloc+0x2c6/0x1220 [i915] [ 321.265044] ? _raw_spin_unlock_irqrestore+0x3d/0x60 [ 321.265226] i915_gem_do_execbuffer+0xac0/0x2a20 [i915] [ 321.265250] ? __lock_acquire+0xceb/0x5450 [ 321.265269] ? entry_SYSCALL_64_fastpath+0x1c/0xb1 [ 321.265291] ? kvmalloc_node+0x6b/0x80 [ 321.265310] ? kvmalloc_node+0x6b/0x80 [ 321.265489] ? eb_relocate_slow+0xbe0/0xbe0 [i915] [ 321.265520] ? ___slab_alloc.constprop.28+0x2ab/0x3d0 [ 321.265549] ? debug_check_no_locks_freed+0x280/0x280 [ 321.265591] ? __might_fault+0xc6/0x1b0 [ 321.265782] i915_gem_execbuffer2+0x14a/0x3f0 [i915] [ 321.265815] drm_ioctl+0x4ba/0xaa0 [ 321.265986] ? i915_gem_execbuffer+0xde0/0xde0 [i915] [ 321.266017] ? drm_getunique+0x270/0x270 [ 321.266068] do_vfs_ioctl+0x17f/0xfa0 [ 321.266091] ? __fget+0x1ba/0x330 [ 321.266112] ? lock_acquire+0x390/0x390 [ 321.266133] ? ioctl_preallocate+0x1d0/0x1d0 [ 321.266164] ? __fget+0x1db/0x330 [ 321.266194] ? __fget_light+0x79/0x1f0 [ 321.266219] SyS_ioctl+0x3c/0x70 [ 321.266247] entry_SYSCALL_64_fastpath+0x1c/0xb1 [ 321.266265] RIP: 0033:0x7fcede207357 [ 321.266279] RSP: 002b:00007ffef0effe58 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 321.266307] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fcede207357 [ 321.266321] RDX: 00007ffef0effef0 RSI: 0000000040406469 RDI: 0000000000000004 [ 321.266335] RBP: ffffffff812097c6 R08: 0000000000000008 R09: 0000000000000000 [ 321.266349] R10: 0000000000000008 R11: 0000000000000246 R12: ffff880116bcff98 [ 321.266363] R13: ffffffff81cb7cb3 R14: ffff880116bcff70 R15: 0000000000000000 [ 321.266385] ? __this_cpu_preempt_check+0x13/0x20 [ 321.266406] ? trace_hardirqs_off_caller+0x1d6/0x2c0 [ 321.266487] Allocated by task 2868: [ 321.266568] save_stack_trace+0x16/0x20 [ 321.266586] kasan_kmalloc+0xee/0x180 [ 321.266602] kasan_slab_alloc+0x12/0x20 [ 321.266620] kmem_cache_alloc+0xc7/0x2e0 [ 321.266795] i915_vma_instance+0x28c/0x1540 [i915] [ 321.266964] eb_lookup_vmas+0x5a7/0x2250 [i915] [ 321.267130] i915_gem_do_execbuffer+0x69a/0x2a20 [i915] [ 321.267296] i915_gem_execbuffer2+0x14a/0x3f0 [i915] [ 321.267315] drm_ioctl+0x4ba/0xaa0 [ 321.267333] do_vfs_ioctl+0x17f/0xfa0 [ 321.267350] SyS_ioctl+0x3c/0x70 [ 321.267369] entry_SYSCALL_64_fastpath+0x1c/0xb1 [ 321.267428] Freed by task 177: [ 321.267502] save_stack_trace+0x16/0x20 [ 321.267521] kasan_slab_free+0xad/0x180 [ 321.267539] kmem_cache_free+0xc5/0x340 [ 321.267710] i915_vma_unbind+0x666/0x10a0 [i915] [ 321.267880] i915_vma_close+0x23a/0x2f0 [i915] [ 321.268048] __i915_gem_free_objects+0x17d/0xc70 [i915] [ 321.268215] __i915_gem_free_work+0x49/0x70 [i915] [ 321.268234] process_one_work+0x66f/0x1410 [ 321.268252] worker_thread+0xe1/0xe90 [ 321.268269] kthread+0x304/0x410 [ 321.268285] ret_from_fork+0x27/0x40 [ 321.268346] The buggy address belongs to the object at ffff880100fc6640 which belongs to the cache i915_vma of size 656 [ 321.268550] The buggy address is located 408 bytes inside of 656-byte region [ffff880100fc6640, ffff880100fc68d0) [ 321.268741] The buggy address belongs to the page: [ 321.268837] page:ffffea000403f000 count:1 mapcount:0 mapping: (null) index:0xffff880100fc5980 compound_mapcount: 0 [ 321.269045] flags: 0x8000000000008100(slab|head) [ 321.269147] raw: 8000000000008100 0000000000000000 ffff880100fc5980 00000001001e001d [ 321.269312] raw: ffffea0004038e20 ffff880116b46240 ffff88011646c640 0000000000000000 [ 321.269484] page dumped because: kasan: bad access detected [ 321.269665] Memory state around the buggy address: [ 321.269778] ffff880100fc6680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 321.269949] ffff880100fc6700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 321.270115] >ffff880100fc6780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 321.270279] ^ [ 321.270410] ffff880100fc6800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 321.270576] ffff880100fc6880: fb fb fb fb fb fb fb fb fb fb fc fc fc fc fc fc [ 321.270740] ================================================================== [ 321.270903] Disabling lock debugging due to kernel taint Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101511 Fixes: 7dd4f6729f92 ("drm/i915: Async GPU relocation processing") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170620124321.1108-2-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> (cherry picked from commit 25ffaa67459e988e73210543f7e05dfbf3f16163) Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-06-16drm/i915: Stash a pointer to the obj's resv in the vmaChris Wilson
During execbuf, a mandatory step is that we add this request (this fence) to each object's reservation_object. Inside execbuf, we track the vma, and to add the fence to the reservation_object then means having to first chase the obj, incurring another cache miss. We can reduce the number of cache misses by stashing a pointer to the reservation_object in the vma itself. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170616140525.6394-1-chris@chris-wilson.co.uk
2017-06-16drm/i915: Async GPU relocation processingChris Wilson
If the user requires patching of their batch or auxiliary buffers, we currently make the alterations on the cpu. If they are active on the GPU at the time, we wait under the struct_mutex for them to finish executing before we rewrite the contents. This happens if shared relocation trees are used between different contexts with separate address space (and the buffers then have different addresses in each), the 3D state will need to be adjusted between execution on each context. However, we don't need to use the CPU to do the relocation patching, as we could queue commands to the GPU to perform it and use fences to serialise the operation with the current activity and future - so the operation on the GPU appears just as atomic as performing it immediately. Performing the relocation rewrites on the GPU is not free, in terms of pure throughput, the number of relocations/s is about halved - but more importantly so is the time under the struct_mutex. v2: Break out the request/batch allocation for clearer error flow. v3: A few asserts to ensure rq ordering is maintained Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Allow execbuffer to use the first object as the batchChris Wilson
Currently, the last object in the execlist is the always the batch. However, when building the batch buffer we often know the batch object first and if we can use the first slot in the execlist we can emit relocation instructions relative to it immediately and avoid a separate pass to adjust the relocations to point to the last execlist slot. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Wait upon userptr get-user-pages within execbufferChris Wilson
This simply hides the EAGAIN caused by userptr when userspace causes resource contention. However, it is quite beneficial with highly contended userptr users as we avoid repeating the setup costs and kernel-user context switches. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
2017-06-16drm/i915: First try the previous execbuffer locationChris Wilson
When choosing a slot for an execbuffer, we ideally want to use the same address as last time (so that we don't have to rebind it) and the same address as expected by the user (so that we don't have to fixup any relocations pointing to it). If we first try to bind the incoming execbuffer->offset from the user, or the currently bound offset that should hopefully achieve the goal of avoiding the rebind cost and the relocation penalty. However, if the object is not currently bound there we don't want to arbitrarily unbind an object in our chosen position and so choose to rebind/relocate the incoming object instead. After we report the new position back to the user, on the next pass the relocations should have settled down. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtien@linux.intel.com>
2017-06-16drm/i915: Store a persistent reference for an object in the execbuffer cacheChris Wilson
If we take a reference to the object/vma when it is first used in an execbuf, we can keep that reference until the object's file-local handle is closed. Thereby saving a frequent ref/unref pair. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Eliminate lots of iterations over the execobjects arrayChris Wilson
The major scaling bottleneck in execbuffer is the processing of the execobjects. Creating an auxiliary list is inefficient when compared to using the execobject array we already have allocated. Reservation is then split into phases. As we lookup up the VMA, we try and bind it back into active location. Only if that fails, do we add it to the unbound list for phase 2. In phase 2, we try and add all those objects that could not fit into their previous location, with fallback to retrying all objects and evicting the VM in case of severe fragmentation. (This is the same as before, except that phase 1 is now done inline with looking up the VMA to avoid an iteration over the execobject array. In the ideal case, we eliminate the separate reservation phase). During the reservation phase, we only evict from the VM between passes (rather than currently as we try to fit every new VMA). In testing with Unreal Engine's Atlantis demo which stresses the eviction logic on gen7 class hardware, this speed up the framerate by a factor of 2. The second loop amalgamation is between move_to_gpu and move_to_active. As we always submit the request, even if incomplete, we can use the current request to track active VMA as we perform the flushes and synchronisation required. The next big advancement is to avoid copying back to the user any execobjects and relocations that are not changed. v2: Add a Theory of Operation spiel. v3: Fall back to slow relocations in preparation for flushing userptrs. v4: Document struct members, factor out eb_validate_vma(), add a few more comments to explain some magic and hide other magic behind macros. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Disable EXEC_OBJECT_ASYNC when doing relocationsChris Wilson
If we write a relocation into the buffer, we require our own implicit synchronisation added after the start of the execbuf, outside of the user's control. As we may end up clflushing, or doing the patch itself on the GPU, asynchronously we need to look at the implicit serialisation on obj->resv and hence need to disable EXEC_OBJECT_ASYNC for this object. If the user does trigger a stall for relocations, we make sure the stall is complete enough so that the batch is not submitted before we complete those relocations. Fixes: 77ae9957897d ("drm/i915: Enable userspace to opt-out of implicit fencing") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Pass vma to relocate entryChris Wilson
We can simplify our tracking of pending writes in an execbuf to the single bit in the vma->exec_entry->flags, but that requires the relocation function knowing the object's vma. Pass it along. Note we have only been using a single bit to track flushing since commit cc889e0f6ce6a63c62db17d702ecfed86d58083f Author: Daniel Vetter <daniel.vetter@ffwll.ch> Date: Wed Jun 13 20:45:19 2012 +0200 drm/i915: disable flushing_list/gpu_write_list unconditionally flushed all render caches before the breadcrumb and commit 6ac42f4148bc27e5ffd18a9ab0eac57f58822af4 Author: Daniel Vetter <daniel.vetter@ffwll.ch> Date: Sat Jul 21 12:25:01 2012 +0200 drm/i915: Replace the complex flushing logic with simple invalidate/flush all did away with the explicit GPU domain tracking. This was then codified into the ABI with NO_RELOC in commit ed5982e6ce5f106abcbf071f80730db344a6da42 Author: Daniel Vetter <daniel.vetter@ffwll.ch> # Oi! Patch stealer! Date: Thu Jan 17 22:23:36 2013 +0100 drm/i915: Allow userspace to hint that the relocations were known Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Store a direct lookup from object handle to vmaChris Wilson
The advent of full-ppgtt lead to an extra indirection between the object and its binding. That extra indirection has a noticeable impact on how fast we can convert from the user handles to our internal vma for execbuffer. In order to bypass the extra indirection, we use a resizable hashtable to jump from the object to the per-ctx vma. rhashtable was considered but we don't need the online resizing feature and the extra complexity proved to undermine its usefulness. Instead, we simply reallocate the hastable on demand in a background task and serialize it before iterating. In non-full-ppgtt modes, multiple files and multiple contexts can share the same vma. This leads to having multiple possible handle->vma links, so we only use the first to establish the fast path. The majority of buffers are not shared and so we should still be able to realise speedups with multiple clients. v2: Prettier names, more magic. v3: Many style tweaks, most notably hiding the misuse of execobj[].rsvd2 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-06-16drm/i915: Store i915_gem_object_is_coherent() as a bit next to cache-dirtyChris Wilson
For ease of use (i.e. avoiding a few checks and function calls), store the object's cache coherency next to the cache is dirty bit. Specifically this patch aims to reduce the frequency of no-op calls to i915_gem_object_clflush() to counter-act the increase of such calls for GPU only objects in the previous patch. v2: Replace cache_dirty & ~cache_coherent with cache_dirty && !cache_coherent as gcc generates much better code for the latter (Tvrtko) Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Dongwon Kim <dongwon.kim@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Tested-by: Dongwon Kim <dongwon.kim@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170616105455.16977-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-06-16drm/i915: Mark CPU cache as dirty on every transition for CPU writesChris Wilson
Currently, we only mark the CPU cache as dirty if we skip a clflush. This leads to some confusion where we have to ask if the object is in the write domain or missed a clflush. If we always mark the cache as dirty, this becomes a much simply question to answer. The goal remains to do as few clflushes as required and to do them as late as possible, in the hope of deferring the work to a kthread and not block the caller (e.g. execbuf, flips). v2: Always call clflush before GPU execution when the cache_dirty flag is set. This may cause some extra work on llc systems that migrate dirty buffers back and forth - but we do try to limit that by only setting cache_dirty at the end of the gpu sequence. v3: Always mark the cache as dirty upon a level change, as we need to invalidate any stale cachelines due to external writes. Reported-by: Dongwon Kim <dongwon.kim@intel.com> Fixes: a6a7cc4b7db6 ("drm/i915: Always flush the dirty CPU cache when pinning the scanout") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Dongwon Kim <dongwon.kim@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Tested-by: Dongwon Kim <dongwon.kim@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170615123850.26843-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-06-15drm/i915: Split vma exec_link/evict_linkChris Wilson
Currently the vma has one link member that is used for both holding its place in the execbuf reservation list, and in any eviction list. This dual property is quite tricky and error prone. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170615081435.17699-3-chris@chris-wilson.co.uk
2017-06-15drm/i915: Use vma->exec_entry as our double-entry placeholderChris Wilson
This has the benefit of not requiring us to manipulate the vma->exec_link list when tearing down the execbuffer, and is a marginally cheaper test to detect the user error. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170615081435.17699-2-chris@chris-wilson.co.uk
2017-06-15drm/i915: Amalgamate execbuffer parameter structuresChris Wilson
Combine the two slightly overlapping parameter structures we pass around the execbuffer routines into one. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170615081435.17699-1-chris@chris-wilson.co.uk
2017-05-30Merge tag 'drm-intel-next-2017-05-29' of ↵Dave Airlie
git://anongit.freedesktop.org/git/drm-intel into drm-next More stuff for 4.13: - skl+ wm fixes from Mahesh Kumar - some refactor and tests for i915_sw_fence (Chris) - tune execlist/scheduler code (Chris) - g4x,g33 gpu reset improvements (Chris, Mika) - guc code cleanup (Michal Wajdeczko, Michał Winiarski) - dp aux backlight improvements (Puthikorn Voravootivat) - buffer based guc/host communication (Michal Wajdeczko) * tag 'drm-intel-next-2017-05-29' of git://anongit.freedesktop.org/git/drm-intel: (253 commits) drm/i915: Update DRIVER_DATE to 20170529 drm/i915: Keep the forcewake timer alive for 1ms past the most recent use drm/i915/guc: capture GuC logs if FW fails to load drm/i915/guc: Introduce buffer based cmd transport drm/i915/guc: Disable send function on fini drm: Add definition for eDP backlight frequency drm/i915: Drop AUX backlight enable check for backlight control drm/i915: Consolidate #ifdef CONFIG_INTEL_IOMMU drm/i915: Only GGTT vma may be pinned and prevent shrinking drm/i915: Serialize GTT/Aperture accesses on BXT drm/i915: Convert i915_gem_object_ops->flags values to use BIT() drm/i915/selftests: Silence compiler warning in igt_ctx_exec drm/i915/guc: Skip port assign on first iteration of GuC dequeue drm/i915: Remove misleading comment in request_alloc drm/i915/g33: Improve reset reliability Revert "drm/i915: Restore lost "Initialized i915" welcome message" drm/i915/huc: Update GLK HuC version drm/i915: Check for allocation failure drm/i915/guc: Remove action status and statistics from debugfs drm/i915/g4x: Improve gpu reset reliability ...
2017-05-18drm: drop drm_[cm]alloc* helpersMichal Hocko
Now that drm_[cm]alloc* helpers are simple one line wrappers around kvmalloc_array and drm_free_large is just kvfree alias we can drop them and replace by their native forms. This shouldn't introduce any functional change. Changes since v1 - fix typo in drivers/gpu//drm/etnaviv/etnaviv_gem.c - noticed by 0day build robot Suggested-by: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Michal Hocko <mhocko@suse.com>drm: drop drm_[cm]alloc* helpers [danvet: Fixup vgem which grew another user very recently.] Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Acked-by: Christian König <christian.koenig@amd.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170517122312.GK18247@dhcp22.suse.cz
2017-04-15drm/i915: Copy user requested buffers into the error stateChris Wilson
Introduce a new execobject.flag (EXEC_OBJECT_CAPTURE) that userspace may use to indicate that it wants the contents of this buffer preserved in the error state (/sys/class/drm/cardN/error) following a GPU hang involving this batch. Use this at your discretion, the contents of the error state. although compressed, are allocated with GFP_ATOMIC (i.e. limited) and kept for all eternity (until the error state is destroyed). Based on an earlier patch by Ben Widawsky <ben@bwidawsk.net> Testcase: igt/gem_exec_capture Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Ben Widawsky <ben@bwidawsk.net> Cc: Matt Turner <mattst88@gmail.com> Acked-by: Ben Widawsky <ben@bwidawsk.net> Acked-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170415093902.22581-1-chris@chris-wilson.co.uk
2017-03-27drm/i915: Align "unfenced" tiled access on gen2, early gen3Chris Wilson
Old devices have quite severe restrictions for using fences, and unlike more recent device (anything from Pineview onwards) we need to enforce those restrictions even for unfenced tiled access from the render pipeline. Fixes: 944397f04f24 ("drm/i915: Store required fence size/alignment for GGTT vma") Reported-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: <drm-intel-fixes@lists.freedesktop.org> # v4.11-rc1+ Link: http://patchwork.freedesktop.org/patch/msgid/20170325113243.16438-1-chris@chris-wilson.co.uk Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Tested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
2017-03-02drm/i915: Drop spinlocks around adding to the client request listChris Wilson
Adding to the tail of the client request list as the only other user is in the throttle ioctl that iterates forwards over the list. It only needs protection against deletion of a request as it reads it, it simply won't see a new request added to the end of the list, or it would be too early and rejected. We can further reduce the number of spinlocks required when throttling by removing stale requests from the client_list as we throttle. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170302122525.19675-1-chris@chris-wilson.co.uk
2017-02-24drm/i915: Drop support for I915_EXEC_CONSTANTS_* execbuf parameters.Kenneth Graunke
This patch makes the I915_PARAM_HAS_EXEC_CONSTANTS getparam return 0 (indicating the optional feature is not supported), and makes execbuf always return -EINVAL if the flags are used. Apparently, no userspace ever shipped which used this optional feature: I checked the git history of Mesa, xf86-video-intel, libva, and Beignet, and there were zero commits showing a use of these flags. Kernel commit 72bfa19c8deb4 apparently introduced the feature prematurely. According to Chris, the intention was to use this in cairo-drm, but "the use was broken for gen6", so I don't think it ever happened. 'relative_constants_mode' has always been tracked per-device, but this has actually been wrong ever since hardware contexts were introduced, as the INSTPM register is saved (and automatically restored) as part of the render ring context. The software per-device value could therefore get out of sync with the hardware per-context value. This meant that using them is actually unsafe: a client which tried to use them could damage the state of other clients, causing the GPU to interpret their BO offsets as absolute pointers, leading to bogus memory reads. These flags were also never ported to execlist mode, making them no-ops on Gen9+ (which requires execlists), and Gen8 in the default mode. On Gen8+, userspace can write these registers directly, achieving the same effect. On Gen6-7.5, it likely makes sense to extend the command parser to support them. I don't think anyone wants this on Gen4-5. Based on a patch by Dave Gordon. v3: Return -ENODEV for the getparam, as this is what we do for other obsolete features. Suggested by Chris Wilson. Cc: stable@vger.kernel.org Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92448 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170215093446.21291-1-kenneth@whitecape.org Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
2017-02-22drm/i915: Perform object clflushing asynchronouslyChris Wilson
Flushing the cachelines for an object is slow, can be as much as 100ms for a large framebuffer. We currently do this under the struct_mutex BKL on execution or on pageflip. But now with the ability to add fences to obj->resv for both flips and execbuf (and we naturally wait on the fence before CPU access), we can move the clflush operation to a workqueue and signal a fence for completion, thereby doing the work asynchronously and not blocking the driver or its clients. v2: Introduce i915_gem_clflush.h and use a new name, split out some extras into separate patches. Suggested-by: Akash Goel <akash.goel@intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170222114049.28456-5-chris@chris-wilson.co.uk
2017-02-22drm/i915: Remove change_domain tracepointChris Wilson
The change_domain tracepoint has been inaccurate for a few years - it doesn't fully capture the domains, especially with userspace bypassing them. It is defunct, misleading and time to be removed. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170222114049.28456-1-chris@chris-wilson.co.uk
2017-02-21drm/i915/tracepoints: Adjust i915_gem_ring_dispatchTvrtko Ursulin
Rename it to i915_gem_request_queue and fix the logged info equivalent to the i915_gem_request even class. Also moved it a bit further apart from the i915_gem_request_add tracepoint since they otherwise provide similar information too close in time. v2: Remove sw fence singalling. We will rely on the soon to come GuC scheduling backend to enable that. (Chris Wilson) v3: Log hex with 0x prefix for clarity. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
2017-02-21drm/i915: Use reservation_object_lock()Chris Wilson
Replace the calls to ww_mutex_lock(&resv->lock) with the helper reservation_object_lock(resv) and similarly for unlock. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170221091723.6219-1-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-02-14drm/i915: Emit to ringbuffer directlyTvrtko Ursulin
This removes the usage of intel_ring_emit in favour of directly writing to the ring buffer. intel_ring_emit was preventing the compiler for optimising fetch and increment of the current ring buffer pointer and therefore generating very verbose code for every write. It had no useful purpose since all ringbuffer operations are started and ended with intel_ring_begin and intel_ring_advance respectively, with no bail out in the middle possible, so it is fine to increment the tail in intel_ring_begin and let the code manage the pointer itself. Useless instruction removal amounts to approximately two and half kilobytes of saved text on my build. Not sure if this has any measurable performance implications but executing a ton of useless instructions on fast paths cannot be good. v2: * Change return from intel_ring_begin to error pointer by popular demand. * Move tail increment to intel_ring_advance to enable some error checking. v3: * Move tail advance back into intel_ring_begin. * Rebase and tidy. v4: * Complete rebase after a few months since v3. v5: * Remove unecessary cast and fix !debug compile. (Chris Wilson) v6: * Make intel_ring_offset take request as well. * Fix recording of request postfix plus a sprinkle of asserts. (Chris Wilson) v7: * Use intel_ring_offset to get the postfix. (Chris Wilson) * Convert GVT code as well. v8: * Rename *out++ to *cs++. v9: * Fix GVT out to cs conversion in GVT. v10: * Rebase for new intel_ring_begin in selftests. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Zhi Wang <zhi.a.wang@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Acked-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170214113242.29241-1-tvrtko.ursulin@linux.intel.com
2017-02-10Merge remote-tracking branch 'airlied/drm-next' into drm-intel-next-queuedDaniel Vetter
Chris Wilson needs the new drm_driver->release callback to make sure the shiny new dma-buf testcases don't oops the driver on unload. Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
2017-02-08drm/i915: Always convert incoming exec offsets to non-canonicalMichał Winiarski
We're using non-canonical addresses in drm_mm, and we're making sure that userspace is using canonical addressing - both in case of softpin (verifying incoming offset) and when relocating (converting to canonical when updating offset returned to userspace). Unfortunately when considering the need for relocations, we're comparing offset from userspace (in canonical form) with drm_mm node (in non-canonical form), and as a result, we end up always relocating if our offsets are in the "problematic" range. Let's always convert the offsets to avoid the performance impact of relocations. Fixes: a5f0edf63bdf ("drm/i915: Avoid writing relocs with addresses in non-canonical form") Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Michel Thierry <michel.thierry@intel.com> Reported-by: Michał Pyrzowski <michal.pyrzowski@intel.com> Signed-off-by: Michał Winiarski <michal.winiarski@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170207195559.18798-1-michal.winiarski@intel.com Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
2017-02-04drm/i915: fix pm refcounting on fence error in execbufDaniele Ceraolo Spurio
Fences are creted/checked before the pm ref is taken, so if we jump to pre_mutex_err we will uncorrectly call intel_runtime_pm_put. v2: Massage unwind error paths Fixes: fec0445caa27 (drm/i915: Support explicit fencing for execbuf) Testcase: igt/gem_exec_params Cc: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1486161930-11764-1-git-send-email-daniele.ceraolospurio@intel.com Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
2017-02-03drm: Improve drm_mm search (and fix topdown allocation) with rbtreesChris Wilson
The drm_mm range manager claimed to support top-down insertion, but it was neither searching for the top-most hole that could fit the allocation request nor fitting the request to the hole correctly. In order to search the range efficiently, we create a secondary index for the holes using either their size or their address. This index allows us to find the smallest hole or the hole at the bottom or top of the range efficiently, whilst keeping the hole stack to rapidly service evictions. v2: Search for holes both high and low. Rename flags to mode. v3: Discover rb_entry_safe() and use it! v4: Kerneldoc for enum drm_mm_insert_mode. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Alex Deucher <alexander.deucher@amd.com> Cc: "Christian König" <christian.koenig@amd.com> Cc: David Airlie <airlied@linux.ie> Cc: Russell King <rmk+kernel@armlinux.org.uk> Cc: Daniel Vetter <daniel.vetter@intel.com> Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: Sean Paul <seanpaul@chromium.org> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Christian Gmeiner <christian.gmeiner@gmail.com> Cc: Rob Clark <robdclark@gmail.com> Cc: Thierry Reding <thierry.reding@gmail.com> Cc: Stephen Warren <swarren@wwwdotorg.org> Cc: Alexandre Courbot <gnurou@gmail.com> Cc: Eric Anholt <eric@anholt.net> Cc: Sinclair Yeh <syeh@vmware.com> Cc: Thomas Hellstrom <thellstrom@vmware.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Sinclair Yeh <syeh@vmware.com> # vmwgfx Reviewed-by: Lucas Stach <l.stach@pengutronix.de> #etnaviv Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: http://patchwork.freedesktop.org/patch/msgid/20170202210438.28702-1-chris@chris-wilson.co.uk
2017-01-27drm/i915: Support explicit fencing for execbufChris Wilson
Now that the user can opt-out of implicit fencing, we need to give them back control over the fencing. We employ sync_file to wrap our drm_i915_gem_request and provide an fd that userspace can merge with other sync_file fds and pass back to the kernel to wait upon before future execution. Testcase: igt/gem_exec_fence Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Acked-by: Chad Versace <chadversary@chromium.org> Link: http://patchwork.freedesktop.org/patch/msgid/20170127094008.27489-2-chris@chris-wilson.co.uk
2017-01-27drm/i915: Enable userspace to opt-out of implicit fencingChris Wilson
Userspace is faced with a dilemma. The kernel requires implicit fencing to manage resource usage (we always must wait for the GPU to finish before releasing its PTE) and for third parties. However, userspace may wish to avoid this serialisation if it is either using explicit fencing between parties and wants more fine-grained access to buffers (e.g. it may partition the buffer between uses and track fences on ranges rather than the implicit fences tracking the whole object). It follows that userspace needs a mechanism to avoid the kernel's serialisation on its implicit fences before execbuf execution. The next question is whether this is an object, execbuf or context flag. Hybrid users (such as using explicit EGL_ANDROID_native_sync fencing on shared winsys buffers, but implicit fencing on internal surfaces) require a per-object level flag. Given that this flag need to be only set once for the lifetime of the object, this reduces the convenience of having an execbuf or context level flag (and avoids having multiple pieces of uABI controlling the same feature). Incorrect use of this flag will result in rendering corruption and GPU hangs - but will not result in use-after-free or similar resource tracking issues. Serious caveat: write ordering is not strictly correct after setting this flag on a render target on multiple engines. This affects all subsequent GEM operations (execbuf, set-domain, pread) and shared dma-buf operations. A fix is possible - but costly (both in terms of further ABI changes and runtime overhead). Testcase: igt/gem_exec_async Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Acked-by: Chad Versace <chadversary@chromium.org> Link: http://patchwork.freedesktop.org/patch/msgid/20170127094008.27489-1-chris@chris-wilson.co.uk
2017-01-19drm/i915: Rename some warts in the VMA APIChris Wilson
Whilst writing testcases to exercise the VMA API, some oddities came to light, such as i915_gem_obj_lookup_or_create(). Joonas suggested i915_vma_instance() as a neat replacement, so rename them, move them to i915_vma.c and add some kerneldoc as a sugary bonus. s/i915_gem_obj_to_vma/i915_vma_lookup/ s/i915_gem_obj_lookup_or_create_vma/i915_vma_instance/ Suggested-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170116152131.18089-2-chris@chris-wilson.co.uk
2017-01-10drm/i915: Replace 4096 with PAGE_SIZE or I915_GTT_PAGE_SIZEChris Wilson
Start converting over from the byte count to its semantic macro, either we want to allocate the size of a physical page in main memory or we want the size of a virtual page in the GTT. 4096 could mean either, but PAGE_SIZE and I915_GTT_PAGE_SIZE are explicit and should help improve code comprehension and future changes. In the future, we may want to use variable GTT page sizes and so have the challenge of knowing which hardcoded values were used to represent a physical page vs the virtual page. v2: Look for a few more 4096s to convert, discover IS_ALIGNED(). v3: 4096ul paranoia, make fence alignment a distinct value of 4096, keep bdw stolen w/a as 4096 until we know better. v4: Add asserts that i915_vma_insert() start/end are aligned to GTT page sizes. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170110144734.26052-1-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2016-12-31drm/i915: Complete kerneldoc for struct i915_gem_contextChris Wilson
The existing kerneldoc was outdated, so time for a refresh. v2: Use single line kdoc, mention functions for manipulation Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: http://patchwork.freedesktop.org/patch/msgid/20161231112012.29263-3-chris@chris-wilson.co.uk
2016-12-18drm/i915: Add a reminder that i915_vma_move_to_active() requires struct_mutexChris Wilson
i915_vma_move_to_active() requires the struct_mutex for serialisation with retirement, so mark it up with lockdep_assert_held(). Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20161218153724.8439-1-chris@chris-wilson.co.uk
2016-12-05drm/i915: Fix i915_gem_evict_for_vma (soft-pinning)Chris Wilson
Soft-pinning depends upon being able to check for availabilty of an interval and evict overlapping object from a drm_mm range manager very quickly. Currently it uses a linear list, and so performance is dire and not suitable as a general replacement. Worse, the current code will oops if it tries to evict an active buffer. It also helps if the routine reports the correct error codes as expected by its callers and emits a tracepoint upon use. For posterity since the wrong patch was pushed (i.e. that missed these key points and had known bugs), this is the changelog that should have been on commit 506a8e87d8d2 ("drm/i915: Add soft-pinning API for execbuffer"): Userspace can pass in an offset that it presumes the object is located at. The kernel will then do its utmost to fit the object into that location. The assumption is that userspace is handling its own object locations (for example along with full-ppgtt) and that the kernel will rarely have to make space for the user's requests. This extends the DRM_IOCTL_I915_GEM_EXECBUFFER2 to do the following: * if the user supplies a virtual address via the execobject->offset *and* sets the EXEC_OBJECT_PINNED flag in execobject->flags, then that object is placed at that offset in the address space selected by the context specifier in execbuffer. * the location must be aligned to the GTT page size, 4096 bytes * as the object is placed exactly as specified, it may be used by this execbuffer call without relocations pointing to it It may fail to do so if: * EINVAL is returned if the object does not have a 4096 byte aligned address * the object conflicts with another pinned object (either pinned by hardware in that address space, e.g. scanouts in the aliasing ppgtt) or within the same batch. EBUSY is returned if the location is pinned by hardware EINVAL is returned if the location is already in use by the batch * EINVAL is returned if the object conflicts with its own alignment (as meets the hardware requirements) or if the placement of the object does not fit within the address space All other execbuffer errors apply. Presence of this execbuf extension may be queried by passing I915_PARAM_HAS_EXEC_SOFTPIN to DRM_IOCTL_I915_GETPARAM and checking for a reported value of 1 (or greater). v2: Combine the hole/adjusted-hole ENOSPC checks v3: More color, more splitting, more blurb. Fixes: 506a8e87d8d2 ("drm/i915: Add soft-pinning API for execbuffer") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20161205142941.21965-2-chris@chris-wilson.co.uk
2016-12-05drm/i915: Mark all non-vma being inserted into the address spacesChris Wilson
We need to distinguish between full i915_vma structs and simple drm_mm_nodes when considering eviction (i.e. we must be careful not to treat a mere drm_mm_node as a much larger i915_vma causing memory corruption, if we are lucky). To do this, color these not-a-vma with -1 (I915_COLOR_UNEVICTABLE). v2...v200: New name for -1. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20161205142941.21965-1-chris@chris-wilson.co.uk
2016-11-24drm/i915: Use the precomputed value for whether to enable command parsingChris Wilson
As i915.enable_cmd_parser is an unsafe option, make it read-only at runtime. Now that it is constant, we can use the value determined during initialisation as to whether we need the cmdparser at execbuffer time. v2: Remove the inline for its single user, it is clear enough (and shorter) without! Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20161124125851.6615-1-chris@chris-wilson.co.uk
2016-11-21drm/i915: Wipe hang stats as an embedded structMika Kuoppala
Bannable property, banned status, guilty and active counts are properties of i915_gem_context. Make them so. v2: rebase Cc: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Mika Kuoppala <mika.kuoppala@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1479309634-28574-1-git-send-email-mika.kuoppala@intel.com
2016-11-18drm/i915: Move frontbuffer CS write tracking from ggtt vma to objectChris Wilson
I tried to avoid having to track the write for every VMA by only tracking writes to the ggtt. However, for the purposes of frontbuffer tracking this is insufficient as we need to invalidate around writes not just to the the ggtt but all aliased ppgtt views of the framebuffer. By moving the critical section to the object and only doing so for framebuffer writes we can reduce the tracking even further by only watching framebuffers and not vma. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20161116190704.5293-1-chris@chris-wilson.co.uk Tested-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2016-11-17drm/i915: Use dev_priv in INTEL_INFO in i915_gem_execbuffer.cTvrtko Ursulin
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
2016-11-11drm/i915: Further assorted dev_priv cleanupsTvrtko Ursulin
A small selection of macros which can only accept dev_priv from now on and a resulting trickle of fixups. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: David Weinehall <david.weinehall@linux.intel.com>