summaryrefslogtreecommitdiff
path: root/Documentation/livepatch/callbacks.txt
blob: c9776f48e45847d9b33bfd0f42ee04cd54dd45e6 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
======================
(Un)patching Callbacks
======================

Livepatch (un)patch-callbacks provide a mechanism for livepatch modules
to execute callback functions when a kernel object is (un)patched.  They
can be considered a "power feature" that extends livepatching abilities
to include:

  - Safe updates to global data

  - "Patches" to init and probe functions

  - Patching otherwise unpatchable code (i.e. assembly)

In most cases, (un)patch callbacks will need to be used in conjunction
with memory barriers and kernel synchronization primitives, like
mutexes/spinlocks, or even stop_machine(), to avoid concurrency issues.

Callbacks differ from existing kernel facilities:

  - Module init/exit code doesn't run when disabling and re-enabling a
    patch.

  - A module notifier can't stop a to-be-patched module from loading.

Callbacks are part of the klp_object structure and their implementation
is specific to that klp_object.  Other livepatch objects may or may not
be patched, irrespective of the target klp_object's current state.

Callbacks can be registered for the following livepatch actions:

  * Pre-patch    - before a klp_object is patched

  * Post-patch   - after a klp_object has been patched and is active
                   across all tasks

  * Pre-unpatch  - before a klp_object is unpatched (ie, patched code is
                   active), used to clean up post-patch callback
                   resources

  * Post-unpatch - after a klp_object has been patched, all code has
                   been restored and no tasks are running patched code,
                   used to cleanup pre-patch callback resources

Each callback is optional, omitting one does not preclude specifying any
other.  However, the livepatching core executes the handlers in
symmetry: pre-patch callbacks have a post-unpatch counterpart and
post-patch callbacks have a pre-unpatch counterpart.  An unpatch
callback will only be executed if its corresponding patch callback was
executed.  Typical use cases pair a patch handler that acquires and
configures resources with an unpatch handler tears down and releases
those same resources.

A callback is only executed if its host klp_object is loaded.  For
in-kernel vmlinux targets, this means that callbacks will always execute
when a livepatch is enabled/disabled.  For patch target kernel modules,
callbacks will only execute if the target module is loaded.  When a
module target is (un)loaded, its callbacks will execute only if the
livepatch module is enabled.

The pre-patch callback, if specified, is expected to return a status
code (0 for success, -ERRNO on error).  An error status code indicates
to the livepatching core that patching of the current klp_object is not
safe and to stop the current patching request.  (When no pre-patch
callback is provided, the transition is assumed to be safe.)  If a
pre-patch callback returns failure, the kernel's module loader will:

  - Refuse to load a livepatch, if the livepatch is loaded after
    targeted code.

    or:

  - Refuse to load a module, if the livepatch was already successfully
    loaded.

No post-patch, pre-unpatch, or post-unpatch callbacks will be executed
for a given klp_object if the object failed to patch, due to a failed
pre_patch callback or for any other reason.

If a patch transition is reversed, no pre-unpatch handlers will be run
(this follows the previously mentioned symmetry -- pre-unpatch callbacks
will only occur if their corresponding post-patch callback executed).

If the object did successfully patch, but the patch transition never
started for some reason (e.g., if another object failed to patch),
only the post-unpatch callback will be called.


Example Use-cases
=================

Update global data
------------------

A pre-patch callback can be useful to update a global variable.  For
example, 75ff39ccc1bd ("tcp: make challenge acks less predictable")
changes a global sysctl, as well as patches the tcp_send_challenge_ack()
function.

In this case, if we're being super paranoid, it might make sense to
patch the data *after* patching is complete with a post-patch callback,
so that tcp_send_challenge_ack() could first be changed to read
sysctl_tcp_challenge_ack_limit with READ_ONCE.


Support __init and probe function patches
-----------------------------------------

Although __init and probe functions are not directly livepatch-able, it
may be possible to implement similar updates via pre/post-patch
callbacks.

48900cb6af42 ("virtio-net: drop NETIF_F_FRAGLIST") change the way that
virtnet_probe() initialized its driver's net_device features.  A
pre/post-patch callback could iterate over all such devices, making a
similar change to their hw_features value.  (Client functions of the
value may need to be updated accordingly.)


Test cases
==========

What follows is not an exhaustive test suite of every possible livepatch
pre/post-(un)patch combination, but a selection that demonstrates a few
important concepts.  Each test case uses the kernel modules located in
the samples/livepatch/ and assumes that no livepatches are loaded at the
beginning of the test.


Test 1
------

Test a combination of loading a kernel module and a livepatch that
patches a function in the first module.  (Un)load the target module
before the livepatch module:

- load target module
- load livepatch
- disable livepatch
- unload target module
- unload livepatch

First load a target module:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [   34.475708] livepatch_callbacks_mod: livepatch_callbacks_mod_init

On livepatch enable, before the livepatch transition starts, pre-patch
callbacks are executed for vmlinux and livepatch_callbacks_mod (those
klp_objects currently loaded).  After klp_objects are patched according
to the klp_patch, their post-patch callbacks run and the transition
completes:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   36.503719] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   36.504213] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   36.504238] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   36.504721] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   36.505849] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   37.727133] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   37.727232] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   37.727860] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   37.728792] livepatch: 'livepatch_callbacks_demo': patching complete

Similarly, on livepatch disable, pre-patch callbacks run before the
unpatching transition starts.  klp_objects are reverted, post-patch
callbacks execute and the transition completes:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   38.510209] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [   38.510234] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   38.510982] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   38.512209] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   39.711132] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   39.711210] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   39.711779] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   39.712735] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko
  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [   42.534183] livepatch_callbacks_mod: livepatch_callbacks_mod_exit


Test 2
------

This test is similar to the previous test, but (un)load the livepatch
module before the target kernel module.  This tests the livepatch core's
module_coming handler:

- load livepatch
- load target module
- disable livepatch
- unload livepatch
- unload target module


On livepatch enable, only pre/post-patch callbacks are executed for
currently loaded klp_objects, in this case, vmlinux:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   44.553328] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   44.553997] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   44.554049] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   44.554845] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   45.727128] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   45.727212] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   45.727961] livepatch: 'livepatch_callbacks_demo': patching complete

When a targeted module is subsequently loaded, only its pre/post-patch
callbacks are executed:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [   46.560845] livepatch: applying patch 'livepatch_callbacks_demo' to loading module 'livepatch_callbacks_mod'
  [   46.561988] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [   46.563452] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [   46.565495] livepatch_callbacks_mod: livepatch_callbacks_mod_init

On livepatch disable, all currently loaded klp_objects' (vmlinux and
livepatch_callbacks_mod) pre/post-unpatch callbacks are executed:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   48.568885] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [   48.568910] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   48.569441] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   48.570502] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   49.759091] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   49.759171] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   49.759742] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   49.760690] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko
  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [   52.592283] livepatch_callbacks_mod: livepatch_callbacks_mod_exit


Test 3
------

Test loading the livepatch after a targeted kernel module, then unload
the kernel module before disabling the livepatch.  This tests the
livepatch core's module_going handler:

- load target module
- load livepatch
- unload target module
- disable livepatch
- unload livepatch

First load a target module, then the livepatch:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [   54.607948] livepatch_callbacks_mod: livepatch_callbacks_mod_init

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   56.613919] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   56.614411] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   56.614436] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   56.614818] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   56.615656] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   57.759070] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   57.759147] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   57.759621] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_LIVE] Normal state
  [   57.760307] livepatch: 'livepatch_callbacks_demo': patching complete

When a target module is unloaded, the livepatch is only reverted from
that klp_object (livepatch_callbacks_mod).  As such, only its pre and
post-unpatch callbacks are executed when this occurs:

  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [   58.623409] livepatch_callbacks_mod: livepatch_callbacks_mod_exit
  [   58.623903] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away
  [   58.624658] livepatch: reverting patch 'livepatch_callbacks_demo' on unloading module 'livepatch_callbacks_mod'
  [   58.625305] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away

When the livepatch is disabled, pre and post-unpatch callbacks are run
for the remaining klp_object, vmlinux:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   60.638420] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [   60.638444] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   60.638996] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   61.727088] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   61.727165] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   61.727985] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko


Test 4
------

This test is similar to the previous test, however the livepatch is
loaded first.  This tests the livepatch core's module_coming and
module_going handlers:

- load livepatch
- load target module
- unload target module
- disable livepatch
- unload livepatch

First load the livepatch:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   64.661552] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   64.662147] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   64.662175] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   64.662850] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   65.695056] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   65.695147] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   65.695561] livepatch: 'livepatch_callbacks_demo': patching complete

When a targeted kernel module is subsequently loaded, only its
pre/post-patch callbacks are executed:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [   66.669196] livepatch: applying patch 'livepatch_callbacks_demo' to loading module 'livepatch_callbacks_mod'
  [   66.669882] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [   66.670744] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [   66.672873] livepatch_callbacks_mod: livepatch_callbacks_mod_init

When the target module is unloaded, the livepatch is only reverted from
the livepatch_callbacks_mod klp_object.  As such, only pre and
post-unpatch callbacks are executed when this occurs:

  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [   68.680065] livepatch_callbacks_mod: livepatch_callbacks_mod_exit
  [   68.680688] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away
  [   68.681452] livepatch: reverting patch 'livepatch_callbacks_demo' on unloading module 'livepatch_callbacks_mod'
  [   68.682094] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   70.689225] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [   70.689256] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   70.689882] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   71.711080] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   71.711481] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   71.711988] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko


Test 5
------

A simple test of loading a livepatch without one of its patch target
klp_objects ever loaded (livepatch_callbacks_mod):

- load livepatch
- disable livepatch
- unload livepatch

Load the livepatch:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   74.711081] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   74.711595] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   74.711639] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   74.712272] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   75.743137] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   75.743219] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   75.743867] livepatch: 'livepatch_callbacks_demo': patching complete

As expected, only pre/post-(un)patch handlers are executed for vmlinux:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   76.716254] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [   76.716278] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   76.716666] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   77.727089] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   77.727194] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   77.727907] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko


Test 6
------

Test a scenario where a vmlinux pre-patch callback returns a non-zero
status (ie, failure):

- load target module
- load livepatch -ENODEV
- unload target module

First load a target module:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [   80.740520] livepatch_callbacks_mod: livepatch_callbacks_mod_init

Load the livepatch module, setting its 'pre_patch_ret' value to -19
(-ENODEV).  When its vmlinux pre-patch callback executed, this status
code will propagate back to the module-loading subsystem.  The result is
that the insmod command refuses to load the livepatch module:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko pre_patch_ret=-19
  [   82.747326] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   82.747743] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   82.747767] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   82.748237] livepatch: pre-patch callback failed for object 'vmlinux'
  [   82.748637] livepatch: failed to enable patch 'livepatch_callbacks_demo'
  [   82.749059] livepatch: 'livepatch_callbacks_demo': canceling transition, going to unpatch
  [   82.749060] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   82.749868] livepatch: 'livepatch_callbacks_demo': unpatching complete
  [   82.765809] insmod: ERROR: could not insert module samples/livepatch/livepatch-callbacks-demo.ko: No such device

  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [   84.774238] livepatch_callbacks_mod: livepatch_callbacks_mod_exit


Test 7
------

Similar to the previous test, setup a livepatch such that its vmlinux
pre-patch callback returns success.  However, when a targeted kernel
module is later loaded, have the livepatch return a failing status code:

- load livepatch
- setup -ENODEV
- load target module
- disable livepatch
- unload livepatch

Load the livepatch, notice vmlinux pre-patch callback succeeds:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   86.787845] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   86.788325] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   86.788427] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   86.788821] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   87.711069] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   87.711143] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   87.711886] livepatch: 'livepatch_callbacks_demo': patching complete

Set a trap so subsequent pre-patch callbacks to this livepatch will
return -ENODEV:

  % echo -19 > /sys/module/livepatch_callbacks_demo/parameters/pre_patch_ret

The livepatch pre-patch callback for subsequently loaded target modules
will return failure, so the module loader refuses to load the kernel
module.  Notice that no post-patch or pre/post-unpatch callbacks are
executed for this klp_object:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [   90.796976] livepatch: applying patch 'livepatch_callbacks_demo' to loading module 'livepatch_callbacks_mod'
  [   90.797834] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [   90.798900] livepatch: pre-patch callback failed for object 'livepatch_callbacks_mod'
  [   90.799652] livepatch: patch 'livepatch_callbacks_demo' failed for module 'livepatch_callbacks_mod', refusing to load module 'livepatch_callbacks_mod'
  [   90.819737] insmod: ERROR: could not insert module samples/livepatch/livepatch-callbacks-mod.ko: No such device

However, pre/post-unpatch callbacks run for the vmlinux klp_object:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [   92.823547] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [   92.823573] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [   92.824331] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [   93.727128] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [   93.727327] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [   93.727861] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko


Test 8
------

Test loading multiple targeted kernel modules.  This test-case is
mainly for comparing with the next test-case.

- load busy target module (0s sleep),
- load livepatch
- load target module
- unload target module
- disable livepatch
- unload livepatch
- unload busy target module


Load a target "busy" kernel module which kicks off a worker function
that immediately exits:

  % insmod samples/livepatch/livepatch-callbacks-busymod.ko sleep_secs=0
  [   96.910107] livepatch_callbacks_busymod: livepatch_callbacks_mod_init
  [   96.910600] livepatch_callbacks_busymod: busymod_work_func, sleeping 0 seconds ...
  [   96.913024] livepatch_callbacks_busymod: busymod_work_func exit

Proceed with loading the livepatch and another ordinary target module,
notice that the post-patch callbacks are executed and the transition
completes quickly:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [   98.917892] livepatch: enabling patch 'livepatch_callbacks_demo'
  [   98.918426] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [   98.918453] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [   98.918955] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [   98.923835] livepatch: 'livepatch_callbacks_demo': starting patching transition
  [   99.743104] livepatch: 'livepatch_callbacks_demo': completing patching transition
  [   99.743156] livepatch_callbacks_demo: post_patch_callback: vmlinux
  [   99.743679] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [   99.744616] livepatch: 'livepatch_callbacks_demo': patching complete

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [  100.930955] livepatch: applying patch 'livepatch_callbacks_demo' to loading module 'livepatch_callbacks_mod'
  [  100.931668] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [  100.932645] livepatch_callbacks_demo: post_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [  100.934125] livepatch_callbacks_mod: livepatch_callbacks_mod_init

  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [  102.942805] livepatch_callbacks_mod: livepatch_callbacks_mod_exit
  [  102.943640] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away
  [  102.944585] livepatch: reverting patch 'livepatch_callbacks_demo' on unloading module 'livepatch_callbacks_mod'
  [  102.945455] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [  104.953815] livepatch: 'livepatch_callbacks_demo': initializing unpatching transition
  [  104.953838] livepatch_callbacks_demo: pre_unpatch_callback: vmlinux
  [  104.954431] livepatch_callbacks_demo: pre_unpatch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [  104.955426] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [  106.719073] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [  106.722633] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [  106.723282] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [  106.724279] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko
  % rmmod samples/livepatch/livepatch-callbacks-busymod.ko
  [  108.975660] livepatch_callbacks_busymod: livepatch_callbacks_mod_exit


Test 9
------

A similar test as the previous one, but force the "busy" kernel module
to do longer work.

The livepatching core will refuse to patch a task that is currently
executing a to-be-patched function -- the consistency model stalls the
current patch transition until this safety-check is met.  Test a
scenario where one of a livepatch's target klp_objects sits on such a
function for a long time.  Meanwhile, load and unload other target
kernel modules while the livepatch transition is in progress.

- load busy target module (30s sleep)
- load livepatch
- load target module
- unload target module
- disable livepatch
- unload livepatch
- unload busy target module


Load the "busy" kernel module, this time make it do 30 seconds worth of
work:

  % insmod samples/livepatch/livepatch-callbacks-busymod.ko sleep_secs=30
  [  110.993362] livepatch_callbacks_busymod: livepatch_callbacks_mod_init
  [  110.994059] livepatch_callbacks_busymod: busymod_work_func, sleeping 30 seconds ...

Meanwhile, the livepatch is loaded.  Notice that the patch transition
does not complete as the targeted "busy" module is sitting on a
to-be-patched function:

  % insmod samples/livepatch/livepatch-callbacks-demo.ko
  [  113.000309] livepatch: enabling patch 'livepatch_callbacks_demo'
  [  113.000764] livepatch: 'livepatch_callbacks_demo': initializing patching transition
  [  113.000791] livepatch_callbacks_demo: pre_patch_callback: vmlinux
  [  113.001289] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [  113.005208] livepatch: 'livepatch_callbacks_demo': starting patching transition

Load a second target module (this one is an ordinary idle kernel
module).  Note that *no* post-patch callbacks will be executed while the
livepatch is still in transition:

  % insmod samples/livepatch/livepatch-callbacks-mod.ko
  [  115.012740] livepatch: applying patch 'livepatch_callbacks_demo' to loading module 'livepatch_callbacks_mod'
  [  115.013406] livepatch_callbacks_demo: pre_patch_callback: livepatch_callbacks_mod -> [MODULE_STATE_COMING] Full formed, running module_init
  [  115.015315] livepatch_callbacks_mod: livepatch_callbacks_mod_init

Request an unload of the simple kernel module.  The patch is still
transitioning, so its pre-unpatch callbacks are skipped:

  % rmmod samples/livepatch/livepatch-callbacks-mod.ko
  [  117.022626] livepatch_callbacks_mod: livepatch_callbacks_mod_exit
  [  117.023376] livepatch: reverting patch 'livepatch_callbacks_demo' on unloading module 'livepatch_callbacks_mod'
  [  117.024533] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_mod -> [MODULE_STATE_GOING] Going away

Finally the livepatch is disabled.  Since none of the patch's
klp_object's post-patch callbacks executed, the remaining klp_object's
pre-unpatch callbacks are skipped:

  % echo 0 > /sys/kernel/livepatch/livepatch_callbacks_demo/enabled
  [  119.035408] livepatch: 'livepatch_callbacks_demo': reversing transition from patching to unpatching
  [  119.035485] livepatch: 'livepatch_callbacks_demo': starting unpatching transition
  [  119.711166] livepatch: 'livepatch_callbacks_demo': completing unpatching transition
  [  119.714179] livepatch_callbacks_demo: post_unpatch_callback: vmlinux
  [  119.714653] livepatch_callbacks_demo: post_unpatch_callback: livepatch_callbacks_busymod -> [MODULE_STATE_LIVE] Normal state
  [  119.715437] livepatch: 'livepatch_callbacks_demo': unpatching complete

  % rmmod samples/livepatch/livepatch-callbacks-demo.ko
  % rmmod samples/livepatch/livepatch-callbacks-busymod.ko
  [  141.279111] livepatch_callbacks_busymod: busymod_work_func exit
  [  141.279760] livepatch_callbacks_busymod: livepatch_callbacks_mod_exit