Commit | Line | Data |
---|---|---|
8f0aa2f2 DH |
1 | ==================================== |
2 | SLOW WORK ITEM EXECUTION THREAD POOL | |
3 | ==================================== | |
4 | ||
5 | By: David Howells <dhowells@redhat.com> | |
6 | ||
7 | The slow work item execution thread pool is a pool of threads for performing | |
8 | things that take a relatively long time, such as making mkdir calls. | |
9 | Typically, when processing something, these items will spend a lot of time | |
10 | blocking a thread on I/O, thus making that thread unavailable for doing other | |
11 | work. | |
12 | ||
13 | The standard workqueue model is unsuitable for this class of work item as that | |
14 | limits the owner to a single thread or a single thread per CPU. For some | |
15 | tasks, however, more threads - or fewer - are required. | |
16 | ||
17 | There is just one pool per system. It contains no threads unless something | |
18 | wants to use it - and that something must register its interest first. When | |
19 | the pool is active, the number of threads it contains is dynamic, varying | |
20 | between a maximum and minimum setting, depending on the load. | |
21 | ||
22 | ||
23 | ==================== | |
24 | CLASSES OF WORK ITEM | |
25 | ==================== | |
26 | ||
27 | This pool support two classes of work items: | |
28 | ||
29 | (*) Slow work items. | |
30 | ||
31 | (*) Very slow work items. | |
32 | ||
33 | The former are expected to finish much quicker than the latter. | |
34 | ||
35 | An operation of the very slow class may do a batch combination of several | |
36 | lookups, mkdirs, and a create for instance. | |
37 | ||
38 | An operation of the ordinarily slow class may, for example, write stuff or | |
39 | expand files, provided the time taken to do so isn't too long. | |
40 | ||
41 | Operations of both types may sleep during execution, thus tying up the thread | |
42 | loaned to it. | |
43 | ||
44 | ||
45 | THREAD-TO-CLASS ALLOCATION | |
46 | -------------------------- | |
47 | ||
48 | Not all the threads in the pool are available to work on very slow work items. | |
49 | The number will be between one and one fewer than the number of active threads. | |
50 | This is configurable (see the "Pool Configuration" section). | |
51 | ||
52 | All the threads are available to work on ordinarily slow work items, but a | |
53 | percentage of the threads will prefer to work on very slow work items. | |
54 | ||
55 | The configuration ensures that at least one thread will be available to work on | |
56 | very slow work items, and at least one thread will be available that won't work | |
57 | on very slow work items at all. | |
58 | ||
59 | ||
60 | ===================== | |
61 | USING SLOW WORK ITEMS | |
62 | ===================== | |
63 | ||
64 | Firstly, a module or subsystem wanting to make use of slow work items must | |
65 | register its interest: | |
66 | ||
67 | int ret = slow_work_register_user(); | |
68 | ||
69 | This will return 0 if successful, or a -ve error upon failure. | |
70 | ||
71 | ||
72 | Slow work items may then be set up by: | |
73 | ||
74 | (1) Declaring a slow_work struct type variable: | |
75 | ||
76 | #include <linux/slow-work.h> | |
77 | ||
78 | struct slow_work myitem; | |
79 | ||
80 | (2) Declaring the operations to be used for this item: | |
81 | ||
82 | struct slow_work_ops myitem_ops = { | |
83 | .get_ref = myitem_get_ref, | |
84 | .put_ref = myitem_put_ref, | |
85 | .execute = myitem_execute, | |
86 | }; | |
87 | ||
88 | [*] For a description of the ops, see section "Item Operations". | |
89 | ||
90 | (3) Initialising the item: | |
91 | ||
92 | slow_work_init(&myitem, &myitem_ops); | |
93 | ||
94 | or: | |
95 | ||
96 | vslow_work_init(&myitem, &myitem_ops); | |
97 | ||
98 | depending on its class. | |
99 | ||
100 | A suitably set up work item can then be enqueued for processing: | |
101 | ||
102 | int ret = slow_work_enqueue(&myitem); | |
103 | ||
104 | This will return a -ve error if the thread pool is unable to gain a reference | |
105 | on the item, 0 otherwise. | |
106 | ||
107 | ||
108 | The items are reference counted, so there ought to be no need for a flush | |
109 | operation. When all a module's slow work items have been processed, and the | |
110 | module has no further interest in the facility, it should unregister its | |
111 | interest: | |
112 | ||
113 | slow_work_unregister_user(); | |
114 | ||
115 | ||
116 | =============== | |
117 | ITEM OPERATIONS | |
118 | =============== | |
119 | ||
120 | Each work item requires a table of operations of type struct slow_work_ops. | |
121 | All members are required: | |
122 | ||
123 | (*) Get a reference on an item: | |
124 | ||
125 | int (*get_ref)(struct slow_work *work); | |
126 | ||
127 | This allows the thread pool to attempt to pin an item by getting a | |
128 | reference on it. This function should return 0 if the reference was | |
129 | granted, or a -ve error otherwise. If an error is returned, | |
130 | slow_work_enqueue() will fail. | |
131 | ||
132 | The reference is held whilst the item is queued and whilst it is being | |
133 | executed. The item may then be requeued with the same reference held, or | |
134 | the reference will be released. | |
135 | ||
136 | (*) Release a reference on an item: | |
137 | ||
138 | void (*put_ref)(struct slow_work *work); | |
139 | ||
140 | This allows the thread pool to unpin an item by releasing the reference on | |
141 | it. The thread pool will not touch the item again once this has been | |
142 | called. | |
143 | ||
144 | (*) Execute an item: | |
145 | ||
146 | void (*execute)(struct slow_work *work); | |
147 | ||
148 | This should perform the work required of the item. It may sleep, it may | |
149 | perform disk I/O and it may wait for locks. | |
150 | ||
151 | ||
152 | ================== | |
153 | POOL CONFIGURATION | |
154 | ================== | |
155 | ||
156 | The slow-work thread pool has a number of configurables: | |
157 | ||
158 | (*) /proc/sys/kernel/slow-work/min-threads | |
159 | ||
160 | The minimum number of threads that should be in the pool whilst it is in | |
161 | use. This may be anywhere between 2 and max-threads. | |
162 | ||
163 | (*) /proc/sys/kernel/slow-work/max-threads | |
164 | ||
165 | The maximum number of threads that should in the pool. This may be | |
166 | anywhere between min-threads and 255 or NR_CPUS * 2, whichever is greater. | |
167 | ||
168 | (*) /proc/sys/kernel/slow-work/vslow-percentage | |
169 | ||
170 | The percentage of active threads in the pool that may be used to execute | |
171 | very slow work items. This may be between 1 and 99. The resultant number | |
172 | is bounded to between 1 and one fewer than the number of active threads. | |
173 | This ensures there is always at least one thread that can process very | |
174 | slow work items, and always at least one thread that won't. |