Commit | Line | Data |
---|---|---|
75b02146 JC |
1 | 2: HOW THE DEVELOPMENT PROCESS WORKS |
2 | ||
3 | Linux kernel development in the early 1990's was a pretty loose affair, | |
4 | with relatively small numbers of users and developers involved. With a | |
5 | user base in the millions and with some 2,000 developers involved over the | |
6 | course of one year, the kernel has since had to evolve a number of | |
7 | processes to keep development happening smoothly. A solid understanding of | |
8 | how the process works is required in order to be an effective part of it. | |
9 | ||
10 | ||
11 | 2.1: THE BIG PICTURE | |
12 | ||
13 | The kernel developers use a loosely time-based release process, with a new | |
14 | major kernel release happening every two or three months. The recent | |
15 | release history looks like this: | |
16 | ||
5c050fb9 JC |
17 | 2.6.38 March 14, 2011 |
18 | 2.6.37 January 4, 2011 | |
19 | 2.6.36 October 20, 2010 | |
20 | 2.6.35 August 1, 2010 | |
21 | 2.6.34 May 15, 2010 | |
22 | 2.6.33 February 24, 2010 | |
75b02146 JC |
23 | |
24 | Every 2.6.x release is a major kernel release with new features, internal | |
5c050fb9 | 25 | API changes, and more. A typical 2.6 release can contain nearly 10,000 |
75b02146 JC |
26 | changesets with changes to several hundred thousand lines of code. 2.6 is |
27 | thus the leading edge of Linux kernel development; the kernel uses a | |
28 | rolling development model which is continually integrating major changes. | |
29 | ||
30 | A relatively straightforward discipline is followed with regard to the | |
31 | merging of patches for each release. At the beginning of each development | |
32 | cycle, the "merge window" is said to be open. At that time, code which is | |
33 | deemed to be sufficiently stable (and which is accepted by the development | |
34 | community) is merged into the mainline kernel. The bulk of changes for a | |
35 | new development cycle (and all of the major changes) will be merged during | |
36 | this time, at a rate approaching 1,000 changes ("patches," or "changesets") | |
37 | per day. | |
38 | ||
39 | (As an aside, it is worth noting that the changes integrated during the | |
40 | merge window do not come out of thin air; they have been collected, tested, | |
41 | and staged ahead of time. How that process works will be described in | |
42 | detail later on). | |
43 | ||
5c050fb9 JC |
44 | The merge window lasts for approximately two weeks. At the end of this |
45 | time, Linus Torvalds will declare that the window is closed and release the | |
46 | first of the "rc" kernels. For the kernel which is destined to be 2.6.40, | |
47 | for example, the release which happens at the end of the merge window will | |
48 | be called 2.6.40-rc1. The -rc1 release is the signal that the time to | |
49 | merge new features has passed, and that the time to stabilize the next | |
50 | kernel has begun. | |
75b02146 JC |
51 | |
52 | Over the next six to ten weeks, only patches which fix problems should be | |
53 | submitted to the mainline. On occasion a more significant change will be | |
54 | allowed, but such occasions are rare; developers who try to merge new | |
55 | features outside of the merge window tend to get an unfriendly reception. | |
56 | As a general rule, if you miss the merge window for a given feature, the | |
57 | best thing to do is to wait for the next development cycle. (An occasional | |
58 | exception is made for drivers for previously-unsupported hardware; if they | |
59 | touch no in-tree code, they cannot cause regressions and should be safe to | |
60 | add at any time). | |
61 | ||
62 | As fixes make their way into the mainline, the patch rate will slow over | |
63 | time. Linus releases new -rc kernels about once a week; a normal series | |
64 | will get up to somewhere between -rc6 and -rc9 before the kernel is | |
65 | considered to be sufficiently stable and the final 2.6.x release is made. | |
66 | At that point the whole process starts over again. | |
67 | ||
5c050fb9 JC |
68 | As an example, here is how the 2.6.38 development cycle went (all dates in |
69 | 2011): | |
70 | ||
71 | January 4 2.6.37 stable release | |
72 | January 18 2.6.38-rc1, merge window closes | |
73 | January 21 2.6.38-rc2 | |
74 | February 1 2.6.38-rc3 | |
75 | February 7 2.6.38-rc4 | |
76 | February 15 2.6.38-rc5 | |
77 | February 21 2.6.38-rc6 | |
78 | March 1 2.6.38-rc7 | |
79 | March 7 2.6.38-rc8 | |
80 | March 14 2.6.38 stable release | |
75b02146 JC |
81 | |
82 | How do the developers decide when to close the development cycle and create | |
83 | the stable release? The most significant metric used is the list of | |
84 | regressions from previous releases. No bugs are welcome, but those which | |
85 | break systems which worked in the past are considered to be especially | |
86 | serious. For this reason, patches which cause regressions are looked upon | |
87 | unfavorably and are quite likely to be reverted during the stabilization | |
5c050fb9 | 88 | period. |
75b02146 JC |
89 | |
90 | The developers' goal is to fix all known regressions before the stable | |
91 | release is made. In the real world, this kind of perfection is hard to | |
92 | achieve; there are just too many variables in a project of this size. | |
93 | There comes a point where delaying the final release just makes the problem | |
94 | worse; the pile of changes waiting for the next merge window will grow | |
95 | larger, creating even more regressions the next time around. So most 2.6.x | |
96 | kernels go out with a handful of known regressions though, hopefully, none | |
97 | of them are serious. | |
98 | ||
99 | Once a stable release is made, its ongoing maintenance is passed off to the | |
5c050fb9 JC |
100 | "stable team," currently consisting of Greg Kroah-Hartman. The stable team |
101 | will release occasional updates to the stable release using the 2.6.x.y | |
102 | numbering scheme. To be considered for an update release, a patch must (1) | |
103 | fix a significant bug, and (2) already be merged into the mainline for the | |
104 | next development kernel. Kernels will typically receive stable updates for | |
105 | a little more than one development cycle past their initial release. So, | |
106 | for example, the 2.6.36 kernel's history looked like: | |
107 | ||
108 | October 10 2.6.36 stable release | |
109 | November 22 2.6.36.1 | |
110 | December 9 2.6.36.2 | |
111 | January 7 2.6.36.3 | |
112 | February 17 2.6.36.4 | |
113 | ||
114 | 2.6.36.4 was the final stable update for the 2.6.36 release. | |
115 | ||
116 | Some kernels are designated "long term" kernels; they will receive support | |
117 | for a longer period. As of this writing, the current long term kernels | |
118 | and their maintainers are: | |
119 | ||
120 | 2.6.27 Willy Tarreau (Deep-frozen stable kernel) | |
121 | 2.6.32 Greg Kroah-Hartman | |
122 | 2.6.35 Andi Kleen (Embedded flag kernel) | |
123 | ||
124 | The selection of a kernel for long-term support is purely a matter of a | |
125 | maintainer having the need and the time to maintain that release. There | |
126 | are no known plans for long-term support for any specific upcoming | |
127 | release. | |
75b02146 JC |
128 | |
129 | ||
130 | 2.2: THE LIFECYCLE OF A PATCH | |
131 | ||
132 | Patches do not go directly from the developer's keyboard into the mainline | |
133 | kernel. There is, instead, a somewhat involved (if somewhat informal) | |
134 | process designed to ensure that each patch is reviewed for quality and that | |
135 | each patch implements a change which is desirable to have in the mainline. | |
136 | This process can happen quickly for minor fixes, or, in the case of large | |
137 | and controversial changes, go on for years. Much developer frustration | |
138 | comes from a lack of understanding of this process or from attempts to | |
5c050fb9 | 139 | circumvent it. |
75b02146 JC |
140 | |
141 | In the hopes of reducing that frustration, this document will describe how | |
142 | a patch gets into the kernel. What follows below is an introduction which | |
143 | describes the process in a somewhat idealized way. A much more detailed | |
144 | treatment will come in later sections. | |
145 | ||
146 | The stages that a patch goes through are, generally: | |
147 | ||
148 | - Design. This is where the real requirements for the patch - and the way | |
149 | those requirements will be met - are laid out. Design work is often | |
150 | done without involving the community, but it is better to do this work | |
151 | in the open if at all possible; it can save a lot of time redesigning | |
152 | things later. | |
153 | ||
154 | - Early review. Patches are posted to the relevant mailing list, and | |
155 | developers on that list reply with any comments they may have. This | |
156 | process should turn up any major problems with a patch if all goes | |
157 | well. | |
158 | ||
159 | - Wider review. When the patch is getting close to ready for mainline | |
ef0eba47 | 160 | inclusion, it should be accepted by a relevant subsystem maintainer - |
75b02146 JC |
161 | though this acceptance is not a guarantee that the patch will make it |
162 | all the way to the mainline. The patch will show up in the maintainer's | |
e4fabad3 | 163 | subsystem tree and into the -next trees (described below). When the |
75b02146 JC |
164 | process works, this step leads to more extensive review of the patch and |
165 | the discovery of any problems resulting from the integration of this | |
166 | patch with work being done by others. | |
167 | ||
ef0eba47 RD |
168 | - Please note that most maintainers also have day jobs, so merging |
169 | your patch may not be their highest priority. If your patch is | |
170 | getting feedback about changes that are needed, you should either | |
171 | make those changes or justify why they should not be made. If your | |
172 | patch has no review complaints but is not being merged by its | |
173 | appropriate subsystem or driver maintainer, you should be persistent | |
174 | in updating the patch to the current kernel so that it applies cleanly | |
175 | and keep sending it for review and merging. | |
176 | ||
75b02146 JC |
177 | - Merging into the mainline. Eventually, a successful patch will be |
178 | merged into the mainline repository managed by Linus Torvalds. More | |
179 | comments and/or problems may surface at this time; it is important that | |
180 | the developer be responsive to these and fix any issues which arise. | |
181 | ||
182 | - Stable release. The number of users potentially affected by the patch | |
183 | is now large, so, once again, new problems may arise. | |
184 | ||
185 | - Long-term maintenance. While it is certainly possible for a developer | |
186 | to forget about code after merging it, that sort of behavior tends to | |
187 | leave a poor impression in the development community. Merging code | |
188 | eliminates some of the maintenance burden, in that others will fix | |
189 | problems caused by API changes. But the original developer should | |
190 | continue to take responsibility for the code if it is to remain useful | |
191 | in the longer term. | |
192 | ||
193 | One of the largest mistakes made by kernel developers (or their employers) | |
194 | is to try to cut the process down to a single "merging into the mainline" | |
195 | step. This approach invariably leads to frustration for everybody | |
196 | involved. | |
197 | ||
198 | ||
199 | 2.3: HOW PATCHES GET INTO THE KERNEL | |
200 | ||
201 | There is exactly one person who can merge patches into the mainline kernel | |
5c050fb9 JC |
202 | repository: Linus Torvalds. But, of the over 9,500 patches which went |
203 | into the 2.6.38 kernel, only 112 (around 1.3%) were directly chosen by Linus | |
75b02146 JC |
204 | himself. The kernel project has long since grown to a size where no single |
205 | developer could possibly inspect and select every patch unassisted. The | |
206 | way the kernel developers have addressed this growth is through the use of | |
207 | a lieutenant system built around a chain of trust. | |
208 | ||
209 | The kernel code base is logically broken down into a set of subsystems: | |
210 | networking, specific architecture support, memory management, video | |
211 | devices, etc. Most subsystems have a designated maintainer, a developer | |
212 | who has overall responsibility for the code within that subsystem. These | |
213 | subsystem maintainers are the gatekeepers (in a loose way) for the portion | |
214 | of the kernel they manage; they are the ones who will (usually) accept a | |
215 | patch for inclusion into the mainline kernel. | |
216 | ||
217 | Subsystem maintainers each manage their own version of the kernel source | |
218 | tree, usually (but certainly not always) using the git source management | |
219 | tool. Tools like git (and related tools like quilt or mercurial) allow | |
220 | maintainers to track a list of patches, including authorship information | |
221 | and other metadata. At any given time, the maintainer can identify which | |
222 | patches in his or her repository are not found in the mainline. | |
223 | ||
224 | When the merge window opens, top-level maintainers will ask Linus to "pull" | |
225 | the patches they have selected for merging from their repositories. If | |
226 | Linus agrees, the stream of patches will flow up into his repository, | |
227 | becoming part of the mainline kernel. The amount of attention that Linus | |
228 | pays to specific patches received in a pull operation varies. It is clear | |
229 | that, sometimes, he looks quite closely. But, as a general rule, Linus | |
230 | trusts the subsystem maintainers to not send bad patches upstream. | |
231 | ||
232 | Subsystem maintainers, in turn, can pull patches from other maintainers. | |
233 | For example, the networking tree is built from patches which accumulated | |
234 | first in trees dedicated to network device drivers, wireless networking, | |
235 | etc. This chain of repositories can be arbitrarily long, though it rarely | |
236 | exceeds two or three links. Since each maintainer in the chain trusts | |
237 | those managing lower-level trees, this process is known as the "chain of | |
5c050fb9 | 238 | trust." |
75b02146 JC |
239 | |
240 | Clearly, in a system like this, getting patches into the kernel depends on | |
241 | finding the right maintainer. Sending patches directly to Linus is not | |
242 | normally the right way to go. | |
243 | ||
244 | ||
e4fabad3 | 245 | 2.4: NEXT TREES |
75b02146 JC |
246 | |
247 | The chain of subsystem trees guides the flow of patches into the kernel, | |
248 | but it also raises an interesting question: what if somebody wants to look | |
249 | at all of the patches which are being prepared for the next merge window? | |
250 | Developers will be interested in what other changes are pending to see | |
251 | whether there are any conflicts to worry about; a patch which changes a | |
252 | core kernel function prototype, for example, will conflict with any other | |
253 | patches which use the older form of that function. Reviewers and testers | |
254 | want access to the changes in their integrated form before all of those | |
255 | changes land in the mainline kernel. One could pull changes from all of | |
256 | the interesting subsystem trees, but that would be a big and error-prone | |
257 | job. | |
258 | ||
e4fabad3 | 259 | The answer comes in the form of -next trees, where subsystem trees are |
75b02146 JC |
260 | collected for testing and review. The older of these trees, maintained by |
261 | Andrew Morton, is called "-mm" (for memory management, which is how it got | |
262 | started). The -mm tree integrates patches from a long list of subsystem | |
5c050fb9 | 263 | trees; it also has some patches aimed at helping with debugging. |
75b02146 JC |
264 | |
265 | Beyond that, -mm contains a significant collection of patches which have | |
266 | been selected by Andrew directly. These patches may have been posted on a | |
267 | mailing list, or they may apply to a part of the kernel for which there is | |
268 | no designated subsystem tree. As a result, -mm operates as a sort of | |
269 | subsystem tree of last resort; if there is no other obvious path for a | |
270 | patch into the mainline, it is likely to end up in -mm. Miscellaneous | |
271 | patches which accumulate in -mm will eventually either be forwarded on to | |
272 | an appropriate subsystem tree or be sent directly to Linus. In a typical | |
5c050fb9 JC |
273 | development cycle, approximately 5-10% of the patches going into the |
274 | mainline get there via -mm. | |
75b02146 | 275 | |
e6a591e5 RD |
276 | The current -mm patch is available in the "mmotm" (-mm of the moment) |
277 | directory at: | |
75b02146 | 278 | |
297957b4 | 279 | http://www.ozlabs.org/~akpm/mmotm/ |
75b02146 JC |
280 | |
281 | Use of the MMOTM tree is likely to be a frustrating experience, though; | |
282 | there is a definite chance that it will not even compile. | |
283 | ||
5c050fb9 | 284 | The primary tree for next-cycle patch merging is linux-next, maintained by |
75b02146 JC |
285 | Stephen Rothwell. The linux-next tree is, by design, a snapshot of what |
286 | the mainline is expected to look like after the next merge window closes. | |
287 | Linux-next trees are announced on the linux-kernel and linux-next mailing | |
288 | lists when they are assembled; they can be downloaded from: | |
289 | ||
297957b4 | 290 | http://www.kernel.org/pub/linux/kernel/next/ |
75b02146 | 291 | |
5c050fb9 JC |
292 | Linux-next has become an integral part of the kernel development process; |
293 | all patches merged during a given merge window should really have found | |
294 | their way into linux-next some time before the merge window opens. | |
295 | ||
75b02146 | 296 | |
f830673f AS |
297 | 2.4.1: STAGING TREES |
298 | ||
5c050fb9 | 299 | The kernel source tree contains the drivers/staging/ directory, where |
f830673f AS |
300 | many sub-directories for drivers or filesystems that are on their way to |
301 | being added to the kernel tree live. They remain in drivers/staging while | |
302 | they still need more work; once complete, they can be moved into the | |
303 | kernel proper. This is a way to keep track of drivers that aren't | |
304 | up to Linux kernel coding or quality standards, but people may want to use | |
305 | them and track development. | |
306 | ||
5c050fb9 JC |
307 | Greg Kroah-Hartman currently maintains the staging tree. Drivers that |
308 | still need work are sent to him, with each driver having its own | |
309 | subdirectory in drivers/staging/. Along with the driver source files, a | |
310 | TODO file should be present in the directory as well. The TODO file lists | |
311 | the pending work that the driver needs for acceptance into the kernel | |
312 | proper, as well as a list of people that should be Cc'd for any patches to | |
313 | the driver. Current rules require that drivers contributed to staging | |
314 | must, at a minimum, compile properly. | |
315 | ||
316 | Staging can be a relatively easy way to get new drivers into the mainline | |
317 | where, with luck, they will come to the attention of other developers and | |
318 | improve quickly. Entry into staging is not the end of the story, though; | |
319 | code in staging which is not seeing regular progress will eventually be | |
320 | removed. Distributors also tend to be relatively reluctant to enable | |
321 | staging drivers. So staging is, at best, a stop on the way toward becoming | |
322 | a proper mainline driver. | |
323 | ||
75b02146 JC |
324 | |
325 | 2.5: TOOLS | |
326 | ||
327 | As can be seen from the above text, the kernel development process depends | |
328 | heavily on the ability to herd collections of patches in various | |
329 | directions. The whole thing would not work anywhere near as well as it | |
330 | does without suitably powerful tools. Tutorials on how to use these tools | |
331 | are well beyond the scope of this document, but there is space for a few | |
332 | pointers. | |
333 | ||
334 | By far the dominant source code management system used by the kernel | |
335 | community is git. Git is one of a number of distributed version control | |
336 | systems being developed in the free software community. It is well tuned | |
337 | for kernel development, in that it performs quite well when dealing with | |
338 | large repositories and large numbers of patches. It also has a reputation | |
339 | for being difficult to learn and use, though it has gotten better over | |
340 | time. Some sort of familiarity with git is almost a requirement for kernel | |
341 | developers; even if they do not use it for their own work, they'll need git | |
342 | to keep up with what other developers (and the mainline) are doing. | |
343 | ||
344 | Git is now packaged by almost all Linux distributions. There is a home | |
ef0eba47 | 345 | page at: |
75b02146 | 346 | |
ef0eba47 | 347 | http://git-scm.com/ |
75b02146 | 348 | |
5c050fb9 | 349 | That page has pointers to documentation and tutorials. |
75b02146 JC |
350 | |
351 | Among the kernel developers who do not use git, the most popular choice is | |
352 | almost certainly Mercurial: | |
353 | ||
354 | http://www.selenic.com/mercurial/ | |
355 | ||
356 | Mercurial shares many features with git, but it provides an interface which | |
357 | many find easier to use. | |
358 | ||
359 | The other tool worth knowing about is Quilt: | |
360 | ||
361 | http://savannah.nongnu.org/projects/quilt/ | |
362 | ||
363 | Quilt is a patch management system, rather than a source code management | |
364 | system. It does not track history over time; it is, instead, oriented | |
365 | toward tracking a specific set of changes against an evolving code base. | |
366 | Some major subsystem maintainers use quilt to manage patches intended to go | |
367 | upstream. For the management of certain kinds of trees (-mm, for example), | |
368 | quilt is the best tool for the job. | |
369 | ||
370 | ||
371 | 2.6: MAILING LISTS | |
372 | ||
373 | A great deal of Linux kernel development work is done by way of mailing | |
374 | lists. It is hard to be a fully-functioning member of the community | |
375 | without joining at least one list somewhere. But Linux mailing lists also | |
376 | represent a potential hazard to developers, who risk getting buried under a | |
377 | load of electronic mail, running afoul of the conventions used on the Linux | |
378 | lists, or both. | |
379 | ||
380 | Most kernel mailing lists are run on vger.kernel.org; the master list can | |
381 | be found at: | |
382 | ||
383 | http://vger.kernel.org/vger-lists.html | |
384 | ||
385 | There are lists hosted elsewhere, though; a number of them are at | |
386 | lists.redhat.com. | |
387 | ||
388 | The core mailing list for kernel development is, of course, linux-kernel. | |
389 | This list is an intimidating place to be; volume can reach 500 messages per | |
390 | day, the amount of noise is high, the conversation can be severely | |
391 | technical, and participants are not always concerned with showing a high | |
392 | degree of politeness. But there is no other place where the kernel | |
393 | development community comes together as a whole; developers who avoid this | |
394 | list will miss important information. | |
395 | ||
396 | There are a few hints which can help with linux-kernel survival: | |
397 | ||
398 | - Have the list delivered to a separate folder, rather than your main | |
399 | mailbox. One must be able to ignore the stream for sustained periods of | |
400 | time. | |
401 | ||
402 | - Do not try to follow every conversation - nobody else does. It is | |
403 | important to filter on both the topic of interest (though note that | |
404 | long-running conversations can drift away from the original subject | |
405 | without changing the email subject line) and the people who are | |
5c050fb9 | 406 | participating. |
75b02146 JC |
407 | |
408 | - Do not feed the trolls. If somebody is trying to stir up an angry | |
409 | response, ignore them. | |
410 | ||
411 | - When responding to linux-kernel email (or that on other lists) preserve | |
412 | the Cc: header for all involved. In the absence of a strong reason (such | |
413 | as an explicit request), you should never remove recipients. Always make | |
414 | sure that the person you are responding to is in the Cc: list. This | |
415 | convention also makes it unnecessary to explicitly ask to be copied on | |
416 | replies to your postings. | |
417 | ||
418 | - Search the list archives (and the net as a whole) before asking | |
419 | questions. Some developers can get impatient with people who clearly | |
420 | have not done their homework. | |
421 | ||
422 | - Avoid top-posting (the practice of putting your answer above the quoted | |
423 | text you are responding to). It makes your response harder to read and | |
424 | makes a poor impression. | |
425 | ||
426 | - Ask on the correct mailing list. Linux-kernel may be the general meeting | |
427 | point, but it is not the best place to find developers from all | |
428 | subsystems. | |
429 | ||
430 | The last point - finding the correct mailing list - is a common place for | |
431 | beginning developers to go wrong. Somebody who asks a networking-related | |
432 | question on linux-kernel will almost certainly receive a polite suggestion | |
433 | to ask on the netdev list instead, as that is the list frequented by most | |
434 | networking developers. Other lists exist for the SCSI, video4linux, IDE, | |
435 | filesystem, etc. subsystems. The best place to look for mailing lists is | |
436 | in the MAINTAINERS file packaged with the kernel source. | |
437 | ||
438 | ||
439 | 2.7: GETTING STARTED WITH KERNEL DEVELOPMENT | |
440 | ||
441 | Questions about how to get started with the kernel development process are | |
442 | common - from both individuals and companies. Equally common are missteps | |
443 | which make the beginning of the relationship harder than it has to be. | |
444 | ||
445 | Companies often look to hire well-known developers to get a development | |
446 | group started. This can, in fact, be an effective technique. But it also | |
447 | tends to be expensive and does not do much to grow the pool of experienced | |
448 | kernel developers. It is possible to bring in-house developers up to speed | |
449 | on Linux kernel development, given the investment of a bit of time. Taking | |
450 | this time can endow an employer with a group of developers who understand | |
451 | the kernel and the company both, and who can help to train others as well. | |
452 | Over the medium term, this is often the more profitable approach. | |
453 | ||
454 | Individual developers are often, understandably, at a loss for a place to | |
455 | start. Beginning with a large project can be intimidating; one often wants | |
456 | to test the waters with something smaller first. This is the point where | |
457 | some developers jump into the creation of patches fixing spelling errors or | |
458 | minor coding style issues. Unfortunately, such patches create a level of | |
459 | noise which is distracting for the development community as a whole, so, | |
460 | increasingly, they are looked down upon. New developers wishing to | |
461 | introduce themselves to the community will not get the sort of reception | |
462 | they wish for by these means. | |
463 | ||
464 | Andrew Morton gives this advice for aspiring kernel developers | |
465 | ||
466 | The #1 project for all kernel beginners should surely be "make sure | |
467 | that the kernel runs perfectly at all times on all machines which | |
468 | you can lay your hands on". Usually the way to do this is to work | |
469 | with others on getting things fixed up (this can require | |
470 | persistence!) but that's fine - it's a part of kernel development. | |
471 | ||
472 | (http://lwn.net/Articles/283982/). | |
473 | ||
474 | In the absence of obvious problems to fix, developers are advised to look | |
475 | at the current lists of regressions and open bugs in general. There is | |
476 | never any shortage of issues in need of fixing; by addressing these issues, | |
477 | developers will gain experience with the process while, at the same time, | |
478 | building respect with the rest of the development community. |