1 <!--#include file="header.html" -->
3 <h3>Frequently Asked Questions</h3>
5 This is a collection of some of the more frequently asked questions
6 about BusyBox. Some of the questions even have answers. If you
7 have additions to this FAQ document, we would love to add them,
9 <h2>General questions</h2>
11 <li><a href="#getting_started">How can I get started using BusyBox?</a></li>
12 <li><a href="#configure">How do I configure busybox?</a></li>
13 <li><a href="#build_system">How do I build a BusyBox-based system?</a></li>
14 <li><a href="#kernel">Which Linux kernel versions are supported?</a></li>
15 <li><a href="#arch">Which architectures does BusyBox run on?</a></li>
16 <li><a href="#libc">Which C libraries are supported?</a></li>
17 <li><a href="#commercial">Can I include BusyBox as part of the software on my device?</a></li>
18 <li><a href="#external">Where can I find other small utilities since busybox does not include the features I want?</a></li></li>
19 <li><a href="#demanding">I demand that you to add <favorite feature> right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems <em>Right Now</em>!</a></li>
20 <li><a href="#helpme">I need help with BusyBox! What should I do?</a></li>
21 <li><a href="#contracts">I need you to add <favorite feature>! Are the BusyBox developers willing to be paid in order to fix bugs or add in <favorite feature>? Are you willing to provide support contracts?</a></li>
24 <h2>Troubleshooting</h2>
26 <li><a href="#bugs">I think I found a bug in BusyBox! What should I do?!</a></li>
27 <li><a href="#init">Busybox init isn't working!</a></li>
28 <li><a href="#sed">I can't configure busybox on my system.</a></li>
29 <li><a href="#job_control">Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?</a></li>
32 <h2>Programming questions</h2>
34 <li><a href="#goals">What are the goals of busybox?</a></li>
35 <li><a href="#design">What is the design of busybox?</a></li>
36 <li><a href="#source">How is the source code organized?</a></li>
38 <li><a href="#source_applets">The applet directories.</a></li>
39 <li><a href="#source_libbb">The busybox shared library (libbb)</a></li>
41 <li><a href="#optimize">I want to make busybox even smaller, how do I go about it?</a></li>
42 <li><a href="#adding">Adding an applet to busybox</a></li>
43 <li><a href="#standards">What standards does busybox adhere to?</a></li>
44 <li><a href="#portability">Portability.</a></li>
45 <li><a href="#tips">Tips and tricks.</a></li>
47 <li><a href="#tips_encrypted_passwords">Encrypted Passwords</a></li>
48 <li><a href="#tips_vfork">Fork and vfork</a></li>
49 <li><a href="#tips_short_read">Short reads and writes</a></li>
50 <li><a href="#tips_memory">Memory used by relocatable code, PIC, and static linking.</a></li>
51 <li><a href="#tips_kernel_headers">Including Linux kernel headers.</a></li>
53 <li><a href="#who">Who are the BusyBox developers?</a></li>
59 <h1>General questions</h1>
63 <h2><a name="getting_started">How can I get started using BusyBox?</a></h2>
64 <p> If you just want to try out busybox without installing it, download the
65 tarball, extract it, run "make defconfig", and then run "make".
68 This will create a busybox binary with almost all features enabled. To try
69 out a busybox applet, type "./busybox [appletname] [options]", for
70 example "./busybox ls -l" or "./busybox cat LICENSE". Type "./busybox"
71 to see a command list, and "busybox appletname --help" to see a brief
72 usage message for a given applet.
75 BusyBox uses the name it was invoked under to determine which applet is
76 being invoked. (Try "mv busybox ls" and then "./ls -l".) Installing
77 busybox consists of creating symlinks (or hardlinks) to the busybox
78 binary for each applet in busybox, and making sure these links are in
79 the shell's command $PATH. The special applet name "busybox" (or with
80 any optional suffix, such as "busybox-static") uses the first argument
81 to determine which applet to run, as shown above.
84 BusyBox also has a feature called the "standalone shell", where the busybox
85 shell runs any built-in applets before checking the command path. This
86 feature is also enabled by "make allyesconfig", and to try it out run
87 the command line "PATH= ./busybox ash". This will blank your command path
88 and run busybox as your command shell, so the only commands it can find
89 (without an explicit path such as /bin/ls) are the built-in busybox ones.
90 This is another good way to see what's built into busybox. (Note that the
91 standalone shell is dependent on the existence of /proc/self/exe, so before
92 using it in a chroot environment you must mount /proc.)
97 <h2><a name="configure">How do I configure busybox?</a></h2>
98 <p> Busybox is configured similarly to the linux kernel. Create a default
99 configuration and then run "make menuconfig" to modify it. The end
100 result is a .config file that tells the busybox build process what features
101 to include. So instead of "./configure; make; make install" the equivalent
102 busybox build would be "make defconfig; make; make install".
105 <p> Busybox configured with all features enabled is a little under a megabyte
106 dynamically linked on x86. To create a smaller busybox, configure it with
107 fewer features. Individual busybox applets cost anywhere from a few
108 hundred bytes to tens of kilobytes. Disable unneeded applets to save,
109 space, using menuconfig.
112 <p>The most important busybox configurators are:</p>
115 <li><p>make <b>defconfig</b> - Create the maximum "sane" configuration. This
116 enables almost all features, minus things like debugging options and features
117 that require changes to the rest of the system to work (such as selinux or
118 devfs device names). Use this if you want to start from a full-featured
119 busybox and remove features until it's small enough.</p></li>
120 <li><p>make <b>allnoconfig</b> - Disable everything. This creates a tiny version
121 of busybox that doesn't do anything. Start here if you know exactly what
122 you want and would like to select only those features.</p></li>
123 <li><p>make <b>menuconfig</b> - Interactively modify a .config file through a
124 multi-level menu interface. Use this after one of the previous two.</p></li>
127 <p>Some other configuration options are:</p>
129 <li><p>make <b>oldconfig</b> - Update an old .config file for a newer version
131 <li><p>make <b>allyesconfig</b> - Select absolutely everything. This creates
132 a statically linked version of busybox full of debug code, with dependencies on
133 selinux, using devfs names... This makes sure everything compiles. Whether
134 or not the result would do anything useful is an open question.</p></li>
135 <li><p>make <b>allbareconfig</b> - Select all applets but disable all sub-features
136 within each applet. More build coverage testing.</p></li>
137 <li><p>make <b>randconfig</b> - Create a random configuration for test purposes.</p></li>
140 <p> Menuconfig modifies your .config file through an interactive menu where you can enable or disable
141 busybox features, and get help about each feature.
146 To build a smaller busybox binary, run "make menuconfig" and disable the
147 features you don't need. (Or run "make allnoconfig" and then use
148 menuconfig to add just the features you need. Don't forget to recompile
149 with "make" once you've finished configuring.)
153 <h2><a name="build_system">How do I build a BusyBox-based system?</a></h2>
155 BusyBox is a package that replaces a dozen standard packages, but it is
156 not by itself a complete bootable system. Building an entire Linux
157 distribution from source is a bit beyond the scope of this FAQ, but it
158 understandably keeps cropping up on the mailing list, so here are some
162 Start by learning how to strip a working system down to the bare essentials
163 needed to run one or two commands, so you know what it is you actually
164 need. An excellent practical place to do
165 this is the <a href="http://www.tldp.org/HOWTO/Bootdisk-HOWTO/">Linux
166 BootDisk Howto</a>, or for a more theoretical approach try
167 <a href="http://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html">From
168 PowerUp to Bash Prompt</a>.
171 To learn how to build a working Linux system entirely from source code,
172 the place to go is the <a href="http://www.linuxfromscratch.org">Linux
173 From Scratch</a> project. They have an entire book of step-by-step
175 <a href="http://www.linuxfromscratch.org/lfs/view/stable/">read online</a>
177 <a href="http://www.linuxfromscratch.org/lfs/downloads/stable/">download</a>.
178 Be sure to check out the other sections of their main page, including
179 Beyond Linux From Scratch, Hardened Linux From Scratch, their Hints
180 directory, and their LiveCD project. (They also have mailing lists which
181 are better sources of answers to Linux-system building questions than
185 If you want an automated yet customizable system builder which produces
186 a BusyBox and uClibc based system, try
187 <a href="http://buildroot.uclibc.org">buildroot</a>, which is
188 another project by the maintainer of the uClibc (Erik Andersen).
189 Download the tarball, extract it, unset CC, make.
190 For more instructions, see the website.
195 <h2><a name="kernel">Which Linux kernel versions are supported?</a></h2>
197 Full functionality requires Linux 2.4.x or better. (Earlier versions may
198 still work, but are no longer regularly tested.) A large fraction of the
199 code should run on just about anything. While the current code is fairly
200 Linux specific, it should be fairly easy to port the majority of the code
201 to support, say, FreeBSD or Solaris, or Mac OS X, or even Windows (if you
202 are into that sort of thing).
206 <h2><a name="arch">Which architectures does BusyBox run on?</a></h2>
208 BusyBox in general will build on any architecture supported by gcc.
209 Kernel module loading for 2.4 Linux kernels is currently
210 limited to ARM, CRIS, H8/300, x86, ia64, x86_64, m68k, MIPS, PowerPC,
211 S390, SH3/4/5, Sparc, v850e, and x86_64 for 2.4.x kernels.
214 With 2.6.x kernels, module loading support should work on all architectures.
218 <h2><a name="libc">Which C libraries are supported?</a></h2>
220 On Linux, BusyBox releases are tested against uClibc (0.9.27 or later) and
221 glibc (2.2 or later). Both should provide full functionality with busybox,
222 and if you find a bug we want to hear about it.
225 Linux-libc5 is no longer maintained (and has no known advantages over
226 uClibc), dietlibc is known to have numerous unfixed bugs, and klibc is
227 missing too many features to build BusyBox. If you require a small C
228 library for Linux, the busybox developers recommend uClibc.
231 Some BusyBox applets have been built and run under a combination
232 of newlib and libgloss (see
233 <a href="http://www.busybox.net/lists/busybox/2005-March/013759.html">this thread</a>).
234 This is still experimental, but may be supported in a future release.
239 <h2><a name="commercial">Can I include BusyBox as part of the software on my device?</a></h2>
243 Yes. As long as you <a href="http://busybox.net/license.html">fully comply
244 with the generous terms of the GPL BusyBox license</a> you can ship BusyBox
245 as part of the software on your device.
250 <h2><a name="external">where can i find other small utilities since busybox
251 does not include the features i want?</a></h2>
253 we maintain such a <a href="tinyutils.html">list</a> on this site!
258 <h2><a name="demanding">I demand that you to add <favorite feature> right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems <em>Right Now</em>!</a></h2>
261 You have not paid us a single cent and yet you still have the product of
262 many years of our work. We are not your slaves! We work on BusyBox
263 because we find it useful and interesting. If you go off flaming us, we
269 <h2><a name="helpme">I need help with BusyBox! What should I do?</a></h2>
272 If you find that you need help with BusyBox, you can ask for help on the
273 BusyBox mailing list at busybox@busybox.net.</p>
275 <p> In addition to the mailing list, Erik Andersen (andersee), Manuel Nova
276 (mjn3), Rob Landley (landley), Mike Frysinger (SpanKY), Bernhard Fischer
277 (blindvt), and other long-time BusyBox developers are known to hang out
278 on the uClibc IRC channel: #uclibc on irc.freenode.net. There is a
279 <a href="http://ibot.Rikers.org/%23uclibc/">web archive of
280 daily logs of the #uclibc IRC channel</a> going back to 2002.
284 <b>Please do not send private email to Rob, Erik, Manuel, or the other
285 BusyBox contributors asking for private help unless you are planning on
286 paying for consulting services.</b>
290 When we answer questions on the BusyBox mailing list, it helps everyone
291 since people with similar problems in the future will be able to get help
292 by searching the mailing list archives. Private help is reserved as a paid
293 service. If you need to use private communication, or if you are serious
294 about getting timely assistance with BusyBox, you should seriously consider
295 paying for consulting services.
300 <h2><a name="contracts">I need you to add <favorite feature>! Are the BusyBox developers willing to be paid in order to fix bugs or add in <favorite feature>? Are you willing to provide support contracts?</a></h2>
304 Yes we are. The easy way to sponsor a new feature is to post an offer on
305 the mailing list to see who's interested. You can also email the project's
306 maintainer and ask them to recommend someone.
309 <p> If you prefer to deal with an organization rather than an individual, Rob
310 Landley (the current BusyBox maintainer) works for
311 <a http://www.timesys.com>TimeSys</a>, and Eric Andersen (the previous
312 busybox maintainer and current uClibc maintainer) owns
313 <a href="http://codepoet-consulting.com/">CodePoet Consulting</a>. Both
314 companies offer support contracts and handle new development, and there
315 are plenty of other companies that do the same.
321 <h1>Troubleshooting</h1>
325 <h2><a name="bugs">I think I found a bug in BusyBox! What should I do?</a></h2>
329 If you simply need help with using or configuring BusyBox, please submit a
330 detailed description of your problem to the BusyBox mailing list at <a
331 href="mailto:busybox@busybox.net"> busybox@busybox.net</a>.
332 Please do not send email to individual developers asking
333 for private help unless you are planning on paying for consulting services.
334 When we answer questions on the BusyBox mailing list, it helps everyone,
335 while private answers help only you...
339 The developers of BusyBox are busy people, and have only so much they can
340 keep in their brains at a time. As a result, bug reports and new feature
341 patches sometimes get lost when posted to the mailing list. To prevent
342 your bug report from getting lost, if you find a bug in BusyBox that isn't
343 immediately addressed, please use the <a
344 href="http://bugs.busybox.net/">BusyBox Bug and Patch Tracking System</a>
345 to submit a detailed explanation and we'll get to it as soon as we can.
350 <h2><a name="init">Busybox init isn't working!</a></h2>
352 Build a statically linked version of the following "hello world" program
353 with your cross compiler toolchain.
356 #include <stdio.h>
358 int main(int argc, char *argv)
360 printf("Hello world!\n");
366 Now try to boot your device with an "init=" argument pointing to your
367 hello world program. Did you see the hello world message? Until you
368 do, don't bother messing with busybox init.
372 Once you've got it working statically linked, try getting it to work
373 dynamically linked. Then read the FAQ entry <a href="#build_system">How
374 do I build a BusyBox-based system?</a>
379 <h2><a name="sed">I can't configure busybox on my system.</a></h2>
381 Configuring Busybox depends on a recent version of sed. Older
382 distributions (Red Hat 7.2, Debian 3.0) may not come with a
383 usable version. Luckily BusyBox can use its own sed to configure itself,
384 although this leads to a bit of a chicken and egg problem.
385 You can work around this by hand-configuring busybox to build with just
386 sed, then putting that sed in your path to configure the rest of busybox
391 tar xvjf sources/busybox-x.x.x.tar.bz2
394 make include/bb_config.h
395 echo "CONFIG_SED=y" >> .config
396 echo "#undef ENABLE_SED" >> include/bb_config.h
397 echo "#define ENABLE_SED 1" >> include/bb_config.h
400 export PATH=`pwd`:"$PATH"
403 <p>Then you can run "make defconfig" or "make menuconfig" normally.</p>
407 <h2><a name="job_control">Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?</a></h2>
410 Job control will be turned off since your shell can not obtain a controlling
411 terminal. This typically happens when you run your shell on /dev/console.
412 The kernel will not provide a controlling terminal on the /dev/console
413 device. Your should run your shell on a normal tty such as tty1 or ttyS0
414 and everything will work perfectly. If you <em>REALLY</em> want your shell
415 to run on /dev/console, then you can hack your kernel (if you are into that
416 sortof thing) by changing drivers/char/tty_io.c to change the lines where
417 it sets "noctty = 1;" to instead set it to "0". I recommend you instead
418 run your shell on a real console...
423 <h2><b><a name="goals">What are the goals of busybox?</a></b></h2>
425 <p>Busybox aims to be the smallest and simplest correct implementation of the
426 standard Linux command line tools. First and foremost, this means the
427 smallest executable size we can manage. We also want to have the simplest
428 and cleanest implementation we can manage, be <a href="#standards">standards
429 compliant</a>, minimize run-time memory usage (heap and stack), run fast, and
430 take over the world.</p>
432 <h2><b><a name="design">What is the design of busybox?</a></b></h2>
434 <p>Busybox is like a swiss army knife: one thing with many functions.
435 The busybox executable can act like many different programs depending on
436 the name used to invoke it. Normal practice is to create a bunch of symlinks
437 pointing to the busybox binary, each of which triggers a different busybox
438 function. (See <a href="FAQ.html#getting_started">getting started</a> in the
439 FAQ for more information on usage, and <a href="BusyBox.html">the
440 busybox documentation</a> for a list of symlink names and what they do.)
442 <p>The "one binary to rule them all" approach is primarily for size reasons: a
443 single multi-purpose executable is smaller then many small files could be.
444 This way busybox only has one set of ELF headers, it can easily share code
445 between different apps even when statically linked, it has better packing
446 efficiency by avoding gaps between files or compression dictionary resets,
449 <p>Work is underway on new options such as "make standalone" to build separate
450 binaries for each applet, and a "libbb.so" to make the busybox common code
451 available as a shared library. Neither is ready yet at the time of this
454 <a name="source"></a>
456 <h2><a name="source_applets"><b>The applet directories</b></a></h2>
458 <p>The directory "applets" contains the busybox startup code (applets.c and
459 busybox.c), and several subdirectories containing the code for the individual
462 <p>Busybox execution starts with the main() function in applets/busybox.c,
463 which sets the global variable bb_applet_name to argv[0] and calls
464 run_applet_by_name() in applets/applets.c. That uses the applets[] array
465 (defined in include/busybox.h and filled out in include/applets.h) to
466 transfer control to the appropriate APPLET_main() function (such as
467 cat_main() or sed_main()). The individual applet takes it from there.</p>
469 <p>This is why calling busybox under a different name triggers different
470 functionality: main() looks up argv[0] in applets[] to get a function pointer
471 to APPLET_main().</p>
473 <p>Busybox applets may also be invoked through the multiplexor applet
474 "busybox" (see busybox_main() in applets/busybox.c), and through the
475 standalone shell (grep for STANDALONE_SHELL in applets/shell/*.c).
476 See <a href="FAQ.html#getting_started">getting started</a> in the
477 FAQ for more information on these alternate usage mechanisms, which are
478 just different ways to reach the relevant APPLET_main() function.</p>
480 <p>The applet subdirectories (archival, console-tools, coreutils,
481 debianutils, e2fsprogs, editors, findutils, init, loginutils, miscutils,
482 modutils, networking, procps, shell, sysklogd, and util-linux) correspond
483 to the configuration sub-menus in menuconfig. Each subdirectory contains the
484 code to implement the applets in that sub-menu, as well as a Config.in
485 file defining that configuration sub-menu (with dependencies and help text
486 for each applet), and the makefile segment (Makefile.in) for that
489 <p>The run-time --help is stored in usage_messages[], which is initialized at
490 the start of applets/applets.c and gets its help text from usage.h. During the
491 build this help text is also used to generate the BusyBox documentation (in
492 html, txt, and man page formats) in the docs directory. See
493 <a href="#adding">adding an applet to busybox</a> for more
496 <h2><a name="source_libbb"><b>libbb</b></a></h2>
498 <p>Most non-setup code shared between busybox applets lives in the libbb
499 directory. It's a mess that evolved over the years without much auditing
500 or cleanup. For anybody looking for a great project to break into busybox
501 development with, documenting libbb would be both incredibly useful and good
504 <p>Common themes in libbb include allocation functions that test
505 for failure and abort the program with an error message so the caller doesn't
506 have to test the return value (xmalloc(), xstrdup(), etc), wrapped versions
507 of open(), close(), read(), and write() that test for their own failures
508 and/or retry automatically, linked list management functions (llist.c),
509 command line argument parsing (getopt_ulflags.c), and a whole lot more.</p>
513 <h2><a name="optimize">I want to make busybox even smaller, how do I go about it?</a></h2>
515 To conserve bytes it's good to know where they're being used, and the
516 size of the final executable isn't always a reliable indicator of
517 the size of the components (since various structures are rounded up,
518 so a small change may not even be visible by itself, but many small
522 <p> The busybox Makefile builds two versions of busybox, one of which
523 (busybox_unstripped) has extra information that various analysis tools
524 can use. (This has nothing to do with CONFIG_DEBUG, leave that off
525 when trying to optimize for size.)
528 <p> The <b>"make bloatcheck"</b> option uses Matt Mackall's bloat-o-meter
529 script to compare two versions of busybox (busybox_unstripped vs
530 busybox_old), and report which symbols changed size and by how much.
531 To use it, first build a base version with <b>"make baseline"</b>.
532 (This creates busybox_old, which should have the original sizes for
533 comparison purposes.) Then build the new version with your changes
534 and run "make bloatcheck" to see the size differences from the old
538 The first line of output has totals: how many symbols were added or
539 removed, how many symbols grew or shrank, the number of bytes added
540 and number of bytes removed by these changes, and finally the total
541 number of bytes difference between the two files. The remaining
542 lines show each individual symbol, the old and new sizes, and the
543 increase or decrease in size (which results are sorted by).
546 The <b>"make sizes"</b> option produces raw symbol size information for
547 busybox_unstripped. This is the output from the "nm --size-sort"
548 command (see "man nm" for more information), and is the information
549 bloat-o-meter parses to produce the comparison report above. For
550 defconfig, this is a good way to find the largest symbols in the tree
551 (which is a good place to start when trying to shrink the code). To
552 take a closer look at individual applets, configure busybox with just
553 one applet (run "make allnoconfig" and then switch on a single applet
554 with menuconfig), and then use "make sizes" to see the size of that
558 The "showasm" command (in the scripts directory) produces an assembly
559 dump of a function, providing a closer look at what changed. Try
560 "scripts/showasm busybox_unstripped" to list available symbols, and
561 "scripts/showasm busybox_unstripped symbolname" to see the assembly
562 for a sepecific symbol.
568 <h2><a name="adding"><b>Adding an applet to busybox</b></a></h2>
570 <p>To add a new applet to busybox, first pick a name for the applet and
571 a corresponding CONFIG_NAME. Then do this:</p>
574 <li>Figure out where in the busybox source tree your applet best fits,
575 and put your source code there. Be sure to use APPLET_main() instead
576 of main(), where APPLET is the name of your applet.</li>
578 <li>Add your applet to the relevant Config.in file (which file you add
579 it to determines where it shows up in "make menuconfig"). This uses
580 the same general format as the linux kernel's configuration system.</li>
582 <li>Add your applet to the relevant Makefile.in file (in the same
583 directory as the Config.in you chose), using the existing entries as a
584 template and the same CONFIG symbol as you used for Config.in. (Don't
585 forget "needlibm" or "needcrypt" if your applet needs libm or
588 <li>Add your applet to "include/applets.h", using one of the existing
589 entries as a template. (Note: this is in alphabetical order. Applets
590 are found via binary search, and if you add an applet out of order it
593 <li>Add your applet's runtime help text to "include/usage.h". You need
594 at least appname_trivial_usage (the minimal help text, always included
595 in the busybox binary when this applet is enabled) and appname_full_usage
596 (extra help text included in the busybox binary with
597 CONFIG_FEATURE_VERBOSE_USAGE is enabled), or it won't compile.
598 The other two help entry types (appname_example_usage and
599 appname_notes_usage) are optional. They don't take up space in the binary,
600 but instead show up in the generated documentation (BusyBox.html,
601 BusyBox.txt, and the man page BusyBox.1).</li>
603 <li>Run menuconfig, switch your applet on, compile, test, and fix the
604 bugs. Be sure to try both "allyesconfig" and "allnoconfig" (and
605 "allbareconfig" if relevant).</li>
609 <h2><a name="standards">What standards does busybox adhere to?</a></h2>
611 <p>The standard we're paying attention to is the "Shell and Utilities"
612 portion of the <a href="http://www.opengroup.org/onlinepubs/009695399/">Open
613 Group Base Standards</a> (also known as the Single Unix Specification version
614 3 or SUSv3). Note that paying attention isn't necessarily the same thing as
617 <p>SUSv3 doesn't even mention things like init, mount, tar, or losetup, nor
618 commonly used options like echo's '-e' and '-n', or sed's '-i'. Busybox is
619 driven by what real users actually need, not the fact the standard believes
620 we should implement ed or sccs. For size reasons, we're unlikely to include
621 much internationalization support beyond UTF-8, and on top of all that, our
622 configuration menu lets developers chop out features to produce smaller but
623 very non-standard utilities.</p>
625 <p>Also, Busybox is aimed primarily at Linux. Unix standards are interesting
626 because Linux tries to adhere to them, but portability to dozens of platforms
627 is only interesting in terms of offering a restricted feature set that works
628 everywhere, not growing dozens of platform-specific extensions. Busybox
629 should be portable to all hardware platforms Linux supports, and any other
630 similar operating systems that are easy to do and won't require much
633 <p>In practice, standards compliance tends to be a clean-up step once an
634 applet is otherwise finished. When polishing and testing a busybox applet,
635 we ensure we have at least the option of full standards compliance, or else
636 document where we (intentionally) fall short.</p>
638 <h2><a name="portability">Portability.</a></h2>
640 <p>Busybox is a Linux project, but that doesn't mean we don't have to worry
641 about portability. First of all, there are different hardware platforms,
642 different C library implementations, different versions of the kernel and
643 build toolchain... The file "include/platform.h" exists to centralize and
644 encapsulate various platform-specific things in one place, so most busybox
645 code doesn't have to care where it's running.</p>
647 <p>To start with, Linux runs on dozens of hardware platforms. We try to test
648 each release on x86, x86-64, arm, power pc, and mips. (Since qemu can handle
649 all of these, this isn't that hard.) This means we have to care about a number
650 of portability issues like endianness, word size, and alignment, all of which
651 belong in platform.h. That header handles conditional #includes and gives
652 us macros we can use in the rest of our code. At some point in the future
653 we might grow a platform.c, possibly even a platform subdirectory. As long
654 as the applets themselves don't have to care.</p>
656 <p>On a related note, we made the "default signedness of char varies" problem
657 go away by feeding the compiler -funsigned-char. This gives us consistent
658 behavior on all platforms, and defaults to 8-bit clean text processing (which
659 gets us halfway to UTF-8 support). NOMMU support is less easily separated
660 (see the tips section later in this document), but we're working on it.</p>
662 <p>Another type of portability is build environments: we unapologetically use
663 a number of gcc and glibc extensions (as does the Linux kernel), but these have
664 been picked up by packages like uClibc, TCC, and Intel's C Compiler. As for
665 gcc, we take advantage of newer compiler optimizations to get the smallest
666 possible size, but we also regression test against an older build environment
667 using the Red Hat 9 image at "http://busybox.net/downloads/qemu". This has a
668 2.4 kernel, gcc 3.2, make 3.79.1, and glibc 2.3, and is the oldest
669 build/deployment environment we still put any effort into maintaining. (If
670 anyone takes an interest in older kernels you're welcome to submit patches,
671 but the effort would probably be better spent
672 <a href="http://www.selenic.com/linux-tiny/">trimming
673 down the 2.6 kernel</a>.) Older gcc versions than that are uninteresting since
674 we now use c99 features, although
675 <a href="http://fabrice.bellard.free.fr/tcc/">tcc</a> might be worth a
678 <p>We also test busybox against the current release of uClibc. Older versions
679 of uClibc aren't very interesting (they were buggy, and uClibc wasn't really
680 usable as a general-purpose C library before version 0.9.26 anyway).</p>
682 <p>Other unix implementations are mostly uninteresting, since Linux binaries
683 have become the new standard for portable Unix programs. Specifically,
684 the ubiquity of Linux was cited as the main reason the Intel Binary
685 Compatability Standard 2 died, by the standards group organized to name a
686 successor to ibcs2: <a href="http://www.telly.org/86open/">the 86open
687 project</a>. That project disbanded in 1999 with the endorsement of an
688 existing standard: Linux ELF binaries. Since then, the major players at the
690 href=http://www-03.ibm.com/servers/aix/products/aixos/linux/index.html>AIX</a>, <a
691 href=http://www.sun.com/software/solaris/ds/linux_interop.jsp#3>Solaris</a>, and
692 <a href=http://www.onlamp.com/pub/a/bsd/2000/03/17/linuxapps.html>FreeBSD</a>)
693 have all either grown Linux support or folded.</p>
695 <p>The major exceptions are newcomer MacOS X, some embedded environments
696 (such as newlib+libgloss) which provide a posix environment but not a full
697 Linux environment, and environments like Cygwin that provide only partial Linux
698 emulation. Also, some embedded Linux systems run a Linux kernel but amputate
699 things like the /proc directory to save space.</p>
701 <p>Supporting these systems is largely a question of providing a clean subset
702 of BusyBox's functionality -- whichever applets can easily be made to
703 work in that environment. Annotating the configuration system to
704 indicate which applets require which prerequisites (such as procfs) is
705 also welcome. Other efforts to support these systems (swapping #include
706 files to build in different environments, adding adapter code to platform.h,
707 adding more extensive special-case supporting infrastructure such as mount's
708 legacy mtab support) are handled on a case-by-case basis. Support that can be
709 cleanly hidden in platform.h is reasonably attractive, and failing that
710 support that can be cleanly separated into a separate conditionally compiled
711 file is at least worth a look. Special-case code in the body of an applet is
712 something we're trying to avoid.</p>
714 <h2><a name="tips" />Programming tips and tricks.</a></h2>
716 <p>Various things busybox uses that aren't particularly well documented
719 <h2><a name="tips_encrypted_passwords">Encrypted Passwords</a></h2>
721 <p>Password fields in /etc/passwd and /etc/shadow are in a special format.
722 If the first character isn't '$', then it's an old DES style password. If
723 the first character is '$' then the password is actually three fields
724 separated by '$' characters:</p>
726 <b>$type$salt$encrypted_password</b>
729 <p>The "type" indicates which encryption algorithm to use: 1 for MD5 and 2 for SHA1.</p>
731 <p>The "salt" is a bunch of ramdom characters (generally 8) the encryption
732 algorithm uses to perturb the password in a known and reproducible way (such
733 as by appending the random data to the unencrypted password, or combining
734 them with exclusive or). Salt is randomly generated when setting a password,
735 and then the same salt value is re-used when checking the password. (Salt is
736 thus stored unencrypted.)</p>
738 <p>The advantage of using salt is that the same cleartext password encrypted
739 with a different salt value produces a different encrypted value.
740 If each encrypted password uses a different salt value, an attacker is forced
741 to do the cryptographic math all over again for each password they want to
742 check. Without salt, they could simply produce a big dictionary of commonly
743 used passwords ahead of time, and look up each password in a stolen password
744 file to see if it's a known value. (Even if there are billions of possible
745 passwords in the dictionary, checking each one is just a binary search against
746 a file only a few gigabytes long.) With salt they can't even tell if two
747 different users share the same password without guessing what that password
748 is and decrypting it. They also can't precompute the attack dictionary for
749 a specific password until they know what the salt value is.</p>
751 <p>The third field is the encrypted password (plus the salt). For md5 this
754 <p>The busybox function to handle all this is pw_encrypt(clear, salt) in
755 "libbb/pw_encrypt.c". The first argument is the clear text password to be
756 encrypted, and the second is a string in "$type$salt$password" format, from
757 which the "type" and "salt" fields will be extracted to produce an encrypted
758 value. (Only the first two fields are needed, the third $ is equivalent to
759 the end of the string.) The return value is an encrypted password in
760 /etc/passwd format, with all three $ separated fields. It's stored in
761 a static buffer, 128 bytes long.</p>
763 <p>So when checking an existing password, if pw_encrypt(text,
764 old_encrypted_password) returns a string that compares identical to
765 old_encrypted_password, you've got the right password. When setting a new
766 password, generate a random 8 character salt string, put it in the right
767 format with sprintf(buffer, "$%c$%s", type, salt), and feed buffer as the
768 second argument to pw_encrypt(text,buffer).</p>
770 <h2><a name="tips_vfork">Fork and vfork</a></h2>
772 <p>On systems that haven't got a Memory Management Unit, fork() is unreasonably
773 expensive to implement (and sometimes even impossible), so a less capable
774 function called vfork() is used instead. (Using vfork() on a system with an
775 MMU is like pounding a nail with a wrench. Not the best tool for the job, but
778 <p>Busybox hides the difference between fork() and vfork() in
779 libbb/bb_fork_exec.c. If you ever want to fork and exec, use bb_fork_exec()
780 (which returns a pid and takes the same arguments as execve(), although in
781 this case envp can be NULL) and don't worry about it. This description is
782 here in case you want to know why that does what it does.</p>
784 <p>Implementing fork() depends on having a Memory Management Unit. With an
785 MMU then you can simply set up a second set of page tables and share the
786 physical memory via copy-on-write. So a fork() followed quickly by exec()
787 only copies a few pages of the parent's memory, just the ones it changes
788 before freeing them.</p>
790 <p>With a very primitive MMU (using a base pointer plus length instead of page
791 tables, which can provide virtual addresses and protect processes from each
792 other, but no copy on write) you can still implement fork. But it's
793 unreasonably expensive, because you have to copy all the parent process'
794 memory into the new process (which could easily be several megabytes per fork).
795 And you have to do this even though that memory gets freed again as soon as the
796 exec happens. (This is not just slow and a waste of space but causes memory
797 usage spikes that can easily cause the system to run out of memory.)</p>
799 <p>Without even a primitive MMU, you have no virtual addresses. Every process
800 can reach out and touch any other process' memory, because all pointers are to
801 physical addresses with no protection. Even if you copy a process' memory to
802 new physical addresses, all of its pointers point to the old objects in the
803 old process. (Searching through the new copy's memory for pointers and
804 redirect them to the new locations is not an easy problem.)</p>
806 <p>So with a primitive or missing MMU, fork() is just not a good idea.</p>
808 <p>In theory, vfork() is just a fork() that writeably shares the heap and stack
809 rather than copying it (so what one process writes the other one sees). In
810 practice, vfork() has to suspend the parent process until the child does exec,
811 at which point the parent wakes up and resumes by returning from the call to
812 vfork(). All modern kernel/libc combinations implement vfork() to put the
813 parent to sleep until the child does its exec. There's just no other way to
814 make it work: the parent has to know the child has done its exec() or exit()
815 before it's safe to return from the function it's in, so it has to block
816 until that happens. In fact without suspending the parent there's no way to
817 even store separate copies of the return value (the pid) from the vfork() call
818 itself: both assignments write into the same memory location.</p>
820 <p>One way to understand (and in fact implement) vfork() is this: imagine
821 the parent does a setjmp and then continues on (pretending to be the child)
822 until the exec() comes around, then the _exec_ does the actual fork, and the
823 parent does a longjmp back to the original vfork call and continues on from
824 there. (It thus becomes obvious why the child can't return, or modify
825 local variables it doesn't want the parent to see changed when it resumes.)
827 <p>Note a common mistake: the need for vfork doesn't mean you can't have two
828 processes running at the same time. It means you can't have two processes
829 sharing the same memory without stomping all over each other. As soon as
830 the child calls exec(), the parent resumes.</p>
832 <p>If the child's attempt to call exec() fails, the child should call _exit()
833 rather than a normal exit(). This avoids any atexit() code that might confuse
834 the parent. (The parent should never call _exit(), only a vforked child that
837 <p>(Now in theory, a nommu system could just copy the _stack_ when it forks
838 (which presumably is much shorter than the heap), and leave the heap shared.
839 Even with no MMU at all
840 In practice, you've just wound up in a multi-threaded situation and you can't
841 do a malloc() or free() on your heap without freeing the other process' memory
842 (and if you don't have the proper locking for being threaded, corrupting the
843 heap if both of you try to do it at the same time and wind up stomping on
844 each other while traversing the free memory lists). The thing about vfork is
845 that it's a big red flag warning "there be dragons here" rather than
846 something subtle and thus even more dangerous.)</p>
848 <h2><a name="tips_sort_read">Short reads and writes</a></h2>
850 <p>Busybox has special functions, bb_full_read() and bb_full_write(), to
851 check that all the data we asked for got read or written. Is this a real
852 world consideration? Try the following:</p>
854 <pre>while true; do echo hello; sleep 1; done | tee out.txt</pre>
856 <p>If tee is implemented with bb_full_read(), tee doesn't display output
857 in real time but blocks until its entire input buffer (generally a couple
858 kilobytes) is read, then displays it all at once. In that case, we _want_
859 the short read, for user interface reasons. (Note that read() should never
860 return 0 unless it has hit the end of input, and an attempt to write 0
861 bytes should be ignored by the OS.)</p>
863 <p>As for short writes, play around with two processes piping data to each
864 other on the command line (cat bigfile | gzip > out.gz) and suspend and
865 resume a few times (ctrl-z to suspend, "fg" to resume). The writer can
866 experience short writes, which are especially dangerous because if you don't
867 notice them you'll discard data. They can also happen when a system is under
868 load and a fast process is piping to a slower one. (Such as an xterm waiting
869 on x11 when the scheduler decides X is being a CPU hog with all that
870 text console scrolling...)</p>
872 <p>So will data always be read from the far end of a pipe at the
873 same chunk sizes it was written in? Nope. Don't rely on that. For one
874 counterexample, see <a href="http://www.faqs.org/rfcs/rfc896.html">rfc 896
875 for Nagle's algorithm</a>, which waits a fraction of a second or so before
876 sending out small amounts of data through a TCP/IP connection in case more
877 data comes in that can be merged into the same packet. (In case you were
878 wondering why action games that use TCP/IP set TCP_NODELAY to lower the latency
879 on their their sockets, now you know.)</p>
881 <h2><a name="tips_memory">Memory used by relocatable code, PIC, and static linking.</a></h2>
883 <p>The downside of standard dynamic linking is that it results in self-modifying
884 code. Although each executable's pages are mmaped() into a process' address
885 space from the executable file and are thus naturally shared between processes
886 out of the page cache, the library loader (ld-linux.so.2 or ld-uClibc.so.0)
887 writes to these pages to supply addresses for relocatable symbols. This
888 dirties the pages, triggering copy-on-write allocation of new memory for each
889 processes' dirtied pages.</p>
891 <p>One solution to this is Position Independent Code (PIC), a way of linking
892 a file so all the relocations are grouped together. This dirties fewer
893 pages (often just a single page) for each process' relocations. The down
894 side is this results in larger executables, which take up more space on disk
895 (and a correspondingly larger space in memory). But when many copies of the
896 same program are running, PIC dynamic linking trades a larger disk footprint
897 for a smaller memory footprint, by sharing more pages.</p>
899 <p>A third solution is static linking. A statically linked program has no
900 relocations, and thus the entire executable is shared between all running
901 instances. This tends to have a significantly larger disk footprint, but
902 on a system with only one or two executables, shared libraries aren't much
905 <p>You can tell the glibc linker to display debugging information about its
906 relocations with the environment variable "LD_DEBUG". Try
907 "LD_DEBUG=help /bin/true" for a list of commands. Learning to interpret
908 "LD_DEBUG=statistics cat /proc/self/statm" could be interesting.</p>
910 <p>For more on this topic, here's Rich Felker:</p>
912 <p>Dynamic linking (without fixed load addresses) fundamentally requires
913 at least one dirty page per dso that uses symbols. Making calls (but
914 never taking the address explicitly) to functions within the same dso
915 does not require a dirty page by itself, but will with ELF unless you
916 use -Bsymbolic or hidden symbols when linking.</p>
918 <p>ELF uses significant additional stack space for the kernel to pass all
919 the ELF data structures to the newly created process image. These are
920 located above the argument list and environment. This normally adds 1
921 dirty page to the process size.</p>
923 <p>The ELF dynamic linker has its own data segment, adding one or more
924 dirty pages. I believe it also performs relocations on itself.</p>
926 <p>The ELF dynamic linker makes significant dynamic allocations to manage
927 the global symbol table and the loaded dso's. This data is never
928 freed. It will be needed again if libdl is used, so unconditionally
929 freeing it is not possible, but normal programs do not use libdl. Of
930 course with glibc all programs use libdl (due to nsswitch) so the
931 issue was never addressed.</p>
933 <p>ELF also has the issue that segments are not page-aligned on disk.
934 This saves up to 4k on disk, but at the expense of using an additional
935 dirty page in most cases, due to a large portion of the first data
936 page being filled with a duplicate copy of the last text page.</p>
938 <p>The above is just a partial list of the tiny memory penalties of ELF
939 dynamic linking, which eventually add up to quite a bit. The smallest
940 I've been able to get a process down to is 8 dirty pages, and the
941 above factors seem to mostly account for it (but some were difficult
945 <h2><a name="tips_kernel_headers"></a>Including kernel headers</h2>
947 <p>The "linux" or "asm" directories of /usr/include contain Linux kernel
948 headers, so that the C library can talk directly to the Linux kernel. In
949 a perfect world, applications shouldn't include these headers directly, but
950 we don't live in a perfect world.</p>
952 <p>For example, Busybox's losetup code wants linux/loop.c because nothing else
953 #defines the structures to call the kernel's loopback device setup ioctls.
954 Attempts to cut and paste the information into a local busybox header file
955 proved incredibly painful, because portions of the loop_info structure vary by
956 architecture, namely the type __kernel_dev_t has different sizes on alpha,
957 arm, x86, and so on. Meaning we either #include <linux/posix_types.h> or
958 we hardwire #ifdefs to check what platform we're building on and define this
959 type appropriately for every single hardware architecture supported by
960 Linux, which is simply unworkable.</p>
962 <p>This is aside from the fact that the relevant type defined in
963 posix_types.h was renamed to __kernel_old_dev_t during the 2.5 series, so
964 to cut and paste the structure into our header we have to #include
965 <linux/version.h> to figure out which name to use. (What we actually do is
966 check if we're building on 2.6, and if so just use the new 64 bit structure
967 instead to avoid the rename entirely.) But we still need the version
968 check, since 2.4 didn't have the 64 bit structure.</p>
970 <p>The BusyBox developers spent <u>two years</u> trying to figure
971 out a clean way to do all this. There isn't one. The losetup in the
972 util-linux package from kernel.org isn't doing it cleanly either, they just
973 hide the ugliness by nesting #include files. Their mount/loop.h
974 #includes "my_dev_t.h", which #includes <linux/posix_types.h> and
975 <linux/version.h> just like we do. There simply is no alternative.</p>
977 <p>Just because directly #including kernel headers is sometimes
978 unavoidable doesn't me we should include them when there's a better
979 way to do it. However, block copying information out of the kernel headers
980 is not a better way.</p>
982 <h2><a name="who">Who are the BusyBox developers?</a></h2>
984 <p>The following login accounts currently exist on busybox.net. (I.E. these
985 people can commit <a href="http://busybox.net/downloads/patches">patches</a>
986 into subversion for the BusyBox, uClibc, and buildroot projects.)</p>
989 aldot :Bernhard Fischer
990 andersen :Erik Andersen <- uClibc and BuildRoot maintainer.
992 davidm :David McCullough
993 gkajmowi :Garrett Kajmowicz <- uClibc++ maintainer
994 jbglaw :Jan-Benedict Glaw
995 jocke :Joakim Tjernlund
996 landley :Rob Landley <- BusyBox maintainer
998 mjn3 :Manuel Novoa III
1001 pkj :Peter Kjellerstedt
1002 prpplague :David Anders
1003 psm :Peter S. Mazinger
1005 sandman :Robert Griebl
1006 sjhill :Steven J. Hill
1009 tobiasa :Tobias Anderberg
1010 vapier :Mike Frysinger
1013 <p>The following accounts used to exist on busybox.net, but don't anymore so
1014 I can't ask /etc/passwd for their names. (If anybody would like to make
1015 a stab at it...)</p>
1021 erik : Also Erik Andersen?
1030 vodz :Vladimir N. Oleynik
1038 <!--#include file="footer.html" -->