1 <!--#include file="header.html" -->
3 <h3>Frequently Asked Questions</h3>
5 This is a collection of some of the more frequently asked questions
6 about BusyBox. Some of the questions even have answers. If you
7 have additions to this FAQ document, we would love to add them,
9 <h2>General questions</h2>
11 <li><a href="#getting_started">How can I get started using BusyBox?</a></li>
12 <li><a href="#configure">How do I configure busybox?</a></li>
13 <li><a href="#build_system">How do I build a BusyBox-based system?</a></li>
14 <li><a href="#kernel">Which Linux kernel versions are supported?</a></li>
15 <li><a href="#arch">Which architectures does BusyBox run on?</a></li>
16 <li><a href="#libc">Which C libraries are supported?</a></li>
17 <li><a href="#commercial">Can I include BusyBox as part of the software on my device?</a></li>
18 <li><a href="#external">Where can I find other small utilities since busybox does not include the features I want?</a></li></li>
19 <li><a href="#demanding">I demand that you to add <favorite feature> right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems <em>Right Now</em>!</a></li>
20 <li><a href="#helpme">I need help with BusyBox! What should I do?</a></li>
21 <li><a href="#contracts">I need you to add <favorite feature>! Are the BusyBox developers willing to be paid in order to fix bugs or add in <favorite feature>? Are you willing to provide support contracts?</a></li>
24 <h2>Troubleshooting</h2>
26 <li><a href="#bugs">I think I found a bug in BusyBox! What should I do?!</a></li>
27 <li><a href="#init">Busybox init isn't working!</a></li>
28 <li><a href="#sed">I can't configure busybox on my system.</a></li>
29 <li><a href="#job_control">Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?</a></li>
32 <h2>Programming questions</h2>
34 <li><a href="#goals">What are the goals of busybox?</a></li>
35 <li><a href="#design">What is the design of busybox?</a></li>
36 <li><a href="#source">How is the source code organized?</a></li>
38 <li><a href="#source_applets">The applet directories.</a></li>
39 <li><a href="#source_libbb">The busybox shared library (libbb)</a></li>
41 <li><a href="#optimize">I want to make busybox even smaller, how do I go about it?</a></li>
42 <li><a href="#adding">Adding an applet to busybox</a></li>
43 <li><a href="#standards">What standards does busybox adhere to?</a></li>
44 <li><a href="#portability">Portability.</a></li>
45 <li><a href="#tips">Tips and tricks.</a></li>
47 <li><a href="#tips_encrypted_passwords">Encrypted Passwords</a></li>
48 <li><a href="#tips_vfork">Fork and vfork</a></li>
49 <li><a href="#tips_short_read">Short reads and writes</a></li>
50 <li><a href="#tips_memory">Memory used by relocatable code, PIC, and static linking.</a></li>
51 <li><a href="#tips_kernel_headers">Including Linux kernel headers.</a></li>
53 <li><a href="#who">Who are the BusyBox developers?</a></li>
59 <h1>General questions</h1>
63 <h2><a name="getting_started">How can I get started using BusyBox?</a></h2>
64 <p> If you just want to try out busybox without installing it, download the
65 tarball, extract it, run "make defconfig", and then run "make".
68 This will create a busybox binary with almost all features enabled. To try
69 out a busybox applet, type "./busybox [appletname] [options]", for
70 example "./busybox ls -l" or "./busybox cat LICENSE". Type "./busybox"
71 to see a command list, and "busybox appletname --help" to see a brief
72 usage message for a given applet.
75 BusyBox uses the name it was invoked under to determine which applet is
76 being invoked. (Try "mv busybox ls" and then "./ls -l".) Installing
77 busybox consists of creating symlinks (or hardlinks) to the busybox
78 binary for each applet in busybox, and making sure these links are in
79 the shell's command $PATH. The special applet name "busybox" (or with
80 any optional suffix, such as "busybox-static") uses the first argument
81 to determine which applet to run, as shown above.
84 BusyBox also has a feature called the "standalone shell", where the busybox
85 shell runs any built-in applets before checking the command path. This
86 feature is also enabled by "make allyesconfig", and to try it out run
87 the command line "PATH= ./busybox ash". This will blank your command path
88 and run busybox as your command shell, so the only commands it can find
89 (without an explicit path such as /bin/ls) are the built-in busybox ones.
90 This is another good way to see what's built into busybox. (Note that the
91 standalone shell is dependent on the existence of /proc/self/exe, so before
92 using it in a chroot environment you must mount /proc.)
97 <h2><a name="configure">How do I configure busybox?</a></h2>
98 <p> Busybox is configured similarly to the linux kernel. Create a default
99 configuration and then run "make menuconfig" to modify it. The end
100 result is a .config file that tells the busybox build process what features
101 to include. So instead of "./configure; make; make install" the equivalent
102 busybox build would be "make defconfig; make; make install".
105 <p> Busybox configured with all features enabled is a little under a megabyte
106 dynamically linked on x86. To create a smaller busybox, configure it with
107 fewer features. Individual busybox applets cost anywhere from a few
108 hundred bytes to tens of kilobytes. Disable unneeded applets to save,
109 space, using menuconfig.
112 <p>The most important busybox configurators are:</p>
115 <li><p>make <b>defconfig</b> - Create the maximum "sane" configuration. This
116 enables almost all features, minus things like debugging options and features
117 that require changes to the rest of the system to work (such as selinux or
118 devfs device names). Use this if you want to start from a full-featured
119 busybox and remove features until it's small enough.</p></li>
120 <li><p>make <b>allnoconfig</b> - Disable everything. This creates a tiny version
121 of busybox that doesn't do anything. Start here if you know exactly what
122 you want and would like to select only those features.</p></li>
123 <li><p>make <b>menuconfig</b> - Interactively modify a .config file through a
124 multi-level menu interface. Use this after one of the previous two.</p></li>
127 <p>Some other configuration options are:</p>
129 <li><p>make <b>oldconfig</b> - Update an old .config file for a newer version
131 <li><p>make <b>allyesconfig</b> - Select absolutely everything. This creates
132 a statically linked version of busybox full of debug code, with dependencies on
133 selinux, using devfs names... This makes sure everything compiles. Whether
134 or not the result would do anything useful is an open question.</p></li>
135 <li><p>make <b>allbareconfig</b> - Select all applets but disable all sub-features
136 within each applet. More build coverage testing.</p></li>
137 <li><p>make <b>randconfig</b> - Create a random configuration for test purposes.</p></li>
140 <p> Menuconfig modifies your .config file through an interactive menu where you can enable or disable
141 busybox features, and get help about each feature.
146 To build a smaller busybox binary, run "make menuconfig" and disable the
147 features you don't need. (Or run "make allnoconfig" and then use
148 menuconfig to add just the features you need. Don't forget to recompile
149 with "make" once you've finished configuring.)
153 <h2><a name="build_system">How do I build a BusyBox-based system?</a></h2>
155 BusyBox is a package that replaces a dozen standard packages, but it is
156 not by itself a complete bootable system. Building an entire Linux
157 distribution from source is a bit beyond the scope of this FAQ, but it
158 understandably keeps cropping up on the mailing list, so here are some
162 Start by learning how to strip a working system down to the bare essentials
163 needed to run one or two commands, so you know what it is you actually
164 need. An excellent practical place to do
165 this is the <a href="http://www.tldp.org/HOWTO/Bootdisk-HOWTO/">Linux
166 BootDisk Howto</a>, or for a more theoretical approach try
167 <a href="http://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html">From
168 PowerUp to Bash Prompt</a>.
171 To learn how to build a working Linux system entirely from source code,
172 the place to go is the <a href="http://www.linuxfromscratch.org">Linux
173 From Scratch</a> project. They have an entire book of step-by-step
175 <a href="http://www.linuxfromscratch.org/lfs/view/stable/">read online</a>
177 <a href="http://www.linuxfromscratch.org/lfs/downloads/stable/">download</a>.
178 Be sure to check out the other sections of their main page, including
179 Beyond Linux From Scratch, Hardened Linux From Scratch, their Hints
180 directory, and their LiveCD project. (They also have mailing lists which
181 are better sources of answers to Linux-system building questions than
185 If you want an automated yet customizable system builder which produces
186 a BusyBox and uClibc based system, try
187 <a href="http://buildroot.uclibc.org">buildroot</a>, which is
188 another project by the maintainer of the uClibc (Erik Andersen).
189 Download the tarball, extract it, unset CC, make.
190 For more instructions, see the website.
195 <h2><a name="kernel">Which Linux kernel versions are supported?</a></h2>
197 Full functionality requires Linux 2.4.x or better. (Earlier versions may
198 still work, but are no longer regularly tested.) A large fraction of the
199 code should run on just about anything. While the current code is fairly
200 Linux specific, it should be fairly easy to port the majority of the code
201 to support, say, FreeBSD or Solaris, or Mac OS X, or even Windows (if you
202 are into that sort of thing).
206 <h2><a name="arch">Which architectures does BusyBox run on?</a></h2>
208 BusyBox in general will build on any architecture supported by gcc.
209 Kernel module loading for 2.4 Linux kernels is currently
210 limited to ARM, CRIS, H8/300, x86, ia64, x86_64, m68k, MIPS, PowerPC,
211 S390, SH3/4/5, Sparc, v850e, and x86_64 for 2.4.x kernels.
214 With 2.6.x kernels, module loading support should work on all architectures.
218 <h2><a name="libc">Which C libraries are supported?</a></h2>
220 On Linux, BusyBox releases are tested against uClibc (0.9.27 or later) and
221 glibc (2.2 or later). Both should provide full functionality with busybox,
222 and if you find a bug we want to hear about it.
225 Linux-libc5 is no longer maintained (and has no known advantages over
226 uClibc), dietlibc is known to have numerous unfixed bugs, and klibc is
227 missing too many features to build BusyBox. If you require a small C
228 library for Linux, the busybox developers recommend uClibc.
231 Some BusyBox applets have been built and run under a combination
232 of newlib and libgloss (see
233 <a href="http://www.busybox.net/lists/busybox/2005-March/013759.html">this thread</a>).
234 This is still experimental, but may be supported in a future release.
239 <h2><a name="commercial">Can I include BusyBox as part of the software on my device?</a></h2>
243 Yes. As long as you <a href="http://busybox.net/license.html">fully comply
244 with the generous terms of the GPL BusyBox license</a> you can ship BusyBox
245 as part of the software on your device.
250 <h2><a name="external">where can i find other small utilities since busybox
251 does not include the features i want?</a></h2>
253 we maintain such a <a href="tinyutils.html">list</a> on this site!
258 <h2><a name="demanding">I demand that you to add <favorite feature> right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems <em>Right Now</em>!</a></h2>
261 You have not paid us a single cent and yet you still have the product of
262 many years of our work. We are not your slaves! We work on BusyBox
263 because we find it useful and interesting. If you go off flaming us, we
269 <h2><a name="helpme">I need help with BusyBox! What should I do?</a></h2>
272 If you find that you need help with BusyBox, you can ask for help on the
273 BusyBox mailing list at busybox@busybox.net.</p>
275 <p> In addition to the mailing list, Erik Andersen (andersee), Manuel Nova
276 (mjn3), Rob Landley (landley), Mike Frysinger (SpanKY), Bernhard Fischer
277 (blindvt), and other long-time BusyBox developers are known to hang out
278 on the uClibc IRC channel: #uclibc on irc.freenode.net. There is a
279 <a href="http://ibot.Rikers.org/%23uclibc/">web archive of
280 daily logs of the #uclibc IRC channel</a> going back to 2002.
284 <b>Please do not send private email to Rob, Erik, Manuel, or the other
285 BusyBox contributors asking for private help unless you are planning on
286 paying for consulting services.</b>
290 When we answer questions on the BusyBox mailing list, it helps everyone
291 since people with similar problems in the future will be able to get help
292 by searching the mailing list archives. Private help is reserved as a paid
293 service. If you need to use private communication, or if you are serious
294 about getting timely assistance with BusyBox, you should seriously consider
295 paying for consulting services.
300 <h2><a name="contracts">I need you to add <favorite feature>! Are the BusyBox developers willing to be paid in order to fix bugs or add in <favorite feature>? Are you willing to provide support contracts?</a></h2>
304 Yes we are. The easy way to sponsor a new feature is to post an offer on
305 the mailing list to see who's interested. You can also email the project's
306 maintainer and ask them to recommend someone.
309 <p> If you prefer to deal with an organization rather than an individual, Rob
310 Landley (the current BusyBox maintainer) works for
311 <a http://www.timesys.com>TimeSys</a>, and Eric Andersen (the previous
312 busybox maintainer and current uClibc maintainer) owns
313 <a href="http://codepoet-consulting.com/">CodePoet Consulting</a>. Both
314 companies offer support contracts and handle new development, and there
315 are plenty of other companies that do the same.
321 <h1>Troubleshooting</h1>
325 <h2><a name="bugs">I think I found a bug in BusyBox! What should I do?</a></h2>
329 If you simply need help with using or configuring BusyBox, please submit a
330 detailed description of your problem to the BusyBox mailing list at <a
331 href="mailto:busybox@busybox.net"> busybox@busybox.net</a>.
332 Please do not send email to individual developers asking
333 for private help unless you are planning on paying for consulting services.
334 When we answer questions on the BusyBox mailing list, it helps everyone,
335 while private answers help only you...
339 The developers of BusyBox are busy people, and have only so much they can
340 keep in their brains at a time. As a result, bug reports and new feature
341 patches sometimes get lost when posted to the mailing list. To prevent
342 your bug report from getting lost, if you find a bug in BusyBox that isn't
343 immediately addressed, please use the <a
344 href="http://bugs.busybox.net/">BusyBox Bug and Patch Tracking System</a>
345 to submit a detailed explanation and we'll get to it as soon as we can.
350 <h2><a name="init">Busybox init isn't working!</a></h2>
352 Init is the first program that runs, so it might be that no programs are
353 working on your new system because of a problem with your cross-compiler,
354 kernel, console settings, shared libraries, root filesystem... To rule all
355 that out, first build a statically linked version of the following "hello
356 world" program with your cross compiler toolchain:
359 #include <stdio.h>
361 int main(int argc, char *argv)
363 printf("Hello world!\n");
369 Now try to boot your device with an "init=" argument pointing to your
370 hello world program. Did you see the hello world message? Until you
371 do, don't bother messing with busybox init.
375 Once you've got it working statically linked, try getting it to work
376 dynamically linked. Then read the FAQ entry <a href="#build_system">How
377 do I build a BusyBox-based system?</a>, and the
378 <a href="/downloads/BusyBox.html#item_init">documentation for BusyBox
384 <h2><a name="sed">I can't configure busybox on my system.</a></h2>
386 Configuring Busybox depends on a recent version of sed. Older
387 distributions (Red Hat 7.2, Debian 3.0) may not come with a
388 usable version. Luckily BusyBox can use its own sed to configure itself,
389 although this leads to a bit of a chicken and egg problem.
390 You can work around this by hand-configuring busybox to build with just
391 sed, then putting that sed in your path to configure the rest of busybox
396 tar xvjf sources/busybox-x.x.x.tar.bz2
399 make include/bb_config.h
400 echo "CONFIG_SED=y" >> .config
401 echo "#undef ENABLE_SED" >> include/bb_config.h
402 echo "#define ENABLE_SED 1" >> include/bb_config.h
405 export PATH=`pwd`:"$PATH"
408 <p>Then you can run "make defconfig" or "make menuconfig" normally.</p>
412 <h2><a name="job_control">Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?</a></h2>
415 Job control will be turned off since your shell can not obtain a controlling
416 terminal. This typically happens when you run your shell on /dev/console.
417 The kernel will not provide a controlling terminal on the /dev/console
418 device. Your should run your shell on a normal tty such as tty1 or ttyS0
419 and everything will work perfectly. If you <em>REALLY</em> want your shell
420 to run on /dev/console, then you can hack your kernel (if you are into that
421 sortof thing) by changing drivers/char/tty_io.c to change the lines where
422 it sets "noctty = 1;" to instead set it to "0". I recommend you instead
423 run your shell on a real console...
428 <h2><b><a name="goals">What are the goals of busybox?</a></b></h2>
430 <p>Busybox aims to be the smallest and simplest correct implementation of the
431 standard Linux command line tools. First and foremost, this means the
432 smallest executable size we can manage. We also want to have the simplest
433 and cleanest implementation we can manage, be <a href="#standards">standards
434 compliant</a>, minimize run-time memory usage (heap and stack), run fast, and
435 take over the world.</p>
437 <h2><b><a name="design">What is the design of busybox?</a></b></h2>
439 <p>Busybox is like a swiss army knife: one thing with many functions.
440 The busybox executable can act like many different programs depending on
441 the name used to invoke it. Normal practice is to create a bunch of symlinks
442 pointing to the busybox binary, each of which triggers a different busybox
443 function. (See <a href="FAQ.html#getting_started">getting started</a> in the
444 FAQ for more information on usage, and <a href="BusyBox.html">the
445 busybox documentation</a> for a list of symlink names and what they do.)
447 <p>The "one binary to rule them all" approach is primarily for size reasons: a
448 single multi-purpose executable is smaller then many small files could be.
449 This way busybox only has one set of ELF headers, it can easily share code
450 between different apps even when statically linked, it has better packing
451 efficiency by avoding gaps between files or compression dictionary resets,
454 <p>Work is underway on new options such as "make standalone" to build separate
455 binaries for each applet, and a "libbb.so" to make the busybox common code
456 available as a shared library. Neither is ready yet at the time of this
459 <a name="source"></a>
461 <h2><a name="source_applets"><b>The applet directories</b></a></h2>
463 <p>The directory "applets" contains the busybox startup code (applets.c and
464 busybox.c), and several subdirectories containing the code for the individual
467 <p>Busybox execution starts with the main() function in applets/busybox.c,
468 which sets the global variable bb_applet_name to argv[0] and calls
469 run_applet_by_name() in applets/applets.c. That uses the applets[] array
470 (defined in include/busybox.h and filled out in include/applets.h) to
471 transfer control to the appropriate APPLET_main() function (such as
472 cat_main() or sed_main()). The individual applet takes it from there.</p>
474 <p>This is why calling busybox under a different name triggers different
475 functionality: main() looks up argv[0] in applets[] to get a function pointer
476 to APPLET_main().</p>
478 <p>Busybox applets may also be invoked through the multiplexor applet
479 "busybox" (see busybox_main() in applets/busybox.c), and through the
480 standalone shell (grep for STANDALONE_SHELL in applets/shell/*.c).
481 See <a href="FAQ.html#getting_started">getting started</a> in the
482 FAQ for more information on these alternate usage mechanisms, which are
483 just different ways to reach the relevant APPLET_main() function.</p>
485 <p>The applet subdirectories (archival, console-tools, coreutils,
486 debianutils, e2fsprogs, editors, findutils, init, loginutils, miscutils,
487 modutils, networking, procps, shell, sysklogd, and util-linux) correspond
488 to the configuration sub-menus in menuconfig. Each subdirectory contains the
489 code to implement the applets in that sub-menu, as well as a Config.in
490 file defining that configuration sub-menu (with dependencies and help text
491 for each applet), and the makefile segment (Makefile.in) for that
494 <p>The run-time --help is stored in usage_messages[], which is initialized at
495 the start of applets/applets.c and gets its help text from usage.h. During the
496 build this help text is also used to generate the BusyBox documentation (in
497 html, txt, and man page formats) in the docs directory. See
498 <a href="#adding">adding an applet to busybox</a> for more
501 <h2><a name="source_libbb"><b>libbb</b></a></h2>
503 <p>Most non-setup code shared between busybox applets lives in the libbb
504 directory. It's a mess that evolved over the years without much auditing
505 or cleanup. For anybody looking for a great project to break into busybox
506 development with, documenting libbb would be both incredibly useful and good
509 <p>Common themes in libbb include allocation functions that test
510 for failure and abort the program with an error message so the caller doesn't
511 have to test the return value (xmalloc(), xstrdup(), etc), wrapped versions
512 of open(), close(), read(), and write() that test for their own failures
513 and/or retry automatically, linked list management functions (llist.c),
514 command line argument parsing (getopt_ulflags.c), and a whole lot more.</p>
518 <h2><a name="optimize">I want to make busybox even smaller, how do I go about it?</a></h2>
520 To conserve bytes it's good to know where they're being used, and the
521 size of the final executable isn't always a reliable indicator of
522 the size of the components (since various structures are rounded up,
523 so a small change may not even be visible by itself, but many small
527 <p> The busybox Makefile builds two versions of busybox, one of which
528 (busybox_unstripped) has extra information that various analysis tools
529 can use. (This has nothing to do with CONFIG_DEBUG, leave that off
530 when trying to optimize for size.)
533 <p> The <b>"make bloatcheck"</b> option uses Matt Mackall's bloat-o-meter
534 script to compare two versions of busybox (busybox_unstripped vs
535 busybox_old), and report which symbols changed size and by how much.
536 To use it, first build a base version with <b>"make baseline"</b>.
537 (This creates busybox_old, which should have the original sizes for
538 comparison purposes.) Then build the new version with your changes
539 and run "make bloatcheck" to see the size differences from the old
543 The first line of output has totals: how many symbols were added or
544 removed, how many symbols grew or shrank, the number of bytes added
545 and number of bytes removed by these changes, and finally the total
546 number of bytes difference between the two files. The remaining
547 lines show each individual symbol, the old and new sizes, and the
548 increase or decrease in size (which results are sorted by).
551 The <b>"make sizes"</b> option produces raw symbol size information for
552 busybox_unstripped. This is the output from the "nm --size-sort"
553 command (see "man nm" for more information), and is the information
554 bloat-o-meter parses to produce the comparison report above. For
555 defconfig, this is a good way to find the largest symbols in the tree
556 (which is a good place to start when trying to shrink the code). To
557 take a closer look at individual applets, configure busybox with just
558 one applet (run "make allnoconfig" and then switch on a single applet
559 with menuconfig), and then use "make sizes" to see the size of that
563 The "showasm" command (in the scripts directory) produces an assembly
564 dump of a function, providing a closer look at what changed. Try
565 "scripts/showasm busybox_unstripped" to list available symbols, and
566 "scripts/showasm busybox_unstripped symbolname" to see the assembly
567 for a sepecific symbol.
573 <h2><a name="adding"><b>Adding an applet to busybox</b></a></h2>
575 <p>To add a new applet to busybox, first pick a name for the applet and
576 a corresponding CONFIG_NAME. Then do this:</p>
579 <li>Figure out where in the busybox source tree your applet best fits,
580 and put your source code there. Be sure to use APPLET_main() instead
581 of main(), where APPLET is the name of your applet.</li>
583 <li>Add your applet to the relevant Config.in file (which file you add
584 it to determines where it shows up in "make menuconfig"). This uses
585 the same general format as the linux kernel's configuration system.</li>
587 <li>Add your applet to the relevant Makefile.in file (in the same
588 directory as the Config.in you chose), using the existing entries as a
589 template and the same CONFIG symbol as you used for Config.in. (Don't
590 forget "needlibm" or "needcrypt" if your applet needs libm or
593 <li>Add your applet to "include/applets.h", using one of the existing
594 entries as a template. (Note: this is in alphabetical order. Applets
595 are found via binary search, and if you add an applet out of order it
598 <li>Add your applet's runtime help text to "include/usage.h". You need
599 at least appname_trivial_usage (the minimal help text, always included
600 in the busybox binary when this applet is enabled) and appname_full_usage
601 (extra help text included in the busybox binary with
602 CONFIG_FEATURE_VERBOSE_USAGE is enabled), or it won't compile.
603 The other two help entry types (appname_example_usage and
604 appname_notes_usage) are optional. They don't take up space in the binary,
605 but instead show up in the generated documentation (BusyBox.html,
606 BusyBox.txt, and the man page BusyBox.1).</li>
608 <li>Run menuconfig, switch your applet on, compile, test, and fix the
609 bugs. Be sure to try both "allyesconfig" and "allnoconfig" (and
610 "allbareconfig" if relevant).</li>
614 <h2><a name="standards">What standards does busybox adhere to?</a></h2>
616 <p>The standard we're paying attention to is the "Shell and Utilities"
617 portion of the <a href="http://www.opengroup.org/onlinepubs/009695399/">Open
618 Group Base Standards</a> (also known as the Single Unix Specification version
619 3 or SUSv3). Note that paying attention isn't necessarily the same thing as
622 <p>SUSv3 doesn't even mention things like init, mount, tar, or losetup, nor
623 commonly used options like echo's '-e' and '-n', or sed's '-i'. Busybox is
624 driven by what real users actually need, not the fact the standard believes
625 we should implement ed or sccs. For size reasons, we're unlikely to include
626 much internationalization support beyond UTF-8, and on top of all that, our
627 configuration menu lets developers chop out features to produce smaller but
628 very non-standard utilities.</p>
630 <p>Also, Busybox is aimed primarily at Linux. Unix standards are interesting
631 because Linux tries to adhere to them, but portability to dozens of platforms
632 is only interesting in terms of offering a restricted feature set that works
633 everywhere, not growing dozens of platform-specific extensions. Busybox
634 should be portable to all hardware platforms Linux supports, and any other
635 similar operating systems that are easy to do and won't require much
638 <p>In practice, standards compliance tends to be a clean-up step once an
639 applet is otherwise finished. When polishing and testing a busybox applet,
640 we ensure we have at least the option of full standards compliance, or else
641 document where we (intentionally) fall short.</p>
643 <h2><a name="portability">Portability.</a></h2>
645 <p>Busybox is a Linux project, but that doesn't mean we don't have to worry
646 about portability. First of all, there are different hardware platforms,
647 different C library implementations, different versions of the kernel and
648 build toolchain... The file "include/platform.h" exists to centralize and
649 encapsulate various platform-specific things in one place, so most busybox
650 code doesn't have to care where it's running.</p>
652 <p>To start with, Linux runs on dozens of hardware platforms. We try to test
653 each release on x86, x86-64, arm, power pc, and mips. (Since qemu can handle
654 all of these, this isn't that hard.) This means we have to care about a number
655 of portability issues like endianness, word size, and alignment, all of which
656 belong in platform.h. That header handles conditional #includes and gives
657 us macros we can use in the rest of our code. At some point in the future
658 we might grow a platform.c, possibly even a platform subdirectory. As long
659 as the applets themselves don't have to care.</p>
661 <p>On a related note, we made the "default signedness of char varies" problem
662 go away by feeding the compiler -funsigned-char. This gives us consistent
663 behavior on all platforms, and defaults to 8-bit clean text processing (which
664 gets us halfway to UTF-8 support). NOMMU support is less easily separated
665 (see the tips section later in this document), but we're working on it.</p>
667 <p>Another type of portability is build environments: we unapologetically use
668 a number of gcc and glibc extensions (as does the Linux kernel), but these have
669 been picked up by packages like uClibc, TCC, and Intel's C Compiler. As for
670 gcc, we take advantage of newer compiler optimizations to get the smallest
671 possible size, but we also regression test against an older build environment
672 using the Red Hat 9 image at "http://busybox.net/downloads/qemu". This has a
673 2.4 kernel, gcc 3.2, make 3.79.1, and glibc 2.3, and is the oldest
674 build/deployment environment we still put any effort into maintaining. (If
675 anyone takes an interest in older kernels you're welcome to submit patches,
676 but the effort would probably be better spent
677 <a href="http://www.selenic.com/linux-tiny/">trimming
678 down the 2.6 kernel</a>.) Older gcc versions than that are uninteresting since
679 we now use c99 features, although
680 <a href="http://fabrice.bellard.free.fr/tcc/">tcc</a> might be worth a
683 <p>We also test busybox against the current release of uClibc. Older versions
684 of uClibc aren't very interesting (they were buggy, and uClibc wasn't really
685 usable as a general-purpose C library before version 0.9.26 anyway).</p>
687 <p>Other unix implementations are mostly uninteresting, since Linux binaries
688 have become the new standard for portable Unix programs. Specifically,
689 the ubiquity of Linux was cited as the main reason the Intel Binary
690 Compatability Standard 2 died, by the standards group organized to name a
691 successor to ibcs2: <a href="http://www.telly.org/86open/">the 86open
692 project</a>. That project disbanded in 1999 with the endorsement of an
693 existing standard: Linux ELF binaries. Since then, the major players at the
695 href=http://www-03.ibm.com/servers/aix/products/aixos/linux/index.html>AIX</a>, <a
696 href=http://www.sun.com/software/solaris/ds/linux_interop.jsp#3>Solaris</a>, and
697 <a href=http://www.onlamp.com/pub/a/bsd/2000/03/17/linuxapps.html>FreeBSD</a>)
698 have all either grown Linux support or folded.</p>
700 <p>The major exceptions are newcomer MacOS X, some embedded environments
701 (such as newlib+libgloss) which provide a posix environment but not a full
702 Linux environment, and environments like Cygwin that provide only partial Linux
703 emulation. Also, some embedded Linux systems run a Linux kernel but amputate
704 things like the /proc directory to save space.</p>
706 <p>Supporting these systems is largely a question of providing a clean subset
707 of BusyBox's functionality -- whichever applets can easily be made to
708 work in that environment. Annotating the configuration system to
709 indicate which applets require which prerequisites (such as procfs) is
710 also welcome. Other efforts to support these systems (swapping #include
711 files to build in different environments, adding adapter code to platform.h,
712 adding more extensive special-case supporting infrastructure such as mount's
713 legacy mtab support) are handled on a case-by-case basis. Support that can be
714 cleanly hidden in platform.h is reasonably attractive, and failing that
715 support that can be cleanly separated into a separate conditionally compiled
716 file is at least worth a look. Special-case code in the body of an applet is
717 something we're trying to avoid.</p>
719 <h2><a name="tips" />Programming tips and tricks.</a></h2>
721 <p>Various things busybox uses that aren't particularly well documented
724 <h2><a name="tips_encrypted_passwords">Encrypted Passwords</a></h2>
726 <p>Password fields in /etc/passwd and /etc/shadow are in a special format.
727 If the first character isn't '$', then it's an old DES style password. If
728 the first character is '$' then the password is actually three fields
729 separated by '$' characters:</p>
731 <b>$type$salt$encrypted_password</b>
734 <p>The "type" indicates which encryption algorithm to use: 1 for MD5 and 2 for SHA1.</p>
736 <p>The "salt" is a bunch of ramdom characters (generally 8) the encryption
737 algorithm uses to perturb the password in a known and reproducible way (such
738 as by appending the random data to the unencrypted password, or combining
739 them with exclusive or). Salt is randomly generated when setting a password,
740 and then the same salt value is re-used when checking the password. (Salt is
741 thus stored unencrypted.)</p>
743 <p>The advantage of using salt is that the same cleartext password encrypted
744 with a different salt value produces a different encrypted value.
745 If each encrypted password uses a different salt value, an attacker is forced
746 to do the cryptographic math all over again for each password they want to
747 check. Without salt, they could simply produce a big dictionary of commonly
748 used passwords ahead of time, and look up each password in a stolen password
749 file to see if it's a known value. (Even if there are billions of possible
750 passwords in the dictionary, checking each one is just a binary search against
751 a file only a few gigabytes long.) With salt they can't even tell if two
752 different users share the same password without guessing what that password
753 is and decrypting it. They also can't precompute the attack dictionary for
754 a specific password until they know what the salt value is.</p>
756 <p>The third field is the encrypted password (plus the salt). For md5 this
759 <p>The busybox function to handle all this is pw_encrypt(clear, salt) in
760 "libbb/pw_encrypt.c". The first argument is the clear text password to be
761 encrypted, and the second is a string in "$type$salt$password" format, from
762 which the "type" and "salt" fields will be extracted to produce an encrypted
763 value. (Only the first two fields are needed, the third $ is equivalent to
764 the end of the string.) The return value is an encrypted password in
765 /etc/passwd format, with all three $ separated fields. It's stored in
766 a static buffer, 128 bytes long.</p>
768 <p>So when checking an existing password, if pw_encrypt(text,
769 old_encrypted_password) returns a string that compares identical to
770 old_encrypted_password, you've got the right password. When setting a new
771 password, generate a random 8 character salt string, put it in the right
772 format with sprintf(buffer, "$%c$%s", type, salt), and feed buffer as the
773 second argument to pw_encrypt(text,buffer).</p>
775 <h2><a name="tips_vfork">Fork and vfork</a></h2>
777 <p>On systems that haven't got a Memory Management Unit, fork() is unreasonably
778 expensive to implement (and sometimes even impossible), so a less capable
779 function called vfork() is used instead. (Using vfork() on a system with an
780 MMU is like pounding a nail with a wrench. Not the best tool for the job, but
783 <p>Busybox hides the difference between fork() and vfork() in
784 libbb/bb_fork_exec.c. If you ever want to fork and exec, use bb_fork_exec()
785 (which returns a pid and takes the same arguments as execve(), although in
786 this case envp can be NULL) and don't worry about it. This description is
787 here in case you want to know why that does what it does.</p>
789 <p>Implementing fork() depends on having a Memory Management Unit. With an
790 MMU then you can simply set up a second set of page tables and share the
791 physical memory via copy-on-write. So a fork() followed quickly by exec()
792 only copies a few pages of the parent's memory, just the ones it changes
793 before freeing them.</p>
795 <p>With a very primitive MMU (using a base pointer plus length instead of page
796 tables, which can provide virtual addresses and protect processes from each
797 other, but no copy on write) you can still implement fork. But it's
798 unreasonably expensive, because you have to copy all the parent process'
799 memory into the new process (which could easily be several megabytes per fork).
800 And you have to do this even though that memory gets freed again as soon as the
801 exec happens. (This is not just slow and a waste of space but causes memory
802 usage spikes that can easily cause the system to run out of memory.)</p>
804 <p>Without even a primitive MMU, you have no virtual addresses. Every process
805 can reach out and touch any other process' memory, because all pointers are to
806 physical addresses with no protection. Even if you copy a process' memory to
807 new physical addresses, all of its pointers point to the old objects in the
808 old process. (Searching through the new copy's memory for pointers and
809 redirect them to the new locations is not an easy problem.)</p>
811 <p>So with a primitive or missing MMU, fork() is just not a good idea.</p>
813 <p>In theory, vfork() is just a fork() that writeably shares the heap and stack
814 rather than copying it (so what one process writes the other one sees). In
815 practice, vfork() has to suspend the parent process until the child does exec,
816 at which point the parent wakes up and resumes by returning from the call to
817 vfork(). All modern kernel/libc combinations implement vfork() to put the
818 parent to sleep until the child does its exec. There's just no other way to
819 make it work: the parent has to know the child has done its exec() or exit()
820 before it's safe to return from the function it's in, so it has to block
821 until that happens. In fact without suspending the parent there's no way to
822 even store separate copies of the return value (the pid) from the vfork() call
823 itself: both assignments write into the same memory location.</p>
825 <p>One way to understand (and in fact implement) vfork() is this: imagine
826 the parent does a setjmp and then continues on (pretending to be the child)
827 until the exec() comes around, then the _exec_ does the actual fork, and the
828 parent does a longjmp back to the original vfork call and continues on from
829 there. (It thus becomes obvious why the child can't return, or modify
830 local variables it doesn't want the parent to see changed when it resumes.)
832 <p>Note a common mistake: the need for vfork doesn't mean you can't have two
833 processes running at the same time. It means you can't have two processes
834 sharing the same memory without stomping all over each other. As soon as
835 the child calls exec(), the parent resumes.</p>
837 <p>If the child's attempt to call exec() fails, the child should call _exit()
838 rather than a normal exit(). This avoids any atexit() code that might confuse
839 the parent. (The parent should never call _exit(), only a vforked child that
842 <p>(Now in theory, a nommu system could just copy the _stack_ when it forks
843 (which presumably is much shorter than the heap), and leave the heap shared.
844 Even with no MMU at all
845 In practice, you've just wound up in a multi-threaded situation and you can't
846 do a malloc() or free() on your heap without freeing the other process' memory
847 (and if you don't have the proper locking for being threaded, corrupting the
848 heap if both of you try to do it at the same time and wind up stomping on
849 each other while traversing the free memory lists). The thing about vfork is
850 that it's a big red flag warning "there be dragons here" rather than
851 something subtle and thus even more dangerous.)</p>
853 <h2><a name="tips_sort_read">Short reads and writes</a></h2>
855 <p>Busybox has special functions, bb_full_read() and bb_full_write(), to
856 check that all the data we asked for got read or written. Is this a real
857 world consideration? Try the following:</p>
859 <pre>while true; do echo hello; sleep 1; done | tee out.txt</pre>
861 <p>If tee is implemented with bb_full_read(), tee doesn't display output
862 in real time but blocks until its entire input buffer (generally a couple
863 kilobytes) is read, then displays it all at once. In that case, we _want_
864 the short read, for user interface reasons. (Note that read() should never
865 return 0 unless it has hit the end of input, and an attempt to write 0
866 bytes should be ignored by the OS.)</p>
868 <p>As for short writes, play around with two processes piping data to each
869 other on the command line (cat bigfile | gzip > out.gz) and suspend and
870 resume a few times (ctrl-z to suspend, "fg" to resume). The writer can
871 experience short writes, which are especially dangerous because if you don't
872 notice them you'll discard data. They can also happen when a system is under
873 load and a fast process is piping to a slower one. (Such as an xterm waiting
874 on x11 when the scheduler decides X is being a CPU hog with all that
875 text console scrolling...)</p>
877 <p>So will data always be read from the far end of a pipe at the
878 same chunk sizes it was written in? Nope. Don't rely on that. For one
879 counterexample, see <a href="http://www.faqs.org/rfcs/rfc896.html">rfc 896
880 for Nagle's algorithm</a>, which waits a fraction of a second or so before
881 sending out small amounts of data through a TCP/IP connection in case more
882 data comes in that can be merged into the same packet. (In case you were
883 wondering why action games that use TCP/IP set TCP_NODELAY to lower the latency
884 on their their sockets, now you know.)</p>
886 <h2><a name="tips_memory">Memory used by relocatable code, PIC, and static linking.</a></h2>
888 <p>The downside of standard dynamic linking is that it results in self-modifying
889 code. Although each executable's pages are mmaped() into a process' address
890 space from the executable file and are thus naturally shared between processes
891 out of the page cache, the library loader (ld-linux.so.2 or ld-uClibc.so.0)
892 writes to these pages to supply addresses for relocatable symbols. This
893 dirties the pages, triggering copy-on-write allocation of new memory for each
894 processes' dirtied pages.</p>
896 <p>One solution to this is Position Independent Code (PIC), a way of linking
897 a file so all the relocations are grouped together. This dirties fewer
898 pages (often just a single page) for each process' relocations. The down
899 side is this results in larger executables, which take up more space on disk
900 (and a correspondingly larger space in memory). But when many copies of the
901 same program are running, PIC dynamic linking trades a larger disk footprint
902 for a smaller memory footprint, by sharing more pages.</p>
904 <p>A third solution is static linking. A statically linked program has no
905 relocations, and thus the entire executable is shared between all running
906 instances. This tends to have a significantly larger disk footprint, but
907 on a system with only one or two executables, shared libraries aren't much
910 <p>You can tell the glibc linker to display debugging information about its
911 relocations with the environment variable "LD_DEBUG". Try
912 "LD_DEBUG=help /bin/true" for a list of commands. Learning to interpret
913 "LD_DEBUG=statistics cat /proc/self/statm" could be interesting.</p>
915 <p>For more on this topic, here's Rich Felker:</p>
917 <p>Dynamic linking (without fixed load addresses) fundamentally requires
918 at least one dirty page per dso that uses symbols. Making calls (but
919 never taking the address explicitly) to functions within the same dso
920 does not require a dirty page by itself, but will with ELF unless you
921 use -Bsymbolic or hidden symbols when linking.</p>
923 <p>ELF uses significant additional stack space for the kernel to pass all
924 the ELF data structures to the newly created process image. These are
925 located above the argument list and environment. This normally adds 1
926 dirty page to the process size.</p>
928 <p>The ELF dynamic linker has its own data segment, adding one or more
929 dirty pages. I believe it also performs relocations on itself.</p>
931 <p>The ELF dynamic linker makes significant dynamic allocations to manage
932 the global symbol table and the loaded dso's. This data is never
933 freed. It will be needed again if libdl is used, so unconditionally
934 freeing it is not possible, but normal programs do not use libdl. Of
935 course with glibc all programs use libdl (due to nsswitch) so the
936 issue was never addressed.</p>
938 <p>ELF also has the issue that segments are not page-aligned on disk.
939 This saves up to 4k on disk, but at the expense of using an additional
940 dirty page in most cases, due to a large portion of the first data
941 page being filled with a duplicate copy of the last text page.</p>
943 <p>The above is just a partial list of the tiny memory penalties of ELF
944 dynamic linking, which eventually add up to quite a bit. The smallest
945 I've been able to get a process down to is 8 dirty pages, and the
946 above factors seem to mostly account for it (but some were difficult
950 <h2><a name="tips_kernel_headers"></a>Including kernel headers</h2>
952 <p>The "linux" or "asm" directories of /usr/include contain Linux kernel
953 headers, so that the C library can talk directly to the Linux kernel. In
954 a perfect world, applications shouldn't include these headers directly, but
955 we don't live in a perfect world.</p>
957 <p>For example, Busybox's losetup code wants linux/loop.c because nothing else
958 #defines the structures to call the kernel's loopback device setup ioctls.
959 Attempts to cut and paste the information into a local busybox header file
960 proved incredibly painful, because portions of the loop_info structure vary by
961 architecture, namely the type __kernel_dev_t has different sizes on alpha,
962 arm, x86, and so on. Meaning we either #include <linux/posix_types.h> or
963 we hardwire #ifdefs to check what platform we're building on and define this
964 type appropriately for every single hardware architecture supported by
965 Linux, which is simply unworkable.</p>
967 <p>This is aside from the fact that the relevant type defined in
968 posix_types.h was renamed to __kernel_old_dev_t during the 2.5 series, so
969 to cut and paste the structure into our header we have to #include
970 <linux/version.h> to figure out which name to use. (What we actually do is
971 check if we're building on 2.6, and if so just use the new 64 bit structure
972 instead to avoid the rename entirely.) But we still need the version
973 check, since 2.4 didn't have the 64 bit structure.</p>
975 <p>The BusyBox developers spent <u>two years</u> trying to figure
976 out a clean way to do all this. There isn't one. The losetup in the
977 util-linux package from kernel.org isn't doing it cleanly either, they just
978 hide the ugliness by nesting #include files. Their mount/loop.h
979 #includes "my_dev_t.h", which #includes <linux/posix_types.h> and
980 <linux/version.h> just like we do. There simply is no alternative.</p>
982 <p>Just because directly #including kernel headers is sometimes
983 unavoidable doesn't me we should include them when there's a better
984 way to do it. However, block copying information out of the kernel headers
985 is not a better way.</p>
987 <h2><a name="who">Who are the BusyBox developers?</a></h2>
989 <p>The following login accounts currently exist on busybox.net. (I.E. these
990 people can commit <a href="http://busybox.net/downloads/patches">patches</a>
991 into subversion for the BusyBox, uClibc, and buildroot projects.)</p>
994 aldot :Bernhard Fischer
995 andersen :Erik Andersen <- uClibc and BuildRoot maintainer.
997 davidm :David McCullough
998 gkajmowi :Garrett Kajmowicz <- uClibc++ maintainer
999 jbglaw :Jan-Benedict Glaw
1000 jocke :Joakim Tjernlund
1001 landley :Rob Landley <- BusyBox maintainer
1003 mjn3 :Manuel Novoa III
1006 pkj :Peter Kjellerstedt
1007 prpplague :David Anders
1008 psm :Peter S. Mazinger
1010 sandman :Robert Griebl
1011 sjhill :Steven J. Hill
1014 tobiasa :Tobias Anderberg
1015 vapier :Mike Frysinger
1018 <p>The following accounts used to exist on busybox.net, but don't anymore so
1019 I can't ask /etc/passwd for their names. (If anybody would like to make
1020 a stab at it...)</p>
1026 erik : Also Erik Andersen?
1035 vodz :Vladimir N. Oleynik
1043 <!--#include file="footer.html" -->