--- /dev/null
+Design document for the unified scheme data
+===========================================
+
+How are things connected?
+-------------------------
+
+The unified scheme takes all its data from the build.info files seen
+throughout the source tree. These files hold the minimum information
+needed to build end product files from diverse sources. See the
+section on build.info files below.
+
+From the information in build.info files, Configure builds up an
+information database as a hash table called %unified_info, which is
+stored in configdata.pm, found at the top of the build tree (which may
+or may not be the same as the source tree).
+
+Configurations/common.tmpl uses the data from %unified_info to
+generate the rules for building end product files as well as
+intermediary files with the help of a few functions found in the
+build-file templates. See the section on build-file templates further
+down for more information.
+
+build.info files
+----------------
+
+As mentioned earlier, build.info files are meant to hold the minimum
+information needed to build output files, and therefore only (with a
+few possible exceptions [1]) have information about end products (such
+as scripts, library files and programs) and source files (such as C
+files, C header files, assembler files, etc). Intermediate files such
+as object files are rarely directly refered to in build.info files (and
+when they are, it's always with the file name extension .o), they are
+infered by Configure. By the same rule of minimalism, end product
+file name extensions (such as .so, .a, .exe, etc) are never mentioned
+in build.info. Their file name extensions will be infered by the
+build-file templates, adapted for the platform they are meant for (see
+sections on %unified_info and build-file templates further down).
+
+The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare
+end products.
+
+The variables SOURCE, DEPEND, INCLUDE and ORDINALS are indexed by a
+produced file, and their values are the source used to produce that
+particular produced file, extra dependencies, include directories
+needed, and ordinal files (explained further below.
+
+All their values in all the build.info throughout the source tree are
+collected together and form a set of programs, libraries, engines and
+scripts to be produced, source files, dependencies, etc etc etc.
+
+Let's have a pretend example, a very limited contraption of OpenSSL,
+composed of the program 'apps/openssl', the libraries 'libssl' and
+'libcrypto', an engine 'engines/ossltest' and their sources and
+dependencies.
+
+ # build.info
+ LIBS=libcrypto libssl
+ ORDINALS[libcrypto]=crypto
+ ORDINALS[libssl]=ssl
+ INCLUDE[libcrypto]=include
+ INCLUDE[libssl]=include
+ DEPEND[libssl]=libcrypto
+
+This is the top directory build.info file, and it tells us that two
+libraries are to be built, there are some ordinals to be used to
+declare what symbols in those libraries are seen as public, the
+include directory 'include/' shall be used throughout when building
+anything that will end up in each library, and that the library
+'libssl' depend on the library 'libcrypto' to function properly.
+
+ # apps/build.info
+ PROGRAMS=openssl
+ SOURCE[openssl]=openssl.c
+ INCLUDE[openssl]=.. ../include
+ DEPEND[openssl]=../libssl
+
+This is the build.info file in 'apps/', one may notice that all file
+paths mentioned are relative to the directory the build.info file is
+located in. This one tells us that there's a program to be built
+called 'apps/openssl' (the file name extension will depend on the
+platform and is therefore not mentioned in the build.info file). It's
+built from one source file, 'apps/openssl.c', and building it requires
+the use of '.' and 'include' include directories (both are declared
+from the point of view of the 'apps/' directory), and that the program
+depends on the library 'libssl' to function properly.
+
+ # crypto/build.info
+ LIBS=../libcrypto
+ SOURCE[../libcrypto]=aes.c evp.c cversion.c
+ DEPEND[cversion.o]=buildinf.h
+
+ BEGINRAW[Makefile(unix)]
+ crypto/buildinf.h : Makefile
+ perl util/mkbuildinf.h "$(CC) $(CFLAGS)" "$(PLATFORM)" \
+ > crypto/buildinf.h
+ ENDRAW[Makefile(unix)]
+
+This is the build.info file in 'crypto', and it tells us a little more
+about what's needed to produce 'libcrypto'. LIBS is used again to
+declare that 'libcrypto' is to be produced. This declaration is
+really unnecessary as it's already mentioned in the top build.info
+file, but can make the info file easier to understand. This is to
+show that duplicate information isn't an issue.
+
+This build.info file informs us that 'libcrypto' is built from a few
+source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'.
+It also shows us that building the object file inferred from
+'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it
+also shows the possibility to include raw build-file statements in a
+build.info file, in this case showing how 'buildinf.h' is built on
+Unix-like operating systems.
+
+Two things are worth an extra note:
+
+'DEPEND[cversion.o]' mentiones an object file. DEPEND indexes is the
+only location where it's valid to mention them
+
+Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as
+seen from the top directory, no exception.
+
+ # ssl/build.info
+ LIBS=../libssl
+ SOURCE[../libssl]=tls.c
+
+This is the build.info file in 'ssl/', and it tells us that the
+library 'libssl' is built from the source file 'ssl/tls.c'.
+
+ # engines/build.info
+ ENGINES=libossltest
+ SOURCE[libossltest]=e_ossltest.c
+ DEPEND[libossltest]=../libcrypto
+ INCLUDE[libossltest]=../include
+
+This is the build.info file in 'engines/', telling us that an engine
+called 'engines/libossltest' shall be built, that it's source is
+'engines/e_ossltest.c' and that the include directory 'include/' may
+be used when building anything that will be part of this engine.
+Finally, the engine 'engines/libossltest' depends on the library
+'libcrypto' to function properly.
+
+When Configure digests these build.info files, the accumulated
+information comes down to this:
+
+ LIBS=libcrypto libssl
+ ORDINALS[libcrypto]=crypto
+ SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
+ DEPEND[crypto/cversion.o]=crypto/buildinf.h
+ INCLUDE[libcrypto]=include
+ ORDINALS[libssl]=ssl
+ SOURCE[libssl]=ssl/tls.c
+ INCLUDE[libssl]=include
+ DEPEND[libssl]=libcrypto
+
+ PROGRAMS=apps/openssl
+ SOURCE[apps/openssl]=apps/openssl.c
+ INCLUDE[apps/openssl]=. include
+ DEPEND[apps/openssl]=libssl
+
+ ENGINES=engines/libossltest
+ SOURCE[engines/libossltest]=engines/e_ossltest.c
+ DEPEND[engines/libossltest]=libcrypto
+ INCLUDE[engines/libossltest]=include
+
+ BEGINRAW[Makefile(unix)]
+ crypto/buildinf.h : Makefile
+ perl util/mkbuildinf.h "$(CC) $(CFLAGS)" "$(PLATFORM)" \
+ > crypto/buildinf.h
+ ENDRAW[Makefile(unix)]
+
+
+A few notes worth mentioning:
+
+LIBS may be used to declare routine libraries only.
+
+PROGRAMS may be used to declare programs only.
+
+ENGINES may be used to declare engines only.
+
+The indexes for SOURCE, INCLUDE and ORDINALS must only be end product
+files, such as libraries, programs or engines. The values of SOURCE
+variables must only be source files (possibly generated)
+
+DEPEND shows a relationship between different end product files, such
+as a program depending on a library, or between an object file and
+some extra source file.
+
+When Configure processes the build.info files, it will take it as
+truth without question, and will therefore perform very few checks.
+If the build tree is separate from the source tree, it will assume
+that all built files and up in the build directory and that all source
+files are to be found in the source tree, if they can be found there.
+Configure will assume that source files that can't be found in the
+source tree (such as 'crypto/bildinf.h' in the example above) are
+generated and will be found in the build tree.
+
+
+The %unified_info database
+--------------------------
+
+The information in all the build.info get digested by Configure and
+collected into the %unified_info database, divided into the following
+indexes:
+
+ depends => a hash table containing 'file' => [ 'dependency' ... ]
+ pairs. These are directly inferred from the DEPEND
+ variables in build.info files.
+
+ engines => a list of engines. These are directly inferred from
+ the ENGINES variable in build.info files.
+
+ includes => a hash table containing 'file' => [ 'include' ... ]
+ pairs. These are directly inferred from the INCLUDE
+ variables in build.info files.
+
+ libraries => a list of libraries. These are directly inferred from
+ the LIBS variable in build.info files.
+
+ ordinals => a hash table containing 'file' => [ 'word', 'ordfile' ]
+ pairs. 'file' and 'word' are directly inferred from
+ the ORDINALS variables in build.info files, while the
+ file 'ofile' comes from internal knowledge in
+ Configure.
+
+ programs => a list of programs. These are directly inferred from
+ the PROGRAMS variable in build.info files.
+
+ rawlines => a list of build-file lines. These are a direct copy of
+ the BEGINRAW..ENDRAW lines in build.info files. Note:
+ only the BEGINRAW..ENDRAW section for the current
+ platform are copied, the rest are ignored.
+
+ scripts => a list of scripts. There are directly inferred from
+ the SCRIPTS variable in build.info files.
+
+ sources => a hash table containing 'file' => [ 'sourcefile' ... ]
+ pairs. These are indirectly inferred from the SOURCE
+ variables in build.info files. Object files are
+ mentioned in this hash table, with source files from
+ SOURCE variables, and AS source files for programs and
+ libraries.
+
+As an example, here is how the build.info files example from the
+section above would be digested into a %unified_info table:
+
+ our %unified_info = (
+ "depends" =>
+ {
+ "apps/openssl" =>
+ [
+ "libssl",
+ ],
+ "crypto/cversion.o" =>
+ [
+ "crypto/buildinf.h",
+ ],
+ "engines/libossltest" =>
+ [
+ "libcrypto",
+ ],
+ "libssl" =>
+ [
+ "libcrypto",
+ ],
+ },
+ "engines" =>
+ [
+ "engines/libossltest",
+ ],
+ "includes" =>
+ {
+ "apps/openssl" =>
+ [
+ ".",
+ "include",
+ ],
+ "engines/libossltest" =>
+ [
+ "include"
+ ],
+ "libcrypto" =>
+ [
+ "include",
+ ],
+ "libssl" =>
+ [
+ "include",
+ ],
+ }
+ "libraries" =>
+ [
+ "libcrypto",
+ "libssl",
+ ],
+ "ordinals" =>
+ {
+ "libcrypto" =>
+ [
+ "crypto",
+ "util/libeay.num",
+ ],
+ "libssl" =>
+ [
+ "ssl",
+ "util/ssleay.num",
+ ],
+ },
+ "programs" =>
+ [
+ "apps/openssl",
+ ],
+ "rawlines" =>
+ [
+ "crypto/buildinf.h : Makefile",
+ " perl util/mkbuildinf.h \"\$(CC) \$(CFLAGS)\" \"\$(PLATFORM)\" \\"
+ " > crypto/buildinf.h"
+ ],
+ "sources" =>
+ {
+ "apps/openssl" =>
+ [
+ "apps/openssl.o",
+ ],
+ "apps/openssl.o" =>
+ [
+ "apps/openssl.c",
+ ],
+ "crypto/aes.o" =>
+ [
+ "crypto/aes.c",
+ ],
+ "crypto/cversion.o" =>
+ [
+ "crypto/cversion.c",
+ ],
+ "crypto/evp.o" =>
+ [
+ "crypto/evp.c",
+ ],
+ "engines/e_ossltest.o" =>
+ [
+ "engines/e_ossltest.c",
+ ],
+ "engines/libossltest" =>
+ [
+ "engines/e_ossltest.o",
+ ],
+ "libcrypto" =>
+ [
+ "crypto/aes.c",
+ "crypto/cversion.c",
+ "crypto/evp.c",
+ ],
+ "libssl" =>
+ [
+ "ssl/tls.c",
+ ],
+ "ssl/tls.o" =>
+ [
+ "ssl/tls.c",
+ ],
+ },
+ );
+
+As can be seen, everything in %unified_info is fairly simple nuggest
+of information. Still, it tells us that to build all programs, we
+must build 'apps/openssl', and to build the latter, we will need to
+build all its sources ('apps/openssl.o' in this case) and all the
+other things it depends on (such as 'libssl'). All those dependencies
+need to be built as well, using the same logic, so to build 'libssl',
+we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the
+latter...
+
+
+Build-file templates
+--------------------
+
+Build-file templates are essentially build-files (such as Makefile on
+Unix) with perl code fragments mixed in. Those perl code fragment
+will generate all the configuration dependent data, including all the
+rules needed to build end product files and intermediary files alike.
+At a minimum, there must be a perl code fragment that defines a set of
+functions that are used to generates specific build-file rules, to
+build static libraries from object files, to build shared libraries
+from static libraries, to programs from object files and libraries,
+etc.
+
+ src2dep - function that produces build file lines to get the
+ dependencies for an object file into a dependency
+ file.
+
+ It's called like this:
+
+ src2dep(obj => "PATH/TO/objectfile",
+ srcs => [ "PATH/TO/sourcefile", ... ],
+ incs => [ "INCL/PATH", ... ]);
+
+ 'obj' has the dependent object file as well as
+ object file the dependencies are for; it's *without*
+ extension, src2dep() is expected to add that.
+ 'srcs' has the list of source files to build the
+ object file, with the first item being the source
+ file that directly corresponds to the object file.
+ 'incs' is a list of include file directories.
+
+ src2obj - function that produces build file lines to build an
+ object file from source files and associated data.
+
+ It's called like this:
+
+ src2obj(obj => "PATH/TO/objectfile",
+ srcs => [ "PATH/TO/sourcefile", ... ],
+ deps => [ "dep1", ... ],
+ incs => [ "INCL/PATH", ... ]);
+
+ 'obj' has the intended object file *without*
+ extension, src2obj() is expected to add that.
+ 'srcs' has the list of source files to build the
+ object file, with the first item being the source
+ file that directly corresponds to the object file.
+ 'deps' is a list of dependencies. 'incs' is a list
+ of include file directories.
+
+ obj2lib - function that produces build file lines to build a
+ static library file ("libfoo.a" in Unix terms) from
+ object files.
+
+ called like this:
+
+ obj2lib(lib => "PATH/TO/libfile",
+ objs => [ "PATH/TO/objectfile", ... ]);
+
+ 'lib' has the intended library file name *without*
+ extension, obj2lib is expected to add that. 'objs'
+ has the list of object files (also *without*
+ extension) to build this library.
+
+ libobj2shlib - function that produces build file lines to build a
+ shareable object library file ("libfoo.so" in Unix
+ terms) from the corresponding static library file
+ or object files.
+
+ called like this:
+
+ libobj2shlib(shlib => "PATH/TO/shlibfile",
+ lib => "PATH/TO/libfile",
+ objs => [ "PATH/TO/objectfile", ... ],
+ deps => [ "PATH/TO/otherlibfile", ... ],
+ ordinals => [ "word", "/PATH/TO/ordfile" ]);
+
+ 'lib' has the intended library file name *without*
+ extension, libobj2shlib is expected to add that.
+ 'shlib' has the correcponding shared library name
+ *without* extension. 'deps' has the list of other
+ libraries (also *without* extension) this library
+ needs to be linked with. 'objs' has the list of
+ object files (also *without* extension) to build
+ this library. 'ordinals' MAY be present, and when
+ it is, its value is an array where the word is
+ "crypto" or "ssl" and the file is one of the ordinal
+ files util/libeay.num or util/ssleay.num in the
+ source directory.
+
+ This function has a choice; it can use the
+ corresponding static library as input to make the
+ shared library, or the list of object files.
+
+ obj2dynlib - function that produces build file lines to build a
+ dynamically loadable library file ("libfoo.so" on
+ Unix) from object files.
+
+ called like this:
+
+ obj2dynlib(lib => "PATH/TO/libfile",
+ objs => [ "PATH/TO/objectfile", ... ],
+ deps => [ "PATH/TO/otherlibfile",
+ ... ]);
+
+ This is almost the same as libobj2shlib, but the
+ intent is to build a shareable library that can be
+ loaded in runtime (a "plugin"...). The differences
+ are subtle, one of the most visible ones is that the
+ resulting shareable library is produced from object
+ files only.
+
+ obj2bin - function that produces build file lines to build an
+ executable file from object files.
+
+ called like this:
+
+ obj2bin(bin => "PATH/TO/binfile",
+ objs => [ "PATH/TO/objectfile", ... ],
+ deps => [ "PATH/TO/libfile", ... ]);
+
+ 'bin' has the intended executable file name
+ *without* extension, obj2bin is expected to add
+ that. 'objs' has the list of object files (also
+ *without* extension) to build this library. 'deps'
+ has the list of library files (also *without*
+ extension) that the programs needs to be linked
+ with.
+
+ in2script - function that produces build file lines to build a
+ script file from some input.
+
+ called like this:
+
+ in2script(script => "PATH/TO/scriptfile",
+ sources => [ "PATH/TO/infile", ... ]);
+
+ 'script' has the intended script file name.
+ 'sources' has the list of source files to build the
+ resulting script from.
+
+Along with the build-file templates is the driving engine
+Configurations/common.tmpl, which looks through all the information in
+%unified_info and generates all the rulesets to build libraries,
+programs and all intermediate files, using the rule generating
+functions defined in the build-file template.
+
+As an example with the smaller build.info set we've seen as an
+example, producing the rules to build 'libssl' would result in the
+following calls:
+
+ # Note: libobj2shlib will only be called if shared libraries are
+ # to be produced.
+ # Note 2: libobj2shlib gets both the name of the static library
+ # and the names of all the object files that go into it. It's up
+ # to the implementation to decide which to use as input.
+ libobj2shlib(shlib => "libssl",
+ lib => "libssl",
+ objs => [ "ssl/tls.o" ],
+ deps => [ "libcrypto" ]
+ ordinals => [ "ssl", "util/ssleay.num" ]);
+
+ obj2lib(lib => "libssl"
+ objs => [ "ssl/tls.o" ]);
+
+ # Note 3: common.tmpl peals off the ".o" extension, as the
+ # platform at hand may have a different one.
+ src2obj(obj => "ssl/tls"
+ srcs => [ "ssl/tls.c" ],
+ deps => [ ],
+ incs => [ "include" ]);
+
+ src2dep(obj => "ssl/tls"
+ srcs => [ "ssl/tls.c" ],
+ incs => [ "include" ]);
+
+The returned strings from all those calls are then concatenated
+together and written to the resulting build-file.
projects / oweals / openssl.git / commitdiff