From: Christian Grothoff Date: Wed, 28 Oct 2009 19:52:25 +0000 (+0000) Subject: getting testing to work X-Git-Tag: initial-import-from-subversion-38251~23215 X-Git-Url: https://git.librecmc.org/?a=commitdiff_plain;h=cbf2a3e7cf2d7acda1d115198671eaccf2616fc4;p=oweals%2Fgnunet.git getting testing to work --- diff --git a/INSTALL b/INSTALL index d3c5b40a9..2550dab75 100644 --- a/INSTALL +++ b/INSTALL @@ -2,15 +2,15 @@ Installation Instructions ************************* Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005, -2006, 2007 Free Software Foundation, Inc. +2006, 2007, 2008, 2009 Free Software Foundation, Inc. -This file is free documentation; the Free Software Foundation gives + This file is free documentation; the Free Software Foundation gives unlimited permission to copy, distribute and modify it. Basic Installation ================== -Briefly, the shell commands `./configure; make; make install' should + Briefly, the shell commands `./configure; make; make install' should configure, build, and install this package. The following more-detailed instructions are generic; see the `README' file for instructions specific to this package. @@ -73,9 +73,9 @@ The simplest way to compile this package is: Compilers and Options ===================== -Some systems require unusual options for compilation or linking that the -`configure' script does not know about. Run `./configure --help' for -details on some of the pertinent environment variables. + Some systems require unusual options for compilation or linking that +the `configure' script does not know about. Run `./configure --help' +for details on some of the pertinent environment variables. You can give `configure' initial values for configuration parameters by setting variables in the command line or in the environment. Here @@ -88,7 +88,7 @@ is an example: Compiling For Multiple Architectures ==================================== -You can compile the package for more than one kind of computer at the + You can compile the package for more than one kind of computer at the same time, by placing the object files for each architecture in their own directory. To do this, you can use GNU `make'. `cd' to the directory where you want the object files and executables to go and run @@ -100,10 +100,24 @@ architecture at a time in the source code directory. After you have installed the package for one architecture, use `make distclean' before reconfiguring for another architecture. + On MacOS X 10.5 and later systems, you can create libraries and +executables that work on multiple system types--known as "fat" or +"universal" binaries--by specifying multiple `-arch' options to the +compiler but only a single `-arch' option to the preprocessor. Like +this: + + ./configure CC="gcc -arch i386 -arch x86_64 -arch ppc -arch ppc64" \ + CXX="g++ -arch i386 -arch x86_64 -arch ppc -arch ppc64" \ + CPP="gcc -E" CXXCPP="g++ -E" + + This is not guaranteed to produce working output in all cases, you +may have to build one architecture at a time and combine the results +using the `lipo' tool if you have problems. + Installation Names ================== -By default, `make install' installs the package's commands under + By default, `make install' installs the package's commands under `/usr/local/bin', include files under `/usr/local/include', etc. You can specify an installation prefix other than `/usr/local' by giving `configure' the option `--prefix=PREFIX'. @@ -126,7 +140,7 @@ option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'. Optional Features ================= -Some packages pay attention to `--enable-FEATURE' options to + Some packages pay attention to `--enable-FEATURE' options to `configure', where FEATURE indicates an optional part of the package. They may also pay attention to `--with-PACKAGE' options, where PACKAGE is something like `gnu-as' or `x' (for the X Window System). The @@ -138,14 +152,46 @@ find the X include and library files automatically, but if it doesn't, you can use the `configure' options `--x-includes=DIR' and `--x-libraries=DIR' to specify their locations. +Particular systems +================== + + On HP-UX, the default C compiler is not ANSI C compatible. If GNU +CC is not installed, it is recommended to use the following options in +order to use an ANSI C compiler: + + ./configure CC="cc -Ae -D_XOPEN_SOURCE=500" + +and if that doesn't work, install pre-built binaries of GCC for HP-UX. + + On OSF/1 a.k.a. Tru64, some versions of the default C compiler cannot +parse its `' header file. The option `-nodtk' can be used as +a workaround. If GNU CC is not installed, it is therefore recommended +to try + + ./configure CC="cc" + +and if that doesn't work, try + + ./configure CC="cc -nodtk" + + On Solaris, don't put `/usr/ucb' early in your `PATH'. This +directory contains several dysfunctional programs; working variants of +these programs are available in `/usr/bin'. So, if you need `/usr/ucb' +in your `PATH', put it _after_ `/usr/bin'. + + On Haiku, software installed for all users goes in `/boot/common', +not `/usr/local'. It is recommended to use the following options: + + ./configure --prefix=/boot/common + Specifying the System Type ========================== -There may be some features `configure' cannot figure out automatically, -but needs to determine by the type of machine the package will run on. -Usually, assuming the package is built to be run on the _same_ -architectures, `configure' can figure that out, but if it prints a -message saying it cannot guess the machine type, give it the + There may be some features `configure' cannot figure out +automatically, but needs to determine by the type of machine the package +will run on. Usually, assuming the package is built to be run on the +_same_ architectures, `configure' can figure that out, but if it prints +a message saying it cannot guess the machine type, give it the `--build=TYPE' option. TYPE can either be a short name for the system type, such as `sun4', or a canonical name which has the form: @@ -153,7 +199,8 @@ type, such as `sun4', or a canonical name which has the form: where SYSTEM can have one of these forms: - OS KERNEL-OS + OS + KERNEL-OS See the file `config.sub' for the possible values of each field. If `config.sub' isn't included in this package, then this package doesn't @@ -171,9 +218,9 @@ eventually be run) with `--host=TYPE'. Sharing Defaults ================ -If you want to set default values for `configure' scripts to share, you -can create a site shell script called `config.site' that gives default -values for variables like `CC', `cache_file', and `prefix'. + If you want to set default values for `configure' scripts to share, +you can create a site shell script called `config.site' that gives +default values for variables like `CC', `cache_file', and `prefix'. `configure' looks for `PREFIX/share/config.site' if it exists, then `PREFIX/etc/config.site' if it exists. Or, you can set the `CONFIG_SITE' environment variable to the location of the site script. @@ -182,7 +229,7 @@ A warning: not all `configure' scripts look for a site script. Defining Variables ================== -Variables not defined in a site shell script can be set in the + Variables not defined in a site shell script can be set in the environment passed to `configure'. However, some packages may run configure again during the build, and the customized values of these variables may be lost. In order to avoid this problem, you should set @@ -201,11 +248,19 @@ an Autoconf bug. Until the bug is fixed you can use this workaround: `configure' Invocation ====================== -`configure' recognizes the following options to control how it operates. + `configure' recognizes the following options to control how it +operates. `--help' `-h' - Print a summary of the options to `configure', and exit. + Print a summary of all of the options to `configure', and exit. + +`--help=short' +`--help=recursive' + Print a summary of the options unique to this package's + `configure', and exit. The `short' variant lists options used + only in the top level, while the `recursive' variant lists options + also present in any nested packages. `--version' `-V' @@ -232,6 +287,16 @@ an Autoconf bug. Until the bug is fixed you can use this workaround: Look for the package's source code in directory DIR. Usually `configure' can determine that directory automatically. +`--prefix=DIR' + Use DIR as the installation prefix. *Note Installation Names:: + for more details, including other options available for fine-tuning + the installation locations. + +`--no-create' +`-n' + Run the configure checks, but stop before creating any output + files. + `configure' also accepts some other, not widely useful, options. Run `configure --help' for more details. diff --git a/TODO b/TODO index e06e63790..a67b83bb8 100644 --- a/TODO +++ b/TODO @@ -1,25 +1,22 @@ Implementable right now (but not necessarily important), with caveats (unavailable components that will limit what can be implemented right away), in order in which they will likely be done: -* TESTING -* FS (DHT not available) +* TESTING [Nate] * SETUP -* DV (distributed testing not available) -* TBENCH (distributed testing not available) +* DV (distributed testing not available) [Nate] +* TBENCH (distributed testing not available) * TRACEKIT (distributed testing not available) +* FRAGMENTATION [Ji Lu] * HTTP transport -* FRAGMENTATION -* MySQL / Postgres plugins (datastore, datacache) -* UPNP +* MySQL / Postgres plugins (datastore, datacache) [Radhika] +* UPNP [Milan] Urgent items (before announcing ng.gnunet.org): * TESTING (needed for DV, DHT, Topology) - implement library for local testing - + modify configuration to allow controlling connections for non-local starts - + CORE service does not start with valid peer ID (all zeros) -- testcase fails! - + consider changing API for peer-group termination to call continuation when done + + consider changing API for peer-group termination to + call continuation when done - implement testcases for library - + get test for basic peer start to work! + test basic peer connect + test group start * TEST: @@ -89,6 +86,7 @@ Urgent items (before announcing ng.gnunet.org): + default generation + need to settle basic design; do we want to keep guile? - testing (RC-pre0) + + modify configuration to allow controlling connections for non-local starts + testbed creation with topology (needs working F2F topology) [Nate] + testbed with churn [Nate] + implement library for distributed testing [Nate] @@ -146,7 +144,7 @@ Urgent items (before announcing ng.gnunet.org): - create good Drupal theme for GNUnet - make a NICE download page and figure out how to enable developers to publish TGZs nicely - - port "contact" page; add impressum + - port "contact" page - add content type for "todo" items? - DNS activation * Plugins to implement: diff --git a/src/Makefile.am b/src/Makefile.am index 50d75916b..c30045c9b 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -20,6 +20,7 @@ SUBDIRS = \ template \ transport \ core \ + testing \ $(HOSTLIST_DIR) \ topology \ fs \ diff --git a/src/arm/arm_api.c b/src/arm/arm_api.c index 00ddddaab..810f2ebe6 100644 --- a/src/arm/arm_api.c +++ b/src/arm/arm_api.c @@ -390,6 +390,7 @@ GNUNET_ARM_stop_service (struct GNUNET_ARM_Handle *h, if (0 == strcmp ("arm", service_name)) { GNUNET_CLIENT_service_shutdown (h->client); + h->client = NULL; if (cb != NULL) cb (cb_cls, GNUNET_NO); return; diff --git a/src/arm/gnunet-service-arm.c b/src/arm/gnunet-service-arm.c index 9aeb620f4..63808d9c9 100644 --- a/src/arm/gnunet-service-arm.c +++ b/src/arm/gnunet-service-arm.c @@ -536,7 +536,6 @@ stop_service (struct GNUNET_SERVER_Client *client, (NULL != (sc = GNUNET_CLIENT_connect (sched, servicename, cfg))) ) { GNUNET_CLIENT_service_shutdown (sc); - GNUNET_CLIENT_disconnect (sc); signal_result (client, servicename, GNUNET_MESSAGE_TYPE_ARM_IS_DOWN); } else diff --git a/src/fs/test_fs_namespace_data.conf b/src/fs/test_fs_namespace_data.conf new file mode 100644 index 000000000..d7c2af4c4 --- /dev/null +++ b/src/fs/test_fs_namespace_data.conf @@ -0,0 +1,42 @@ +[PATHS] +SERVICEHOME = /tmp/gnunet-test-fs-namespace/ +DEFAULTCONFIG = test_fs_namespace_data.conf + +[gnunetd] +HOSTKEY = $SERVICEHOME/.hostkey + +[resolver] +PORT = 42464 +HOSTNAME = localhost + +[transport] +PORT = 42465 +PLUGINS = + +[arm] +PORT = 42466 +HOSTNAME = localhost +DEFAULTSERVICES = resolver datastore transport core fs + +[datastore] +#DEBUG = YES + +[statistics] +PORT = 42467 +HOSTNAME = localhost + +[peerinfo] +PORT = 42469 +HOSTNAME = localhost + +[core] +PORT = 42470 +HOSTNAME = localhost + +[fs] +PORT = 42471 +HOSTNAME = localhost +#DEBUG = YES + +[testing] +WEAKRANDOM = YES diff --git a/src/include/gnunet_client_lib.h b/src/include/gnunet_client_lib.h index 2ba9e135f..3773a55d1 100644 --- a/src/include/gnunet_client_lib.h +++ b/src/include/gnunet_client_lib.h @@ -179,7 +179,10 @@ GNUNET_CLIENT_transmit_and_get_response (struct GNUNET_CLIENT_Connection *sock, /** * Request that the service should shutdown. - * Afterwards, the connection should be disconnected. + * Afterwards, the connection will automatically be + * disconnected. Hence the "sock" shoud not + * be used by the caller after this call + * (calling this function frees "sock" after a while). * * @param sock the socket connected to the service */ diff --git a/src/testing/test_testing.c b/src/testing/test_testing.c index 886427402..f56ebe2c0 100644 --- a/src/testing/test_testing.c +++ b/src/testing/test_testing.c @@ -24,7 +24,7 @@ #include "platform.h" #include "gnunet_testing_lib.h" -#define VERBOSE GNUNET_YES +#define VERBOSE GNUNET_NO static int ok; @@ -48,7 +48,8 @@ static void my_cb(void *cls, GNUNET_assert (id != NULL); #if VERBOSE GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, - "Daemon started, will now stop it.\n"); + "Daemon `%s' started, will now stop it.\n", + GNUNET_i2s (id)); #endif GNUNET_TESTING_daemon_stop (d, &end_cb, NULL); } diff --git a/src/testing/test_testing_data.conf b/src/testing/test_testing_data.conf index f326e72bd..f314028d4 100644 --- a/src/testing/test_testing_data.conf +++ b/src/testing/test_testing_data.conf @@ -8,6 +8,8 @@ PORT = 2564 [transport] PORT = 2565 PLUGINS = tcp +#PREFIX = xterm -e xterm -T transport -e gdb -x cmd --args +#PREFIX = valgrind --tool=memcheck --log-file=logs%p [arm] PORT = 2566 diff --git a/src/testing/testing.c b/src/testing/testing.c index e4cc74827..35a872619 100644 --- a/src/testing/testing.c +++ b/src/testing/testing.c @@ -40,13 +40,13 @@ #include "gnunet_testing_lib.h" #include "gnunet_transport_service.h" -#define DEBUG_TESTING GNUNET_YES +#define DEBUG_TESTING GNUNET_NO /** * How long do we wait after starting gnunet-service-arm * for the core service to be alive? */ -#define ARM_START_WAIT GNUNET_TIME_relative_multiply (GNUNET_TIME_UNIT_SECONDS, 2) +#define ARM_START_WAIT GNUNET_TIME_relative_multiply (GNUNET_TIME_UNIT_SECONDS, 10) /** * How many times are we willing to try to wait for "scp" or @@ -632,8 +632,13 @@ GNUNET_TESTING_daemon_start (struct GNUNET_SCHEDULER_Handle *sched, ret->username = username; /* 2) copy file to remote host */ - if (NULL != hostname) + if (NULL != hostname) { +#if DEBUG_TESTING + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Copying configuration file to host `%s'.\n", + hostname); +#endif ret->phase = SP_COPYING; if (NULL != username) GNUNET_asprintf (&arg, @@ -732,7 +737,6 @@ void GNUNET_TESTING_daemon_stop (struct GNUNET_TESTING_Daemon *d, "arm", d->cfg); GNUNET_CLIENT_service_shutdown (cc); - GNUNET_CLIENT_disconnect (cc); /* state clean up and notifications */ if (0 != UNLINK (d->cfgfile)) @@ -741,6 +745,11 @@ void GNUNET_TESTING_daemon_stop (struct GNUNET_TESTING_Daemon *d, d->cfgfile); if (d->hostname != NULL) { +#if DEBUG_TESTING + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Removing configuration file on remote host `%s'.\n", + d->hostname); +#endif if (NULL != d->username) GNUNET_asprintf (&dst, "%s@%s", @@ -832,6 +841,11 @@ void GNUNET_TESTING_daemon_reconfigure (struct GNUNET_TESTING_Daemon *d, cb (cb_cls, NULL); return; } +#if DEBUG_TESTING + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Copying updated configuration file to remote host `%s'.\n", + d->hostname); +#endif d->phase = SP_CONFIG_UPDATE; if (NULL != d->username) GNUNET_asprintf (&arg, @@ -873,28 +887,30 @@ void GNUNET_TESTING_daemon_reconfigure (struct GNUNET_TESTING_Daemon *d, d); } + /** - * FIXME. + * Data kept for each pair of peers that we try + * to connect. */ struct ConnectContext { /** - * FIXME. + * Testing handle to the first daemon. */ struct GNUNET_TESTING_Daemon *d1; /** - * FIXME. + * Testing handle to the second daemon. */ struct GNUNET_TESTING_Daemon *d2; /** - * FIXME. + * Transport handle to the first daemon. */ struct GNUNET_TRANSPORT_Handle *d1th; /** - * FIXME. + * Transport handle to the second daemon. */ struct GNUNET_TRANSPORT_Handle *d2th; @@ -919,7 +935,7 @@ struct ConnectContext /** * Success, connection is up. Signal client our success. * - * @param cls FIXME + * @param cls our "struct ConnectContext" * @param size number of bytes available in buf * @param buf where to copy the message, NULL on error * @return number of bytes copied to buf diff --git a/src/transport/gnunet-service-transport.c b/src/transport/gnunet-service-transport.c index 6803e3179..0dc3f859d 100644 --- a/src/transport/gnunet-service-transport.c +++ b/src/transport/gnunet-service-transport.c @@ -2587,6 +2587,42 @@ client_disconnect_notification (void *cls, } +/** + * Function called when the service shuts down. Unloads our plugins. + * + * @param cls closure, unused + * @param tc task context (unused) + */ +static void +unload_plugins (void *cls, + const struct GNUNET_SCHEDULER_TaskContext *tc) +{ + struct TransportPlugin *plug; + struct AddressList *al; + +#if DEBUG_TRANSPORT + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Transport service is unloading plugins...\n"); +#endif + while (NULL != (plug = plugins)) + { + plugins = plug->next; + GNUNET_break (NULL == GNUNET_PLUGIN_unload (plug->lib_name, plug->api)); + GNUNET_free (plug->lib_name); + GNUNET_free (plug->short_name); + while (NULL != (al = plug->addresses)) + { + plug->addresses = al->next; + GNUNET_free (al); + } + GNUNET_free (plug); + } + if (my_private_key != NULL) + GNUNET_CRYPTO_rsa_key_free (my_private_key); + GNUNET_free_non_null (our_hello); +} + + /** * Initiate transport service. * @@ -2661,6 +2697,12 @@ run (void *cls, } GNUNET_free (plugs); } + GNUNET_SCHEDULER_add_delayed (sched, + GNUNET_YES, + GNUNET_SCHEDULER_PRIORITY_IDLE, + GNUNET_SCHEDULER_NO_TASK, + GNUNET_TIME_UNIT_FOREVER_REL, + &unload_plugins, NULL); if (no_transports) refresh_hello (); #if DEBUG_TRANSPORT @@ -2672,43 +2714,6 @@ run (void *cls, } -/** - * Function called when the service shuts - * down. Unloads our plugins. - * - * @param cls closure - * @param cfg configuration to use - */ -static void -unload_plugins (void *cls, - const struct GNUNET_CONFIGURATION_Handle *cfg) -{ - struct TransportPlugin *plug; - struct AddressList *al; - -#if DEBUG_TRANSPORT - GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, - "Transport service is unloading plugins...\n"); -#endif - while (NULL != (plug = plugins)) - { - plugins = plug->next; - GNUNET_break (NULL == GNUNET_PLUGIN_unload (plug->lib_name, plug->api)); - GNUNET_free (plug->lib_name); - GNUNET_free (plug->short_name); - while (NULL != (al = plug->addresses)) - { - plug->addresses = al->next; - GNUNET_free (al); - } - GNUNET_free (plug); - } - if (my_private_key != NULL) - GNUNET_CRYPTO_rsa_key_free (my_private_key); - GNUNET_free_non_null (our_hello); -} - - /** * The main function for the transport service. * @@ -2723,7 +2728,7 @@ main (int argc, char *const *argv) GNUNET_SERVICE_run (argc, argv, "transport", - &run, NULL, &unload_plugins, NULL)) ? 0 : 1; + &run, NULL, NULL, NULL)) ? 0 : 1; } /* end of gnunet-service-transport.c */ diff --git a/src/transport/plugin_transport_tcp.c b/src/transport/plugin_transport_tcp.c index 65abf5fa4..eeb333a53 100644 --- a/src/transport/plugin_transport_tcp.c +++ b/src/transport/plugin_transport_tcp.c @@ -2016,7 +2016,10 @@ process_hostname_ips (void *cls, struct Plugin *plugin = cls; if (addr == NULL) - return; + { + hostname_dns = NULL; + return; + } process_interfaces (plugin, "", GNUNET_YES, diff --git a/src/util/client.c b/src/util/client.c index 8c429995c..3c6f0c405 100644 --- a/src/util/client.c +++ b/src/util/client.c @@ -468,13 +468,28 @@ GNUNET_CLIENT_receive (struct GNUNET_CLIENT_Connection *sock, } +/** + * If possible, write a shutdown message to the target + * buffer and destroy the client connection. + * + * @param cls the "struct GNUNET_CLIENT_Connection" to destroy + * @param size number of bytes available in buf + * @param buf NULL on error, otherwise target buffer + * @return number of bytes written to buf + */ static size_t write_shutdown (void *cls, size_t size, void *buf) { struct GNUNET_MessageHeader *msg; + struct GNUNET_CLIENT_Connection *sock = cls; + GNUNET_CLIENT_disconnect (sock); if (size < sizeof (struct GNUNET_MessageHeader)) - return 0; /* client disconnected */ + { + GNUNET_log (GNUNET_ERROR_TYPE_INFO, + _("Failed to transmit shutdown request to client.\n")); + return 0; /* client disconnected */ + } msg = (struct GNUNET_MessageHeader *) buf; msg->type = htons (GNUNET_MESSAGE_TYPE_SHUTDOWN); msg->size = htons (sizeof (struct GNUNET_MessageHeader)); @@ -492,9 +507,9 @@ void GNUNET_CLIENT_service_shutdown (struct GNUNET_CLIENT_Connection *sock) { GNUNET_CONNECTION_notify_transmit_ready (sock->sock, - sizeof (struct GNUNET_MessageHeader), - GNUNET_TIME_UNIT_FOREVER_REL, - &write_shutdown, NULL); + sizeof (struct GNUNET_MessageHeader), + GNUNET_TIME_UNIT_FOREVER_REL, + &write_shutdown, sock); } diff --git a/src/util/test_service.c b/src/util/test_service.c index f615e7eb4..037f6edf7 100644 --- a/src/util/test_service.c +++ b/src/util/test_service.c @@ -47,7 +47,6 @@ end_it (void *cls, const struct GNUNET_SCHEDULER_TaskContext *tc) GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "Shutting down service\n"); GNUNET_CLIENT_service_shutdown (client); - GNUNET_CLIENT_disconnect (client); if (sctx != NULL) GNUNET_SERVICE_stop (sctx); }