From: Christian Grothoff Date: Fri, 3 Dec 2010 03:11:42 +0000 (+0000) Subject: air plane hacking X-Git-Tag: initial-import-from-subversion-38251~19554 X-Git-Url: https://git.librecmc.org/?a=commitdiff_plain;h=03b10ac22fb7e80a02568af591e0ead6c76d8563;p=oweals%2Fgnunet.git air plane hacking --- diff --git a/TODO b/TODO index 0cf8c7829..c3a81fe3a 100644 --- a/TODO +++ b/TODO @@ -1,37 +1,17 @@ 0.9.0pre2: * BIG CORE REFACTORING: - - fix transport service API (ATS!) - - fix transport plugin API (ATS!) - - actually transmit ATS data through core API - - fix FS 'latency' ATS function - - fix DV + - fix transport plugin API (ATS!) [mw] + - fix DV [nate] * Integration test: - test bootstrapping via hostlist + transport/core/dht connect - -0.9.0pre3: -* Determine RC bugs and fix those (release should have no known real bugs) -* ARM: [CG/Safey] - - start_forwarding should not use 'listen_info->service_addr' for connecting since that one can be 0.0.0.0; - if it is, we need to use (IPv4 & IPv6) loopback [LRN] - - better tracking of which config changes actually need to cause process restarts by ARM. - - handle gnunet-arm -k in combination with auto-start magic (what is the right thing here?) - - discover dependencies between services * CORE: - - provide 'reliable' P2P transmission API and queueing infrastructure - Jun 27 11:51:54 core-7670 ERROR Assertion failed at gnunet-service-core.c:3616. (transport notified us that we connected to ourselves!!!; can we still reproduce this?) -* DHT: [Nate] - - track paths content travels (PUT, reply-to-get) in messages, - pass to client (client API & protocol already support this!) -* DATASTORE: - - GNUNET_DATASTORE_cancel method not tested -* TESTING: [Nate] - - test basic peer re-configure -* TOPOLOGY: [Nate] - - needs more testing (especially F2F topology) & transport blacklisting -* TRANSPORT-TCP [MW]: - - should use hash map to look up sessions -* TRANSPORT: + => Also, we may want to issue a 'connect to ourselves' automatically on start of + core_api AND allow messages to self [CG/phillip] + +0.9.0pre3: +* TRANSPORT: [MW] - [#1585, #1583, #1582] transport-level disconnect (i.e. TCP) does not cause core-level disconnect in a timely fashion (waits for connection timeout); need to figure a way to make it near-instant in those cases @@ -41,73 +21,44 @@ likely good enough until we get ATS going; still should be tested... => "peers connected (transport)" now instantly goes to ZERO (core statistic), but "established sessions" stays up... -* NAT/UPNP: [Milan] - - finalize API design +* NAT/UPNP: [Milan / Ayush / MW] - [#1609] code clean up - testing - - integration with transport service - - also do UPnP-based (external) IP detection - (Note: build library always, build UPnP service when dependencies like libxml2 are available) + - integration with transport service: + + test TCP + + implement UDP, HTTP/HTTPS +* DHT: [Nate] + - track paths content travels (PUT, reply-to-get) in messages, + pass to client (client API & protocol already support this!) * FS: [CG] - - library: - + reconstruct IBLOCKS from DBLOCKS if possible (during download; see FIXME in fs_download) - + add support for pushing "already seen" search results to FS service for bloomfilter - + use different 'priority' for probe downloads vs. normal downloads - implement multi-peer FS performance tests + insert + download + search - + unindex - - implement anonymity level > 1 - - re-implement gnunet-auto-share -* GNUNET-GTK: - - finish publish dialog details: - + normalize keywords (edit subdialog) - - implement download by URI dialog; figure out where to display those downloads! - - figure out where in the GUI we should show active uploads/unindex operations and allow aborts - - implement unindex operation (use dialog with all indexed files for selection) - - events: +* GNUNET-GTK: [CG] + - figure out where in the GUI we should show active upload operations and allow aborts + - handle events: + search error + publish error - + unindex error -* MONKEY: [Safey] - - better crash management (attach debugging support, capture and analyze - debug output, detect random vs. deterministic crashes) - - '-m EMAIL' option for alternative e-mail TO - - '-f FILENAME' option to write report to file instead of e-mail (for testing!) +* Determine RC bugs and fix those (release should have no known real bugs) 0.9.0: -* Determine RC bugs and fix those (release should have no known real bugs) -* UTIL: - - only connect() sockets that are ready (select()) [Nils] - [On W32, we need to select after calling socket before doing connect etc.] -* new webpage: [BL] - - convert documentation pages to books +* new webpage: - update books (especially for developers) - make a NICE download page and figure out how to enable developers to publish TGZs nicely - port "contact" page - - add content type for "todo" items -* TBENCH: [MW] - - good to have for transport/DV evaluation! -* TRACEKIT: [MW] - - good to have for DHT evaluation! -* DHT: [Nate] - - performance tests -* BLOCK: - - more testing (KBlock, SBlock, NBlock) * FS migration: - - exclude content that will "soon" expire from migration? - - exclude content with zero-anonymity from gap migration? + - exclude content that will "soon" expire from migration - make migration data rate & datastore IO-rate configurable - exclude certain peers as targets (based on hash values) in each iteration => same peer can only be picked every n-th iteration for the same content => fewer duplicate sending! - +* big code review +* Determine RC bugs and fix those (release should have no known real bugs) 0.9.1: * TRANSPORT: [MW] - WiFi transport backend [DB] - - implement gnunet-transport (transport configurator / tester) - Implement method of learning our external addresses from other peers; need some kind of threshold-based scheme, limiting both the total number of addresses that we accept @@ -124,8 +75,7 @@ => If MiM attacker uses vetoed address, blacklist the specific IP for the presumed neighbour! - need to periodically probe latency/transport cost changes & possibly switch transport - - should use hash map to look up Neighbours (service AND plugins!) -* DV: [Nate] +* DV: [Nate?] - proper bandwidth allocation - performance tests * PEERINFO: @@ -133,11 +83,12 @@ (theoretically reduces overhead; bounds message queue size) - merge multiple iteration requests over "all" peers in the queue (theoretically reduces overhead; bounds messgae queue size) -* STATISTICS: [CG] +* STATISTICS: - should use BIO instead of mmap * FS: [CG] - Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009) - use different queue prioritization for probe-downloads vs. normal downloads + - re-implement gnunet-auto-share * UTIL: [CG] - allow limiting UNIX socket access by UID/GID * GNUNET-GTK: [CG] @@ -148,6 +99,19 @@ - right-clicking on NS list in search dialog should open menu that allows * viewing full meta data * deletion of namespace info + - implement unindex operation (use dialog with all indexed files for selection) + - finish publish dialog details: + + normalize keywords (edit subdialog) + - implement download by URI dialog; figure out where to display those downloads! +* ARM: [CG/Safey] + - better tracking of which config changes actually need to cause process restarts by ARM. + - handle gnunet-arm -k in combination with auto-start magic (what is the right thing here?) + - discover dependencies between services +* MONKEY: [Safey] + - better crash management (attach debugging support, capture and analyze + debug output, detect random vs. deterministic crashes) + - '-m EMAIL' option for alternative e-mail TO + - '-f FILENAME' option to write report to file instead of e-mail (for testing!) 0.9.2: * PEERINFO: [NN] @@ -167,7 +131,7 @@ - testcase would be nice - generic block support for DHT * STATISTICS: - - test notification-based statistics API [LT] + - test notification-based statistics API - implement statistics GUI (=> start from gnunet-gtk by button!) * PEERINFO: [NN] - move peerinfo to new GUI (=> start from gnunet-gtk by button!) @@ -182,10 +146,7 @@ - improved batching - resource limit integration with ATS * VPN [PT] - - DNS hijacking - - DNS exit - TCP entry/exit - - UDP entry/exit - internal services - integration with DHT routing - optimized routes (beyond DHT/DV) diff --git a/src/core/gnunet-service-core.c b/src/core/gnunet-service-core.c index 6053ec082..ac9a015d8 100644 --- a/src/core/gnunet-service-core.c +++ b/src/core/gnunet-service-core.c @@ -1015,7 +1015,7 @@ handle_peer_status_change (struct Neighbour *n) GNUNET_i2s (&n->peer)); #endif size = sizeof (struct PeerStatusNotifyMessage) + - (n->ats_count+1) * sizeof (struct GNUNET_TRANSPORT_ATS_Information); + n->ats_count * sizeof (struct GNUNET_TRANSPORT_ATS_Information); if (size >= GNUNET_SERVER_MAX_MESSAGE_SIZE) { GNUNET_break (0); @@ -1024,7 +1024,7 @@ handle_peer_status_change (struct Neighbour *n) n->ats_count, 0); size = sizeof (struct PeerStatusNotifyMessage) + - (n->ats_count+1) * sizeof (struct GNUNET_TRANSPORT_ATS_Information); + n->ats_count * sizeof (struct GNUNET_TRANSPORT_ATS_Information); } psnm = (struct PeerStatusNotifyMessage*) buf; psnm->header.size = htons (size); @@ -1428,7 +1428,7 @@ handle_client_iterate_peers (void *cls, n->ats_count, 0); size = sizeof (struct PeerStatusNotifyMessage) + - (n->ats_count) * sizeof (struct GNUNET_TRANSPORT_ATS_Information); + n->ats_count * sizeof (struct GNUNET_TRANSPORT_ATS_Information); } cnm = (struct ConnectNotifyMessage*) buf; cnm->header.size = htons (size); @@ -3535,7 +3535,7 @@ handle_pong (struct Neighbour *n, n->ats_count, 0); size = sizeof (struct PeerStatusNotifyMessage) + - (n->ats_count) * sizeof (struct GNUNET_TRANSPORT_ATS_Information); + n->ats_count * sizeof (struct GNUNET_TRANSPORT_ATS_Information); } cnm = (struct ConnectNotifyMessage*) buf; cnm->header.size = htons (size); diff --git a/src/fs/fs.c b/src/fs/fs.c index 736c3b026..c8ce4d651 100644 --- a/src/fs/fs.c +++ b/src/fs/fs.c @@ -1815,7 +1815,9 @@ GNUNET_FS_download_sync_ (struct GNUNET_FS_DownloadContext *dc) (GNUNET_OK != GNUNET_BIO_write_int32 (wh, (uint32_t) dc->has_finished)) || (GNUNET_OK != - GNUNET_BIO_write_int32 (wh, num_pending)) ) + GNUNET_BIO_write_int32 (wh, num_pending)) || + (GNUNET_OK != + GNUNET_BIO_write_int32 (wh, dc->start_task != GNUNET_SCHEDULER_NO_TASK)) ) { GNUNET_break (0); goto cleanup; @@ -2583,6 +2585,7 @@ deserialize_download (struct GNUNET_FS_Handle *h, uint32_t options; uint32_t status; uint32_t num_pending; + int32_t start_pending; uris = NULL; emsg = NULL; @@ -2621,7 +2624,9 @@ deserialize_download (struct GNUNET_FS_Handle *h, (GNUNET_OK != GNUNET_BIO_read_int32 (rh, &status)) || (GNUNET_OK != - GNUNET_BIO_read_int32 (rh, &num_pending)) ) + GNUNET_BIO_read_int32 (rh, &num_pending)) || + (GNUNET_OK != + GNUNET_BIO_read_int32 (rh, &start_pending)) ) { GNUNET_break (0); goto cleanup; @@ -2692,6 +2697,9 @@ deserialize_download (struct GNUNET_FS_Handle *h, signal_download_resume (dc); } GNUNET_free (uris); + if (start_pending) + dc->start_task + = GNUNET_SCHEDULER_add_now (&GNUNET_FS_download_start_task_, dc); return; cleanup: GNUNET_free_non_null (uris); diff --git a/src/fs/fs.h b/src/fs/fs.h index ccd949c59..6e6d0dd8e 100644 --- a/src/fs/fs.h +++ b/src/fs/fs.h @@ -1,4 +1,3 @@ - /* This file is part of GNUnet. (C) 2003, 2004, 2005, 2006, 2007, 2008, 2009 Christian Grothoff (and other contributing authors) @@ -914,6 +913,18 @@ GNUNET_FS_download_make_status_ (struct GNUNET_FS_ProgressInfo *pi, struct GNUNET_FS_DownloadContext *dc); +/** + * Task that creates the initial (top-level) download + * request for the file. + * + * @param cls the 'struct GNUNET_FS_DownloadContext' + * @param tc scheduler context + */ +void +GNUNET_FS_download_start_task_ (void *cls, + const struct GNUNET_SCHEDULER_TaskContext *tc); + + /** * Fill in all of the generic fields for * an unindex event and call the callback. @@ -1601,8 +1612,20 @@ struct GNUNET_FS_SearchContext * when the search is being stopped (if not * GNUNET_SCHEDULER_NO_TASK). Used for the task that adds some * artificial delay when trying to reconnect to the FS service. - */ +o */ GNUNET_SCHEDULER_TaskIdentifier task; + + /** + * How many of the entries in the search request + * map have been passed to the service so far? + */ + unsigned int search_request_map_offset; + + /** + * How many of the keywords in the KSK + * map have been passed to the service so far? + */ + unsigned int keyword_offset; /** * Anonymity level for the search. @@ -1791,6 +1814,11 @@ struct GNUNET_FS_DownloadContext */ GNUNET_SCHEDULER_TaskIdentifier task; + /** + * Task used to start the download. + */ + GNUNET_SCHEDULER_TaskIdentifier start_task; + /** * What was the size of the file on disk that we're downloading * before we started? Used to detect if there is a point in @@ -1849,6 +1877,12 @@ struct GNUNET_FS_DownloadContext * data from the meta data yet? */ int tried_full_data; + + /** + * Have we tried to reconstruct an IBLOCK from disk + * and failed (and should hence not try again?) + */ + int reconstruct_failed; }; diff --git a/src/fs/fs_download.c b/src/fs/fs_download.c index ebe9b5cac..6943f10b5 100644 --- a/src/fs/fs_download.c +++ b/src/fs/fs_download.c @@ -24,7 +24,6 @@ * * TODO: * - different priority for scheduling probe downloads? - * - check if iblocks can be computed from existing blocks (can wait, hard) */ #include "platform.h" #include "gnunet_constants.h" @@ -32,7 +31,7 @@ #include "fs.h" #include "fs_tree.h" -#define DEBUG_DOWNLOAD GNUNET_YES +#define DEBUG_DOWNLOAD GNUNET_NO /** * Determine if the given download (options and meta data) should cause @@ -415,6 +414,223 @@ match_full_data (void *cls, } + +/** + * Closure for 'reconstruct_cont' and 'reconstruct_cb'. + */ +struct ReconstructContext +{ + /** + * File handle open for the reconstruction. + */ + struct GNUNET_DISK_FileHandle *fh; + + /** + * the download context. + */ + struct GNUNET_FS_DownloadContext *dc; + + /** + * Tree encoder used for the reconstruction. + */ + struct GNUNET_FS_TreeEncoder *te; + + /** + * CHK of block we are trying to reconstruct. + */ + struct ContentHashKey chk; + + /** + * Request that was generated. + */ + struct DownloadRequest *sm; + + /** + * Offset of block we are trying to reconstruct. + */ + uint64_t offset; + + /** + * Depth of block we are trying to reconstruct. + */ + unsigned int depth; + +}; + + +/** + * Continuation after a possible attempt to reconstruct + * the current IBlock from the existing file. + * + * @param cls the 'struct ReconstructContext' + * @param tc scheduler context + */ +static void +reconstruct_cont (void *cls, + const struct GNUNET_SCHEDULER_TaskContext *tc) +{ + struct ReconstructContext *rcc = cls; + + if (rcc->te != NULL) + GNUNET_FS_tree_encoder_finish (rcc->te, NULL, NULL); + rcc->dc->reconstruct_failed = GNUNET_YES; + if (rcc->fh != NULL) + GNUNET_break (GNUNET_OK == GNUNET_DISK_file_close (rcc->fh)); + if ( (rcc->dc->th == NULL) && + (rcc->dc->client != NULL) ) + { +#if DEBUG_DOWNLOAD + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Asking for transmission to FS service\n"); +#endif + rcc->dc->th = GNUNET_CLIENT_notify_transmit_ready (rcc->dc->client, + sizeof (struct SearchMessage), + GNUNET_CONSTANTS_SERVICE_TIMEOUT, + GNUNET_NO, + &transmit_download_request, + rcc->dc); + } + else + { +#if DEBUG_DOWNLOAD + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Transmission request not issued (%p %p)\n", + rcc->dc->th, + rcc->dc->client); +#endif + } + GNUNET_free (rcc); +} + + +static void +get_next_block (void *cls, + const struct GNUNET_SCHEDULER_TaskContext *tc) +{ + struct ReconstructContext *rcc = cls; + GNUNET_FS_tree_encoder_next (rcc->te); +} + + +/** + * Function called asking for the current (encoded) + * block to be processed. After processing the + * client should either call "GNUNET_FS_tree_encode_next" + * or (on error) "GNUNET_FS_tree_encode_finish". + * + * This function checks if the content on disk matches + * the expected content based on the URI. + * + * @param cls closure + * @param query the query for the block (key for lookup in the datastore) + * @param offset offset of the block + * @param type type of the block (IBLOCK or DBLOCK) + * @param block the (encrypted) block + * @param block_size size of block (in bytes) + */ +static void +reconstruct_cb (void *cls, + const GNUNET_HashCode *query, + uint64_t offset, + unsigned int depth, + enum GNUNET_BLOCK_Type type, + const void *block, + uint16_t block_size) +{ + struct ReconstructContext *rcc = cls; + struct ProcessResultClosure prc; + struct GNUNET_FS_TreeEncoder *te; + uint64_t off; + uint64_t boff; + uint64_t roff; + unsigned int i; + + roff = offset / DBLOCK_SIZE; + for (i=rcc->dc->treedepth;i>depth;i--) + roff /= CHK_PER_INODE; + boff = roff * DBLOCK_SIZE; + for (i=rcc->dc->treedepth;i>depth;i--) + boff *= CHK_PER_INODE; + /* convert reading offset into IBLOCKs on-disk offset */ + off = compute_disk_offset (GNUNET_FS_uri_chk_get_file_size (rcc->dc->uri), + boff, + depth, + rcc->dc->treedepth); + if ( (off == rcc->offset) && + (depth == rcc->depth) && + (0 == memcmp (query, + &rcc->chk.query, + sizeof (GNUNET_HashCode))) ) + { + /* already got it! */ + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + _("Block reconstruction at offset %llu and depth %u successful\n"), + (unsigned long long) offset, + depth); + prc.dc = rcc->dc; + prc.data = block; + prc.size = block_size; + prc.type = type; + prc.query = rcc->chk.query; + prc.do_store = GNUNET_NO; + process_result_with_request (&prc, + &rcc->chk.key, + rcc->sm); + te = rcc->te; + rcc->te = NULL; + GNUNET_FS_tree_encoder_finish (te, NULL, NULL); + GNUNET_free (rcc); + return; + } + GNUNET_SCHEDULER_add_continuation (&get_next_block, + rcc, + GNUNET_SCHEDULER_REASON_PREREQ_DONE); +} + + +/** + * Function called by the tree encoder to obtain + * a block of plaintext data (for the lowest level + * of the tree). + * + * @param cls our 'struct ReconstructContext' + * @param offset identifies which block to get + * @param max (maximum) number of bytes to get; returning + * fewer will also cause errors + * @param buf where to copy the plaintext buffer + * @param emsg location to store an error message (on error) + * @return number of bytes copied to buf, 0 on error + */ +static size_t +fh_reader (void *cls, + uint64_t offset, + size_t max, + void *buf, + char **emsg) +{ + struct ReconstructContext *rcc = cls; + struct GNUNET_DISK_FileHandle *fh = rcc->fh; + ssize_t ret; + + *emsg = NULL; + if (offset != + GNUNET_DISK_file_seek (fh, + offset, + GNUNET_DISK_SEEK_SET)) + { + *emsg = GNUNET_strdup (strerror (errno)); + return 0; + } + ret = GNUNET_DISK_file_read (fh, buf, max); + if (ret < 0) + { + *emsg = GNUNET_strdup (strerror (errno)); + return 0; + } + return ret; +} + + /** * Schedule the download of the specified block in the tree. * @@ -440,8 +656,9 @@ schedule_block_download (struct GNUNET_FS_DownloadContext *dc, GNUNET_HashCode key; struct MatchDataContext mdc; struct GNUNET_DISK_FileHandle *fh; + struct ReconstructContext *rcc; - total = GNUNET_ntohll (dc->uri->data.chk.file_length); + total = GNUNET_FS_uri_chk_get_file_size (dc->uri); len = GNUNET_FS_tree_calculate_block_size (total, dc->treedepth, offset, @@ -485,12 +702,15 @@ schedule_block_download (struct GNUNET_FS_DownloadContext *dc, GNUNET_h2s (&chk->query)); #endif fh = NULL; - if ( (dc->old_file_size > off) && + if ( ( (dc->old_file_size > off) || + ( (depth < dc->treedepth) && + (dc->reconstruct_failed == GNUNET_NO) ) ) && (dc->filename != NULL) ) fh = GNUNET_DISK_file_open (dc->filename, GNUNET_DISK_OPEN_READ, GNUNET_DISK_PERM_NONE); if ( (fh != NULL) && + (dc->old_file_size > off) && (off == GNUNET_DISK_file_seek (fh, off, @@ -517,46 +737,31 @@ schedule_block_download (struct GNUNET_FS_DownloadContext *dc, return; } } - if (fh != NULL) - GNUNET_break (GNUNET_OK == GNUNET_DISK_file_close (fh)); - if (depth < dc->treedepth) - { - // FIXME: try if we could - // reconstitute this IBLOCK - // from the existing blocks on disk (can wait) - // (read block(s), encode, compare with - // query; if matches, simply return) - } - - if ( (dc->th == NULL) && - (dc->client != NULL) ) - { -#if DEBUG_DOWNLOAD - GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, - "Asking for transmission to FS service\n"); -#endif - dc->th = GNUNET_CLIENT_notify_transmit_ready (dc->client, - sizeof (struct SearchMessage), - GNUNET_CONSTANTS_SERVICE_TIMEOUT, - GNUNET_NO, - &transmit_download_request, - dc); - } - else + rcc = GNUNET_malloc (sizeof (struct ReconstructContext)); + rcc->fh = fh; + rcc->dc = dc; + rcc->sm = sm; + rcc->chk = *chk; + rcc->offset = off; + rcc->depth = depth; + if ( (depth < dc->treedepth) && + (dc->reconstruct_failed == GNUNET_NO) && + (fh != NULL) ) { -#if DEBUG_DOWNLOAD - GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, - "Transmission request not issued (%p %p)\n", - dc->th, - dc->client); -#endif - + rcc->te = GNUNET_FS_tree_encoder_create (dc->h, + dc->old_file_size, + rcc, + fh_reader, + &reconstruct_cb, + NULL, + &reconstruct_cont); + GNUNET_FS_tree_encoder_next (rcc->te); + return; } - + reconstruct_cont (rcc, NULL); } - /** * Suggest a filename based on given metadata. * @@ -1623,6 +1828,29 @@ deactivate_fs_download (void *cls) } +/** + * Task that creates the initial (top-level) download + * request for the file. + * + * @param cls the 'struct GNUNET_FS_DownloadContext' + * @param tc scheduler context + */ +void +GNUNET_FS_download_start_task_ (void *cls, + const struct GNUNET_SCHEDULER_TaskContext *tc) +{ + struct GNUNET_FS_DownloadContext *dc = cls; + + dc->start_task = GNUNET_SCHEDULER_NO_TASK; + schedule_block_download (dc, + (dc->uri->type == chk) + ? &dc->uri->data.chk.chk + : &dc->uri->data.loc.fi.chk, + 0, + 1 /* 0 == CHK, 1 == top */); +} + + /** * Create SUSPEND event for the given download operation * and then clean up our state (without stop signal). @@ -1634,7 +1862,12 @@ GNUNET_FS_download_signal_suspend_ (void *cls) { struct GNUNET_FS_DownloadContext *dc = cls; struct GNUNET_FS_ProgressInfo pi; - + + if (dc->start_task != GNUNET_SCHEDULER_NO_TASK) + { + GNUNET_SCHEDULER_cancel (dc->start_task); + dc->start_task = GNUNET_SCHEDULER_NO_TASK; + } if (dc->top != NULL) GNUNET_FS_end_top (dc->h, dc->top); while (NULL != dc->child_head) @@ -1785,12 +2018,7 @@ GNUNET_FS_download_start (struct GNUNET_FS_Handle *h, pi.status = GNUNET_FS_STATUS_DOWNLOAD_START; pi.value.download.specifics.start.meta = meta; GNUNET_FS_download_make_status_ (&pi, dc); - schedule_block_download (dc, - (dc->uri->type == chk) - ? &dc->uri->data.chk.chk - : &dc->uri->data.loc.fi.chk, - 0, - 1 /* 0 == CHK, 1 == top */); + dc->start_task = GNUNET_SCHEDULER_add_now (&GNUNET_FS_download_start_task_, dc); GNUNET_FS_download_sync_ (dc); GNUNET_FS_download_start_downloading_ (dc); return dc; @@ -1913,16 +2141,12 @@ GNUNET_FS_download_start_from_search (struct GNUNET_FS_Handle *h, pi.status = GNUNET_FS_STATUS_DOWNLOAD_START; pi.value.download.specifics.start.meta = dc->meta; GNUNET_FS_download_make_status_ (&pi, dc); - schedule_block_download (dc, - &dc->uri->data.chk.chk, - 0, - 1 /* 0 == CHK, 1 == top */); + dc->start_task = GNUNET_SCHEDULER_add_now (&GNUNET_FS_download_start_task_, dc); GNUNET_FS_download_sync_ (dc); GNUNET_FS_download_start_downloading_ (dc); return dc; } - /** * Start the downloading process (by entering the queue). * @@ -1931,6 +2155,8 @@ GNUNET_FS_download_start_from_search (struct GNUNET_FS_Handle *h, void GNUNET_FS_download_start_downloading_ (struct GNUNET_FS_DownloadContext *dc) { + if (dc->completed == dc->length) + return; GNUNET_assert (dc->job_queue == NULL); dc->job_queue = GNUNET_FS_queue_ (dc->h, &activate_fs_download, @@ -1955,6 +2181,11 @@ GNUNET_FS_download_stop (struct GNUNET_FS_DownloadContext *dc, if (dc->top != NULL) GNUNET_FS_end_top (dc->h, dc->top); + if (dc->start_task != GNUNET_SCHEDULER_NO_TASK) + { + GNUNET_SCHEDULER_cancel (dc->start_task); + dc->start_task = GNUNET_SCHEDULER_NO_TASK; + } if (dc->search != NULL) { dc->search->download = NULL; diff --git a/src/fs/fs_publish.c b/src/fs/fs_publish.c index 5c1cd0ea2..752677f13 100644 --- a/src/fs/fs_publish.c +++ b/src/fs/fs_publish.c @@ -509,6 +509,7 @@ encode_cont (void *cls, * @param cls closure * @param query the query for the block (key for lookup in the datastore) * @param offset offset of the block in the file + * @param depth depth of the block in the file * @param type type of the block (IBLOCK or DBLOCK) * @param block the (encrypted) block * @param block_size size of block (in bytes) @@ -517,6 +518,7 @@ static void block_proc (void *cls, const GNUNET_HashCode *query, uint64_t offset, + unsigned int depth, enum GNUNET_BLOCK_Type type, const void *block, uint16_t block_size) diff --git a/src/fs/fs_search.c b/src/fs/fs_search.c index 1ffd681aa..109fc3272 100644 --- a/src/fs/fs_search.c +++ b/src/fs/fs_search.c @@ -22,10 +22,6 @@ * @file fs/fs_search.c * @brief Helper functions for searching. * @author Christian Grothoff - * - * TODO: - * - add support for pushing "already seen" information - * to FS service for bloomfilter (can wait) */ #include "platform.h" @@ -912,6 +908,110 @@ receive_results (void *cls, } +/** + * Schedule the transmission of the (next) search request + * to the service. + * + * @param sc context for the search + */ +static void +schedule_transmit_search_request (struct GNUNET_FS_SearchContext *sc); + + +/** + * Closure for 'build_result_set'. + */ +struct MessageBuilderContext +{ + /** + * How many entries can we store to xoff. + */ + unsigned int put_cnt; + + /** + * How many entries should we skip. + */ + unsigned int skip_cnt; + + /** + * Where to store the keys. + */ + GNUNET_HashCode *xoff; + + /** + * Search context we are iterating for. + */ + struct GNUNET_FS_SearchContext *sc; + + /** + * URI the search result must match, NULL for any + */ + struct GNUNET_FS_Uri *uri; +}; + + +/** + * Iterating over the known results, pick those + * matching the given result range and store + * their keys at 'xoff'. + * + * @param cls the 'struct MessageBuilderContext' + * @param key key for a result + * @param value the search result + * @return GNUNET_OK to continue iterating + */ +static int +build_result_set (void *cls, + const GNUNET_HashCode *key, + void *value) +{ + struct MessageBuilderContext *mbc = cls; + struct GNUNET_FS_SearchResult *sr = value; + + if ( (mbc->uri != NULL) && + (GNUNET_YES != GNUNET_FS_uri_test_equal (mbc->uri, + sr->uri)) ) + return GNUNET_OK; + if (mbc->skip_cnt > 0) + { + mbc->skip_cnt--; + return GNUNET_OK; + } + if (mbc->put_cnt == 0) + return GNUNET_SYSERR; + mbc->sc->search_request_map_offset++; + mbc->xoff[--mbc->put_cnt] = *key; + return GNUNET_OK; +} + + +/** + * Iterating over the known results, count those + * matching the given result range and increment + * put count for each. + * + * @param cls the 'struct MessageBuilderContext' + * @param key key for a result + * @param value the search result + * @return GNUNET_OK to continue iterating + */ +static int +find_result_set (void *cls, + const GNUNET_HashCode *key, + void *value) +{ + struct MessageBuilderContext *mbc = cls; + struct GNUNET_FS_SearchResult *sr = value; + + if ( (mbc->uri != NULL) && + (GNUNET_YES != GNUNET_FS_uri_test_equal (mbc->uri, + sr->uri)) ) + return GNUNET_OK; + mbc->put_cnt++; + return GNUNET_OK; +} + + /** * We're ready to transmit the search request to the * file-sharing service. Do it. @@ -927,38 +1027,66 @@ transmit_search_request (void *cls, void *buf) { struct GNUNET_FS_SearchContext *sc = cls; + struct MessageBuilderContext mbc; size_t msize; struct SearchMessage *sm; - unsigned int i; const char *identifier; GNUNET_HashCode key; GNUNET_HashCode idh; + unsigned int sqms; if (NULL == buf) { try_reconnect (sc); return 0; } + mbc.sc = sc; + mbc.skip_cnt = sc->search_request_map_offset; + sm = buf; + sm->header.type = htons (GNUNET_MESSAGE_TYPE_FS_START_SEARCH); + mbc.xoff = (GNUNET_HashCode* ) &sm[1]; if (GNUNET_FS_uri_test_ksk (sc->uri)) { - msize = sizeof (struct SearchMessage) * sc->uri->data.ksk.keywordCount; + msize = sizeof (struct SearchMessage); GNUNET_assert (size >= msize); - sm = buf; - memset (sm, 0, msize); - for (i=0;iuri->data.ksk.keywordCount;i++) + mbc.uri = NULL; + mbc.put_cnt = 0; + GNUNET_CONTAINER_multihashmap_iterate (sc->master_result_map, + &find_result_set, + &mbc); + sqms = mbc.put_cnt; + mbc.put_cnt = (size - msize) / sizeof (GNUNET_HashCode); + mbc.put_cnt = GNUNET_MIN (mbc.put_cnt, + sqms - mbc.skip_cnt); + if (sc->search_request_map_offset < sqms) + GNUNET_assert (mbc.put_cnt > 0); + + sm->header.size = htons (msize); + if (0 != (sc->options & GNUNET_FS_SEARCH_OPTION_LOOPBACK_ONLY)) + sm->options = htonl (1); + else + sm->options = htonl (0); + sm->type = htonl (GNUNET_BLOCK_TYPE_ANY); + sm->anonymity_level = htonl (sc->anonymity); + sm->query = sc->requests[sc->keyword_offset].query; + msize += sizeof (GNUNET_HashCode) * mbc.put_cnt; + GNUNET_CONTAINER_multihashmap_iterate (sc->master_result_map, + &build_result_set, + &mbc); + sm->header.size = htons (msize); + if (sqms != sc->search_request_map_offset) { - sm[i].header.size = htons (sizeof (struct SearchMessage)); - sm[i].header.type = htons (GNUNET_MESSAGE_TYPE_FS_START_SEARCH); - if (0 != (sc->options & GNUNET_FS_SEARCH_OPTION_LOOPBACK_ONLY)) - sm[i].options = htonl (1); - else - sm[i].options = htonl (0); - sm[i].type = htonl (GNUNET_BLOCK_TYPE_ANY); - sm[i].anonymity_level = htonl (sc->anonymity); - sm[i].query = sc->requests[i].query; - /* FIXME: should transmit hash codes of all already-known results here! - (and if they do not fit, add another message with the same - header and additional already-seen results!) */ + /* more requesting to be done... */ + schedule_transmit_search_request (sc); + return msize; + } + sc->keyword_offset++; + if (sc->uri->data.ksk.keywordCount != + sc->keyword_offset) + { + /* more requesting to be done... */ + schedule_transmit_search_request (sc); + return msize; } } else @@ -966,10 +1094,6 @@ transmit_search_request (void *cls, GNUNET_assert (GNUNET_FS_uri_test_sks (sc->uri)); msize = sizeof (struct SearchMessage); GNUNET_assert (size >= msize); - sm = buf; - memset (sm, 0, msize); - sm->header.size = htons (sizeof (struct SearchMessage)); - sm->header.type = htons (GNUNET_MESSAGE_TYPE_FS_START_SEARCH); if (0 != (sc->options & GNUNET_FS_SEARCH_OPTION_LOOPBACK_ONLY)) sm->options = htonl (1); else @@ -987,10 +1111,26 @@ transmit_search_request (void *cls, GNUNET_CRYPTO_hash_xor (&idh, &sm->target, &sm->query); - /* FIXME: should transmit hash codes of all already-known results here! - (and if they do not fit, add another message with the same - header and additional already-seen results!) */ - } + mbc.skip_cnt = sc->search_request_map_offset; + mbc.put_cnt = (size - msize) / sizeof (GNUNET_HashCode); + sqms = GNUNET_CONTAINER_multihashmap_size (sc->master_result_map); + mbc.put_cnt = GNUNET_MIN (mbc.put_cnt, + sqms - mbc.skip_cnt); + mbc.uri = NULL; + if (sc->search_request_map_offset < sqms) + GNUNET_assert (mbc.put_cnt > 0); + msize += sizeof (GNUNET_HashCode) * mbc.put_cnt; + GNUNET_CONTAINER_multihashmap_iterate (sc->master_result_map, + &build_result_set, + &mbc); + sm->header.size = htons (msize); + if (sqms != sc->search_request_map_offset) + { + /* more requesting to be done... */ + schedule_transmit_search_request (sc); + return msize; + } + } GNUNET_CLIENT_receive (sc->client, &receive_results, sc, @@ -999,6 +1139,34 @@ transmit_search_request (void *cls, } +/** + * Schedule the transmission of the (next) search request + * to the service. + * + * @param sc context for the search + */ +static void +schedule_transmit_search_request (struct GNUNET_FS_SearchContext *sc) +{ + size_t size; + unsigned int sqms; + unsigned int fit; + + size = sizeof (struct SearchMessage); + sqms = GNUNET_CONTAINER_multihashmap_size (sc->master_result_map) - sc->search_request_map_offset; + fit = (GNUNET_SERVER_MAX_MESSAGE_SIZE - 1 - size) / sizeof (GNUNET_HashCode); + fit = GNUNET_MIN (fit, sqms); + size += sizeof (GNUNET_HashCode) * fit; + GNUNET_CLIENT_notify_transmit_ready (sc->client, + size, + GNUNET_CONSTANTS_SERVICE_TIMEOUT, + GNUNET_NO, + &transmit_search_request, + sc); + +} + + /** * Reconnect to the FS service and transmit * our queries NOW. @@ -1012,8 +1180,7 @@ do_reconnect (void *cls, { struct GNUNET_FS_SearchContext *sc = cls; struct GNUNET_CLIENT_Connection *client; - size_t size; - + sc->task = GNUNET_SCHEDULER_NO_TASK; client = GNUNET_CLIENT_connect ("fs", sc->h->cfg); @@ -1023,16 +1190,9 @@ do_reconnect (void *cls, return; } sc->client = client; - if (GNUNET_FS_uri_test_ksk (sc->uri)) - size = sizeof (struct SearchMessage) * sc->uri->data.ksk.keywordCount; - else - size = sizeof (struct SearchMessage); - GNUNET_CLIENT_notify_transmit_ready (client, - size, - GNUNET_CONSTANTS_SERVICE_TIMEOUT, - GNUNET_NO, - &transmit_search_request, - sc); + sc->search_request_map_offset = 0; + sc->keyword_offset = 0; + schedule_transmit_search_request (sc); } @@ -1124,24 +1284,8 @@ GNUNET_FS_search_start_searching_ (struct GNUNET_FS_SearchContext *sc) GNUNET_HashCode hc; struct GNUNET_CRYPTO_RsaPublicKeyBinaryEncoded pub; struct GNUNET_CRYPTO_RsaPrivateKey *pk; - size_t size; GNUNET_assert (NULL == sc->client); - if (GNUNET_FS_uri_test_ksk (sc->uri)) - { - size = sizeof (struct SearchMessage) * sc->uri->data.ksk.keywordCount; - } - else - { - GNUNET_assert (GNUNET_FS_uri_test_sks (sc->uri)); - size = sizeof (struct SearchMessage); - } - if (size >= GNUNET_SERVER_MAX_MESSAGE_SIZE) - { - GNUNET_log (GNUNET_ERROR_TYPE_ERROR, - _("Too many keywords specified for a single search.")); - return GNUNET_SYSERR; - } if (GNUNET_FS_uri_test_ksk (sc->uri)) { GNUNET_assert (0 != sc->uri->data.ksk.keywordCount); @@ -1150,7 +1294,9 @@ GNUNET_FS_search_start_searching_ (struct GNUNET_FS_SearchContext *sc) for (i=0;iuri->data.ksk.keywordCount;i++) { keyword = &sc->uri->data.ksk.keywords[i][1]; - GNUNET_CRYPTO_hash (keyword, strlen (keyword), &hc); + GNUNET_CRYPTO_hash (keyword, + strlen (keyword), + &hc); pk = GNUNET_CRYPTO_rsa_key_create_from_hash (&hc); GNUNET_assert (pk != NULL); GNUNET_CRYPTO_rsa_key_get_public (pk, &pub); @@ -1171,12 +1317,7 @@ GNUNET_FS_search_start_searching_ (struct GNUNET_FS_SearchContext *sc) sc->h->cfg); if (NULL == sc->client) return GNUNET_SYSERR; - GNUNET_CLIENT_notify_transmit_ready (sc->client, - size, - GNUNET_CONSTANTS_SERVICE_TIMEOUT, - GNUNET_NO, - &transmit_search_request, - sc); + schedule_transmit_search_request (sc); return GNUNET_OK; } diff --git a/src/fs/fs_tree.c b/src/fs/fs_tree.c index b38a9c382..211714b86 100644 --- a/src/fs/fs_tree.c +++ b/src/fs/fs_tree.c @@ -109,6 +109,11 @@ struct GNUNET_FS_TreeEncoder */ struct ContentHashKey *chk_tree; + /** + * Are we currently in 'GNUNET_FS_tree_encoder_next'? + * Flag used to prevent recursion. + */ + int in_next; }; @@ -307,7 +312,8 @@ compute_chk_offset (unsigned int height, * * @param te tree encoder to use */ -void GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) +void +GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) { struct ContentHashKey *mychk; const void *pt_block; @@ -318,6 +324,8 @@ void GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) struct GNUNET_CRYPTO_AesInitializationVector iv; unsigned int off; + GNUNET_assert (GNUNET_NO == te->in_next); + te->in_next = GNUNET_YES; if (te->current_depth == te->chk_tree_depth) { pt_size = GNUNET_MIN(DBLOCK_SIZE, @@ -332,6 +340,7 @@ void GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) GNUNET_SCHEDULER_add_continuation (te->cont, te->cls, GNUNET_SCHEDULER_REASON_TIMEOUT); + te->in_next = GNUNET_NO; return; } pt_block = iob; @@ -352,6 +361,7 @@ void GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) GNUNET_SCHEDULER_add_continuation (te->cont, te->cls, GNUNET_SCHEDULER_REASON_PREREQ_DONE); + te->in_next = GNUNET_NO; return; } off = compute_chk_offset (te->chk_tree_depth - te->current_depth, @@ -382,6 +392,7 @@ void GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) te->proc (te->cls, &mychk->query, te->publish_offset, + te->current_depth, (te->current_depth == te->chk_tree_depth) ? GNUNET_BLOCK_TYPE_FS_DBLOCK : GNUNET_BLOCK_TYPE_FS_IBLOCK, @@ -408,6 +419,7 @@ void GNUNET_FS_tree_encoder_next (struct GNUNET_FS_TreeEncoder * te) else te->current_depth = te->chk_tree_depth; } + te->in_next = GNUNET_NO; } diff --git a/src/fs/fs_tree.h b/src/fs/fs_tree.h index 2cc627899..ebeb47e63 100644 --- a/src/fs/fs_tree.h +++ b/src/fs/fs_tree.h @@ -60,6 +60,7 @@ struct GNUNET_FS_TreeEncoder; * @param cls closure * @param query the query for the block (key for lookup in the datastore) * @param offset offset of the block + * @param depth depth of the block * @param type type of the block (IBLOCK or DBLOCK) * @param block the (encrypted) block * @param block_size size of block (in bytes) @@ -67,6 +68,7 @@ struct GNUNET_FS_TreeEncoder; typedef void (*GNUNET_FS_TreeBlockProcessor)(void *cls, const GNUNET_HashCode *query, uint64_t offset, + unsigned int depth, enum GNUNET_BLOCK_Type type, const void *block, uint16_t block_size); diff --git a/src/fs/fs_unindex.c b/src/fs/fs_unindex.c index b3bcd48bf..aa73bc01d 100644 --- a/src/fs/fs_unindex.c +++ b/src/fs/fs_unindex.c @@ -190,6 +190,7 @@ process_cont (void *cls, * @param cls closure * @param query the query for the block (key for lookup in the datastore) * @param offset offset of the block + * @param depth depth of the block * @param type type of the block (IBLOCK or DBLOCK) * @param block the (encrypted) block * @param block_size size of block (in bytes) @@ -198,6 +199,7 @@ static void unindex_process (void *cls, const GNUNET_HashCode *query, uint64_t offset, + unsigned int depth, enum GNUNET_BLOCK_Type type, const void *block, uint16_t block_size) diff --git a/src/fs/gnunet-download.c b/src/fs/gnunet-download.c index 99ebb1435..5301edb05 100644 --- a/src/fs/gnunet-download.c +++ b/src/fs/gnunet-download.c @@ -184,7 +184,7 @@ run (void *cls, (! GNUNET_FS_uri_test_loc (uri)) ) { fprintf (stderr, - "Only CHK or LOC URIs supported.\n"); + _("Only CHK or LOC URIs supported.\n")); ret = 1; GNUNET_FS_uri_destroy (uri); return; @@ -192,7 +192,7 @@ run (void *cls, if (NULL == filename) { fprintf (stderr, - "Target filename must be specified.\n"); + _("Target filename must be specified.\n")); ret = 1; GNUNET_FS_uri_destroy (uri); return; diff --git a/src/fs/gnunet-service-fs.c b/src/fs/gnunet-service-fs.c index f02e85bba..08b01fbbf 100644 --- a/src/fs/gnunet-service-fs.c +++ b/src/fs/gnunet-service-fs.c @@ -84,6 +84,13 @@ */ #define TRUST_FLUSH_FREQ GNUNET_TIME_relative_multiply (GNUNET_TIME_UNIT_MINUTES, 5) +/** + * How quickly do we age cover traffic? At the given + * time interval, remaining cover traffic counters are + * decremented by 1/16th. + */ +#define COVER_AGE_FREQUENCY GNUNET_TIME_relative_multiply (GNUNET_TIME_UNIT_SECONDS, 5) + /** * How often do we at most PUT content into the DHT? */ @@ -495,9 +502,6 @@ struct UsedTargetEntry }; - - - /** * Doubly-linked list of messages we are performing * due to a pending request. @@ -928,6 +932,34 @@ static struct GNUNET_LOAD_Value *datastore_put_load; */ static struct GNUNET_LOAD_Value *rt_entry_lifetime; +/** + * How many query messages have we received 'recently' that + * have not yet been claimed as cover traffic? + */ +static unsigned int cover_query_count; + +/** + * How many content messages have we received 'recently' that + * have not yet been claimed as cover traffic? + */ +static unsigned int cover_content_count; + +/** + * ID of our task that we use to age the cover counters. + */ +static GNUNET_SCHEDULER_TaskIdentifier cover_age_task; + +static void +age_cover_counters (void *cls, + const struct GNUNET_SCHEDULER_TaskContext *tc) +{ + cover_content_count = (cover_content_count * 15) / 16; + cover_query_count = (cover_query_count * 15) / 16; + cover_age_task = GNUNET_SCHEDULER_add_delayed (COVER_AGE_FREQUENCY, + &age_cover_counters, + NULL); +} + /** * We've just now completed a datastore request. Update our * datastore load calculations. @@ -2084,6 +2116,8 @@ shutdown_task (void *cls, cfg = NULL; GNUNET_free_non_null (trustDirectory); trustDirectory = NULL; + GNUNET_SCHEDULER_cancel (cover_age_task); + cover_age_task = GNUNET_SCHEDULER_NO_TASK; } @@ -3015,6 +3049,26 @@ forward_request_task (void *cls, &process_dht_reply, pr); } + + if ( (pr->anonymity_level > 1) && + (cover_query_count < pr->anonymity_level - 1) ) + { + delay = get_processing_delay (); +#if DEBUG_FS + GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, + "Not enough cover traffic to forward query `%s', will try again in %llu ms!\n", + GNUNET_h2s (&pr->query), + delay.rel_value); +#endif + pr->task = GNUNET_SCHEDULER_add_delayed (delay, + &forward_request_task, + pr); + return; + } + /* consume cover traffic */ + if (pr->anonymity_level > 1) + cover_query_count -= pr->anonymity_level - 1; + /* (1) select target */ psc.pr = pr; psc.target_score = -DBL_MAX; @@ -3220,6 +3274,11 @@ struct ProcessReplyClosure */ uint32_t priority; + /** + * Anonymity requirements for this reply. + */ + uint32_t anonymity_level; + /** * Evaluation result (returned). */ @@ -3264,6 +3323,22 @@ struct GNUNET_TIME_Relative art_delay; size_t msize; unsigned int i; + if (NULL == pr->client_request_list) + { + /* reply will go over the network, check for cover traffic */ + if ( (prq->anonymity_level > 1) && + (cover_content_count < prq->anonymity_level - 1) ) + { + /* insufficient cover traffic, skip */ + GNUNET_STATISTICS_update (stats, + gettext_noop ("# replies suppressed due to lack of cover traffic"), + 1, + GNUNET_NO); + return GNUNET_YES; + } + if (prq->anonymity_level > 1) + cover_content_count -= prq->anonymity_level - 1; + } #if DEBUG_FS GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "Matched result (type %u) for query `%s' with pending request\n", @@ -3603,6 +3678,7 @@ handle_p2p_put (void *cls, GNUNET_break_op (0); return GNUNET_SYSERR; } + cover_content_count++; #if DEBUG_FS GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "Received result for query `%s' from peer `%4s'\n", @@ -3624,6 +3700,7 @@ handle_p2p_put (void *cls, prq.type = type; prq.expiration = expiration; prq.priority = 0; + prq.anonymity_level = 1; prq.finished = GNUNET_NO; prq.request_found = GNUNET_NO; GNUNET_CONTAINER_multihashmap_get_multiple (query_request_map, @@ -3905,6 +3982,7 @@ process_local_reply (void *cls, prq.priority = priority; prq.finished = GNUNET_NO; prq.request_found = GNUNET_NO; + prq.anonymity_level = anonymity; if ( (old_rf == 0) && (pr->results_found == 0) ) update_datastore_delays (pr->start_time); @@ -4100,6 +4178,7 @@ handle_p2p_get (void *cls, GNUNET_break_op (0); return GNUNET_SYSERR; } + cover_query_count++; bm = ntohl (gm->hash_bitmap); bits = 0; cps = GNUNET_CONTAINER_multihashmap_get (connected_peers, @@ -4625,6 +4704,9 @@ main_init (struct GNUNET_SERVER_Handle *server, GNUNET_SERVER_add_handlers (server, handlers); + cover_age_task = GNUNET_SCHEDULER_add_delayed (COVER_AGE_FREQUENCY, + &age_cover_counters, + NULL); GNUNET_SCHEDULER_add_delayed (GNUNET_TIME_UNIT_FOREVER_REL, &shutdown_task, NULL);