X-Git-Url: https://git.librecmc.org/?a=blobdiff_plain;f=TODO;h=c9141e21fc688b69e0d3d02742d5e45c8f3a0ac3;hb=7eb4b4b3ab14ef6433b08cca8c673d00f8405062;hp=0a812b45a81b62bf4b36b00ddc29f5d267ff0e58;hpb=e927f9bbf828bfc5038323c39aa3046ef948afe5;p=oweals%2Fgnunet.git diff --git a/TODO b/TODO index 0a812b45a..c9141e21f 100644 --- a/TODO +++ b/TODO @@ -1,172 +1,197 @@ -Implementable right now (but not necessarily important), with caveats -(unavailable components that will limit what can be implemented right -away), in order in which they will likely be done: -* TESTING-TESTBED [Nate] -* SETUP -* DV [Nate] -* TBENCH -* TRACEKIT -* FRAGMENTATION [Ji Lu] -* HTTP transport -* MySQL / Postgres plugins (datastore, datacache) -* UPNP [Milan] -* ARM [Safey] - -Urgent items (before announcing ng.gnunet.org): -* Get IPv6 hooked up, change DNS - -0.9.0pre0: +0.9.0pre1: +* FS: [CG] + - implement linking of downloads to searches in syncing (serialize/deserialize) + - generate SUSPEND events (publish, unindex, search, download) AND free memory! + => test SUSPEND events + - actually call 'sync' functions (publish, unindex, search, download) + - code review: + => refactor fs.c to join common code segments! + => document directory structure (and use #define's for the directory names!) + => ensure all files & dirs are cleaned up! (at least during 'clean' runs) + - persistence testing (publish, unindex, search, download): + => need driver! + => schedule suspending tasks DURING event handler => good coverage! + - gnunet-service-fs (hot-path routing, load-based routing, nitpicks) + - [gnunet-service-fs.c:208]: member 'LocalGetContext::results_bf_size' is never used + - [gnunet-service-fs.c:501]: member 'PendingRequest::used_pids_size' is never used + - [gnunet-service-fs.c:654]: member 'ConnectedPeer::last_client_replies' is never used + - [gnunet-service-fs.c:669]: member 'ConnectedPeer::avg_delay' is never used + - [gnunet-service-fs.c:675]: member 'ConnectedPeer::avg_priority' is never used + - [gnunet-service-fs.c:688]: member 'ConnectedPeer::pending_requests' is never used + - [gnunet-service-fs.c:694]: member 'ConnectedPeer::last_p2p_replies_woff' is never used + - [gnunet-service-fs.c:700]: member 'ConnectedPeer::last_client_replies_woff' is never used + - GAP improvements: + + active reply route caching design & implementation of service; gap extension! +* MIGRATION [CG] + - on-demand encoding => move logic to block-library!? + - peer selection => how to consider latency/bw/etc? + - content transmission => how often the same block? + - how to select delay before next migration? + - migration to us + - testing + - integrate with FS or not? (peer list, index/on-demand encoding, block code, + inbound priority assignment; all would be easier with tight integration!) +* TBENCH: [MW] + - good to have for transport/DV evaluation! +* DV: [Nate] + - write DV API (need to move declarations from dv_api.c to gnunet_dv_service.h!) + - implement DV service + - implement DV library (looks done) + - implement DV transport plugin + - implement testcases + - implement performance tests (needs tbench) +* TOPOLOGY: + - needs more testing (especially F2F topology) & transport blacklisting * UTIL: - - load: need way to determine network load (up/down) - - trust: need *fast* way to check/update trust in peers - (async peerinfo would not be right) + - only connect() sockets that are ready (select()) [Nils] + [On W32, we need to select after calling socket before doing connect etc.] +* GNUNET-GTK: + - use g_main_context_set_poll_func to integrate GTK with GNUnet Scheduler!? (YUCK!) + - OR: add scheduler API to enable integration with GTK main loop instead of doing our own select + - use g_main_context_pending, g_main_context_query / g_main_context_check / g_main_context_dispatch + and NEVER g_main_loop_run (can this be done? might be the clean way to do this! But how + to integrate this with "gtk_main"? Docu says: + "It's OK to use the GLib main loop directly instead of gtk_main(), though it involves + slightly more typing. See GMainLoop in the GLib documentation." + => so maybe it "just works"? +* SETUP: + - design & implement new setup tool + +0.9.0pre2: +* TRACEKIT: [MW] + - good to have for DV/DHT evaluation! +* DHT: [Nate] + - implement DHT service + - implement testcases + - implement performance tests * TRANSPORT: - - disconnect notifications from TCP not always as nice as we might want - them to be (but how can we do this cleanly given that UDP is connectionless?) - - TCP not used bi-directionally (especially important also for PONG!) - - hostlist testcase shows that (randomly) we jump to an address that - seems to be a plugin function *after* the plugin has been unloaded - (directly from scheduler); check that *all* plugin tasks are cancelled - after plugin is unloaded! (need better testcases for that...) -* STATISTICS: - - synchronous/asynchronous API (& implementation) is not nice; - => provide notification-based API - - does not seem to work with timeouts (especially if service is not running) -* ARM: + - HTTP backend [MW] +* ARM: [Safey] - need to get rid of synchronous API for service starts (cause all kinds of problems) [=> eliminate for need to tell ARM about service starts most of the time!] [Safey] -* HELLO: - - need function to test "equivalency" of HELLOs; use in topology! -* TOPOLOGY: - - needs more testing (especially F2F topology) -* CORE: - - needs more testing -* FS: - - gnunet-search cannot be aborted using CTRL-C (why?) - - 2-peer download is still too slow (why?) -* Module features to implement: - - advanced FS API parts - + namespaces: fundamental namespace API - + search: SBlocks, KSBlocks, probes, notify FS-service of known results - + collection - + location URIs (publish, search, download) - + persistence support (publish, unindex, search, download) - + indexing: index-failure-cleanup - + download: management/bounding of parallel downloads (for recursive dl.) - + datastore reservation (publishing) - + gnunet-publish (URI args) - + gnunet-download (directory) - + gnunet-search (options, incl. namespace search) - + gnunet-download (many options) - + gnunet-directory (man page, options) - + gnunet-pseudonym (all of it) - + gnunet-service-fs (remove failing on-demand blocks, stats, hot-path routing, load-based routing, nitpicks) - + datastore: do active migration support here? - - implement adv. FS testcases - + getopt API - + insert: sblocks, loc uris - + download: loc uris - + search: sblocks, skblocks, loc uris - + namespaces - + collection - - implement FS performance tests (needs TESTING) + - better tracking of which config changes actually need to cause process restarts by ARM. + - listen for requests to discover dependencies between services (and avoid + having to explicitly program start requests) + - better crash management (attach debugging support, capture and analyze + debug output, detect random vs. deterministic crashes) + - shutdown sequence? +* FS: [CG] + - datastore reservation (publishing) + - location URIs (publish, search, download) + - utilize in-line files in meta data always (including in search results or + when download is triggered manually and for probes); currently the data is + only used when users do a general 'recursive' download + - non-anonymous FS service (needs DHT) + + DHT integration for search + + CS-DHT-functions (DHT-put of LOC) + + P2P-functions (DHT-get) + - collection API & tests + + gnunet-pseudonym (collection support) + - implement FS performance tests + insert + download + search + unindex - - non-anonymous FS service (needs DHT) - + basic DHT integration - + CS-DHT-functions (DHT-put of LOC) - + P2P-functions (DHT-get) - - setup (RC-pre0) - + default generation - + need to settle basic design; do we want to keep guile? - - testing (RC-pre0) - + modify configuration to allow controlling connections for non-local starts - + testbed creation with topology (needs working F2F topology) [Nate] - + testbed with churn [Nate] - + implement library for distributed testing [Nate] - + implement testcases for distributed testing [Nate] - + test basic peer re-configure [Nate] - + test topology creation [Nate] - + test churn generation [Nate] - -0.9.0pre1: -* Module features to implement: - - tbench (RC-pre1) - + good to have for DV evaluation! - - DV (RC-pre1) - + write DV API - + implement DV service [Nate & CG] - + implement DV library [Nate] - + implement DV transport plugin [Nate & CG] - + implement testcases [Nate] - + implement performance tests [Nate] -* GUIs to implement: - - gtk (RC-pre1) - + how to integrate scheduler with GTK event loop! - -0.9.0pre2: -* Module features to implement: - - tracekit (RC-pre2) - + good to have for DV/DHT evaluation! - - DHT (RC-pre2) - + implement DHT service (needs DV, DATACACHE) - + implement DHT library - + implement testcases - + implement performance tests -* GUIs to implement: - - fuse (RC-pre2) -* Plugins to implement: - - UDP backend (RC-pre2) - + Fragmentation library - + actual plugin - - HTTP backend (RC-pre2) +* MIGRATION: + - improved content selection (not just 'get_random') 0.9.0pre3: -* GUIs to implement: - - qt (RC-pre3) - + see discussions @ FISL about integration with event loop! * Determine RC bugs and fix those! +* TESTING: [Nate] + - modify configuration to allow controlling connections for non-local starts + - testbed creation with topology (needs working F2F topology) + - testbed with churn + - implement testcases for distributed testing + - test basic peer re-configure + - test topology creation + - test churn generation + - consider changing API for peer-group termination to + call continuation when done +* NAT/UPNP: [MW] + - finalize API design + - code clean up + - testing + - integration with transport service +* MYSQL database backends: [CG] + - datacache + - datastore +* FS: + - reconstruct IBLOCKS from DBLOCKS if possible (during download; see FIXME in fs_download) + 0.9.0: -* Documentation - - update man pages - - update webpage documentation * new webpage: - expand bibliography - convert documentation pages to books - update books (especially for developers) - create good Drupal theme for GNUnet - - make a NICE download page and figure out how to - enable developers to publish TGZs nicely + - make a NICE download page and figure out how to enable developers to publish TGZs nicely - port "contact" page - add content type for "todo" items? - - DNS activation -* Plugins to implement: - - MySQL database backends - + datacache - + datastore - - Postgres database backends - + datacache - + datastore - - vpn +* POSTGRES database backends: [CG] + - datacache + - datastore * Determine RC bugs and fix those! - 0.9.x: -* Plugins to implement: +* TRANSPORT: [MW] - SMTP transport backend - HTTPS transport backend + improved HTTPS support in MHD + actual plugin -* Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009) -* Allow checking of presence of search results and/or content via command-line tools - (add options to gnunet-search / gnunet-download to limit search to local peer) -* STATISTICS: - - should use BIO instead of mmap -* PEERINFO: + - Implement method of learning our external addresses from + other peers; need some kind of threshold-based + scheme, limiting both the total number of addresses that we accept + this way as well as requiring multiple confirmations; also, we + should possibly try to confirm that the given address works for + us ourselves (loopback-style) before adding it to the list + + we may be able to simplify WELCOME messages (no need to add + addresses there anymore, but may help to learn them there anyway...). + + we probably want some kind of voting/counting for learning IP addresses + (maybe including IP addresses in ads proportional to how often others + report them? we at least need some protection against >64k HELLOs!), + + provide a way to give the user a list of "learned" IP addresses and + a way to easily "veto" addresses off the list! + => If MiM attacker uses vetoed address, blacklist the specific IP for + the presumed neighbour! + - implement gnunet-transport (transport configurator / tester) + - UPnP-based IP detection + (Note: build library always, build service when libxml2/etc. are available) +* FS: [CG] + - Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009) +* PEERINFO: [CG] - expire 'ancient' HELLOs (those without valid addresses AND that we have not 'used' (for their public keys) in a while; need a way to track actual 'use') - make sue we also trigger notifications whenever HELLOs expire +* VPN + + + +Optimizations: +* TCP: + - should use hash map to look up sessions +* STATISTICS: + - should use BIO instead of mmap +* TRANSPORT: + - need to periodically probe latency/transport cost changes & possibly switch transport + - should use hash map to look up Neighbours (service AND plugins!) +* DATASTORE (?): + - check for duplicates on insertion (currently, same content is frequently + stored again [seen with KBLOCKS and SBLOCKS]!) +* PEERINFO: + - merge multiple HELLOs of the same peer in the transmission queue + (theoretically reduces overhead; bounds message queue size) + - merge multiple iteration requests over "all" peers in the queue + (theoretically reduces overhead; bounds messgae queue size) +* FS: + - use different queue prioritization for probe-downloads vs. normal downloads (!?) + +Minor features: +* TCP: + - repeatedly resolve hostname and look up interfaces to determine our own IP + - [./transport/plugin_transport_tcp.c:391]: (style) struct or union member 'Plugin::address_update_task' is never used (related to issue above) +* TRANSPORT: + - [./transport/gnunet-service-transport.c:173]: (style) struct or union member 'TransportPlugin::rebuild' is never used (related to TCP not refreshing external addresses?) +* BLOCKS: + - testcase would be nice...