X-Git-Url: https://git.librecmc.org/?a=blobdiff_plain;f=TODO;h=d99128a968d60eda9c3c8d03edf1bc3187d897da;hb=b93c5a4845284ddf08f57d6487a8950c36405bdc;hp=6fb70aebd86cc74119a6569eee8de8887da5cced;hpb=7b7ec4fefd44742d9368638df9670771b8eaf259;p=oweals%2Fgnunet.git diff --git a/TODO b/TODO index 6fb70aebd..d99128a96 100644 --- a/TODO +++ b/TODO @@ -1,168 +1,65 @@ -Implementable right now (but not necessarily important), with caveats -(unavailable components that will limit what can be implemented right -away), in order in which they will likely be done: -* TESTING-TESTBED [Nate] -* SETUP -* DV [Nate] -* TBENCH -* TRACEKIT -* FRAGMENTATION [Ji Lu] -* HTTP transport -* MySQL / Postgres plugins (datastore, datacache) -* UPNP [Milan] -* ARM [Safey] -* FS [Christian] - -0.9.0pre0: -* TRANSPORT: - - implement transport API to pretty-print transport address - + resolve cyclic dependency (gnunet-peerinfo -> transport_api -> peerinfo!) - + gnunet-transport-service crashes when asked to resolve address (!) - + asynchronous transport lookup and non-CPS iterator in peerinfo require - some work to ensure printing in the proper order... -* CORE: - - SET_KEY is triggered every 2.5 minutes (after session is up); should - just do PING (not SET_KEY + PING) - - "Forcing disconnect of XXX due to inactivity" -- can happen every few MS!? - (disconnect does not really succeed, or what?) - Also, core does not seem to actually decrement the number of active - sessions even if no connections exist for a while! (likely related...) - - transport reports bw quota violations on big-endian systems (core or - transport issue?) -* HELLO: - - need function to test "equivalency" of HELLOs; use in topology! -* FS: - - GAP improvements: - + active reply route caching design & implementation of service, - gap extension! - - gnunet-publish cannot be aborted using CTRL-C - - on some systems, keyword search does not find locally published content - (need testcase of command-line tools!) - - 2-peer download is still too slow (why?) - - advanced FS API parts - + namespaces: fundamental namespace API - + search: SBlocks, KSBlocks, probes, notify FS-service of known results - + collection - + location URIs (publish, search, download) - + indexing: index-failure-cleanup - + download: management/bounding of parallel downloads (for recursive dl.) - + datastore reservation (publishing) - + gnunet-publish (URI args) - + gnunet-download (directory) - + gnunet-search (options, incl. namespace search) - + gnunet-download (many options) - + gnunet-directory (man page, options) - + gnunet-pseudonym (all of it) - + gnunet-service-fs (remove failing on-demand blocks, hot-path routing, load-based routing, nitpicks) - - implement adv. FS testcases - + getopt API - + insert: sblocks, loc uris - + download: loc uris - + search: sblocks, skblocks, loc uris - + namespaces - + collection - - implement FS performance tests - + insert - + download - + search - + unindex - - [./fs/gnunet-service-fs.c:208]: (style) struct or union member 'LocalGetContext::results_bf_size' is never used - - [./fs/gnunet-service-fs.c:501]: (style) struct or union member 'PendingRequest::used_pids_size' is never used - - [./fs/gnunet-service-fs.c:654]: (style) struct or union member 'ConnectedPeer::last_client_replies' is never used - - [./fs/gnunet-service-fs.c:669]: (style) struct or union member 'ConnectedPeer::avg_delay' is never used - - [./fs/gnunet-service-fs.c:675]: (style) struct or union member 'ConnectedPeer::avg_priority' is never used - - [./fs/gnunet-service-fs.c:688]: (style) struct or union member 'ConnectedPeer::pending_requests' is never used - - [./fs/gnunet-service-fs.c:694]: (style) struct or union member 'ConnectedPeer::last_p2p_replies_woff' is never used - - [./fs/gnunet-service-fs.c:700]: (style) struct or union member 'ConnectedPeer::last_client_replies_woff' is never used - -* WWW: - - Get IPv6 hooked up - - change DNS - 0.9.0pre1: -* Module features to implement: - - tbench (RC-pre1) - + good to have for DV evaluation! - - DV (RC-pre1) - + write DV API - + implement DV service [Nate & CG] - + implement DV library [Nate] - + implement DV transport plugin [Nate & CG] - + implement testcases [Nate] - + implement performance tests [Nate] -* SETUP: - - design & implement new setup tool -* GUIs to implement: - - gtk (RC-pre1) - + how to integrate scheduler with GTK event loop! -* HOSTLIST: - - 'server' uses 'GNUNET_PEERINFO_iterate', should probably switch to notification API - (for more instant / up-to-date hostlists at lower cost) [OPTIMIZATION] -* STATISTICS: - - does not seem to work with timeouts (especially if service is not running) +* FS: [CG] + - persistence testing (publish, unindex, search, download): + => need driver! + => schedule suspending tasks DURING event handler => good coverage! + => test SUSPEND events + - gnunet-service-fs (hot-path routing, load-based routing, nitpicks) + - [gnunet-service-fs.c:208]: member 'LocalGetContext::results_bf_size' is never used + - [gnunet-service-fs.c:501]: member 'PendingRequest::used_pids_size' is never used + - [gnunet-service-fs.c:654]: member 'ConnectedPeer::last_client_replies' is never used + - [gnunet-service-fs.c:669]: member 'ConnectedPeer::avg_delay' is never used + - [gnunet-service-fs.c:675]: member 'ConnectedPeer::avg_priority' is never used + - [gnunet-service-fs.c:688]: member 'ConnectedPeer::pending_requests' is never used + - [gnunet-service-fs.c:694]: member 'ConnectedPeer::last_p2p_replies_woff' is never used + - [gnunet-service-fs.c:700]: member 'ConnectedPeer::last_client_replies_woff' is never used + - GAP improvements: + + active reply route caching design & implementation of service; gap extension! +* MIGRATION [CG] + - on-demand encoding => move logic to block-library!? + - peer selection => how to consider latency/bw/etc? + - content transmission => how often the same block? + - how to select delay before next migration? + - migration to us + - testing + - integrate with FS or not? (peer list, index/on-demand encoding, block code, + inbound priority assignment; all would be easier with tight integration!) +* TBENCH: [MW] + - good to have for transport/DV evaluation! +* DV: [Nate] + - write DV API (need to move declarations from dv_api.c to gnunet_dv_service.h!) + - implement DV service + - implement DV library (looks done) + - implement DV transport plugin + - implement testcases + - implement performance tests (needs tbench) +* TOPOLOGY: + - needs more testing (especially F2F topology) & transport blacklisting * UTIL: - - trust: need *fast* way to check/update trust in peers - (async peerinfo would not be right) - - scheduler should change OS process priority based on task priority; - should make better use of task priorities in general - only connect() sockets that are ready (select()) [Nils] - [On W32, we need to select after calling socket before - doing connect etc.] -* DATACACHE: - - add stats (# bytes available, # bytes used, # PUTs, # GETs, # GETs satisfied) -* FS: - - persistence support (publish, unindex, search, download) - - non-anonymous FS service (needs DHT) - + basic DHT integration - + CS-DHT-functions (DHT-put of LOC) - + P2P-functions (DHT-get) - - active migration support (in fs or in datastore?) -* TOPOLOGY: - - needs more testing (especially F2F topology) - - needs to re-try connecting after disconnect (currently, it - initially triggers a connection request, but if that connection - fails / goes down, it does not retry in a timely fashion; - cause seems to be the 'blacklist_after_attempt' being set to 1h, - which is rather long -- and should probably be adjusted based on - the number of connections / known peers) - - If the topology daemon crashes, peers that were put on the - blacklist with transport will never be removed from it (until - transport service dies); we should use the blacklist notification - API to learn about the exact set of blacklisted peers at all times - (FIXME: the transport_api implementation of blacklisting - also does not work nicely for this since it won't let us know about - disconnect-reconnect events and the implicit whitelisting - that might happen here; that's not so bad since we will - re-blacklist on pre-connect attempts anyway, so this is - a minor issue). - - the code uses the term 'blacklist' for both peers that are forbidden - to connect (i.e. F2F mode) as well as peers that we currently - won't try to actively connect to ourselves (since we just tried); - This is confusing. We need two distinct terms. - - move code to use hash table instead of linked list - - instead of periodically discarding blacklisted entries, - simply add task that is triggered at the right time (earlier free, - more balanced load) - - check if new HELLO learned is different from old HELLO - before resetting entire state! + [On W32, we need to select after calling socket before doing connect etc.] +* GNUNET-GTK: + - use g_main_context_set_poll_func to integrate GTK with GNUnet Scheduler!? (YUCK!) + - OR: add scheduler API to enable integration with GTK main loop instead of doing our own select + - use g_main_context_pending, g_main_context_query / g_main_context_check / g_main_context_dispatch + and NEVER g_main_loop_run (can this be done? might be the clean way to do this! But how + to integrate this with "gtk_main"? Docu says: + "It's OK to use the GLib main loop directly instead of gtk_main(), though it involves + slightly more typing. See GMainLoop in the GLib documentation." + => so maybe it "just works"? +* SETUP: + - design & implement new setup tool 0.9.0pre2: -* Module features to implement: - - tracekit (RC-pre2) - + good to have for DV/DHT evaluation! - - DHT (RC-pre2) - + implement DHT service (needs DV, DATACACHE) - + implement DHT library - + implement testcases - + implement performance tests -* GUIs to implement: - - fuse (RC-pre2) -* Plugins to implement: - - UDP backend (RC-pre2) - + Fragmentation library - + actual plugin - - HTTP backend (RC-pre2) -* ARM: +* TRACEKIT: [MW] + - good to have for DV/DHT evaluation! +* DHT: [Nate] + - implement DHT service + - implement testcases + - implement performance tests +* TRANSPORT: + - HTTP backend [MW] +* ARM: [Safey] - need to get rid of synchronous API for service starts (cause all kinds of problems) [=> eliminate for need to tell ARM about service starts most of the time!] [Safey] - better tracking of which config changes actually need to cause process restarts by ARM. @@ -171,52 +68,68 @@ away), in order in which they will likely be done: - better crash management (attach debugging support, capture and analyze debug output, detect random vs. deterministic crashes) - shutdown sequence? - +* FS: [CG] + - datastore reservation (publishing) + - location URIs (publish, search, download) + - unindex on index failure + - utilize in-line files in meta data always (including in search results or + when download is triggered manually and for probes); currently the data is + only used when users do a general 'recursive' download + - non-anonymous FS service (needs DHT) + + DHT integration for search + + CS-DHT-functions (DHT-put of LOC) + + P2P-functions (DHT-get) + - collection API & tests + + gnunet-pseudonym (collection support) + - implement FS performance tests + + insert + + download + + search + + unindex +* MIGRATION: + - improved content selection (not just 'get_random') 0.9.0pre3: -* GUIs to implement: - - qt (RC-pre3) - + see discussions @ FISL about integration with event loop! * Determine RC bugs and fix those! -* TESTING: +* TESTING: [Nate] - modify configuration to allow controlling connections for non-local starts - - testbed creation with topology (needs working F2F topology) [Nate] - - testbed with churn [Nate] - - implement testcases for distributed testing [Nate] - - test basic peer re-configure [Nate] - - test topology creation [Nate] - - test churn generation [Nate] + - testbed creation with topology (needs working F2F topology) + - testbed with churn + - implement testcases for distributed testing + - test basic peer re-configure + - test topology creation + - test churn generation - consider changing API for peer-group termination to call continuation when done - +* NAT/UPNP: [MW] + - finalize API design + - code clean up + - testing + - integration with transport service +* MYSQL database backends: [CG] + - datacache + - datastore +* FS: + - reconstruct IBLOCKS from DBLOCKS if possible (during download; see FIXME in fs_download) + - add support for pushing "already seen" search results to FS service for bloomfilter (can wait) + - use different 'priority' for probe downloads vs. normal downloads 0.9.0: -* Documentation - - update man pages - - update webpage documentation * new webpage: - expand bibliography - convert documentation pages to books - update books (especially for developers) - create good Drupal theme for GNUnet - - make a NICE download page and figure out how to - enable developers to publish TGZs nicely + - make a NICE download page and figure out how to enable developers to publish TGZs nicely - port "contact" page - add content type for "todo" items? - - DNS activation -* Plugins to implement: - - MySQL database backends - + datacache - + datastore - - Postgres database backends - + datacache - + datastore - - vpn +* POSTGRES database backends: [CG] + - datacache + - datastore * Determine RC bugs and fix those! - 0.9.x: -* TRANSPORT: +* TRANSPORT: [MW] - SMTP transport backend - HTTPS transport backend + improved HTTPS support in MHD @@ -239,35 +152,35 @@ away), in order in which they will likely be done: - implement gnunet-transport (transport configurator / tester) - UPnP-based IP detection (Note: build library always, build service when libxml2/etc. are available) -* FS: +* FS: [CG] - Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009) - - Allow checking of presence of search results and/or content via command-line tools - (add options to gnunet-search / gnunet-download to limit search to local peer) -* PEERINFO: +* PEERINFO: [CG] - expire 'ancient' HELLOs (those without valid addresses AND that we have not 'used' (for their public keys) in a while; need a way to track actual 'use') - make sue we also trigger notifications whenever HELLOs expire +* VPN Optimizations: * TCP: - should use hash map to look up sessions -* PEERINFO: - - api creates many, many short-lived TCP connections; either some - clients should use the API differently or we need to change the - API to enable re-use of connections to the service * STATISTICS: - should use BIO instead of mmap * TRANSPORT: - need to periodically probe latency/transport cost changes & possibly switch transport - - instantly filter addresses from *other* peers that - are *equal* to our own address + port (i.e., localhost:2086). We - no longer filter those for outgoing (helps with loopback testing - and keeps the code clean), but we should filter strictly *impossible* - incoming addresses! This is for efficiency, not correctness. - - should use hash map to look up Neighbours + - should use hash map to look up Neighbours (service AND plugins!) +* DATASTORE (?): + - check for duplicates on insertion (currently, same content is frequently + stored again [seen with KBLOCKS and SBLOCKS]!) +* PEERINFO: + - merge multiple HELLOs of the same peer in the transmission queue + (theoretically reduces overhead; bounds message queue size) + - merge multiple iteration requests over "all" peers in the queue + (theoretically reduces overhead; bounds messgae queue size) +* FS: + - use different queue prioritization for probe-downloads vs. normal downloads (!?) Minor features: * TCP: @@ -275,6 +188,5 @@ Minor features: - [./transport/plugin_transport_tcp.c:391]: (style) struct or union member 'Plugin::address_update_task' is never used (related to issue above) * TRANSPORT: - [./transport/gnunet-service-transport.c:173]: (style) struct or union member 'TransportPlugin::rebuild' is never used (related to TCP not refreshing external addresses?) - - - +* BLOCKS: + - testcase would be nice...