X-Git-Url: https://git.librecmc.org/?a=blobdiff_plain;f=TODO;h=ad814b9473ed13fe40b36f444f3d7a80c0f820b5;hb=a985411066a41bc9d656d844d4bfd8f57ce16036;hp=3e30c44c6b4553710837e4493b7bd0e2799cab37;hpb=8de34260292a2e6f7bb980dcefd8d95f5763ad25;p=oweals%2Fgnunet.git diff --git a/TODO b/TODO index 3e30c44c6..ad814b947 100644 --- a/TODO +++ b/TODO @@ -1,247 +1,175 @@ -PHASE #1: (Goal: settle key design questions) - - -Util: -* improve disk API [Nils] (Nils, is this done? -Christian) -* Windows: use events instead of pipes to signal select()s [Nils] -* only connect() sockets that are ready (select()) [Nils] - [On W32, we need to select after calling socket before - doing connect etc.] - -TESTCASES WANTED: -For these functions, it would be nice if we had testcases ("make check") -that would cause them to be executed and check that they are working: -* gnunet-service-peerinfo: - - change_host_trust / flush_trust - - remove_garbage / - - discard_hosts_helper / cron_clean_data_hosts -* gnunet-service-transport: - - try_unvalidated_addresses - - lookup_address_callback - - lookup_hello_callback - - plugin_env_lookup_address - - notify_clients_disconnect - - list_validated_addresses - - cleanup_validation - - disconnect_neighbour - - handle_set_quota -* plugin_transport_tcp.c: - - tcp_plugin_cancel - - tcp_plugin_address_pretty_printer / append_port - - tcp_plugin_set_receive_quota - - delayed_done -* transport_api: - - GNUNET_TRANSPORT_set_qutoa / send_set_quota - - hello_wait_timeout - - transmit_ready - - transmit_timeout - - remove_from_any_list / remove_neighbour - - GNUNET_TRANSPORT_notify_transmit_ready_cancel -* core_api: - - timeout_request - - solicit_traffic / copy_and_free - - GNUNET_CORE_peer_configure / produce_configure_message -* gnunet-service-core: - - update_window - - find_client - - handle_client_request_configure - - set_key_retry_task - - align_and_deliver - - handle_transport_notify_disconnect -* hostlist (everything) -* topology (everything) - - - -PHASE #2: (Goal: recover basic file-sharing functionality) - -Datastores: -* implement sqlite-based sqstore/datastore service - + complete service implementation and improve testcase coverage -* implement sqlite-based dstore services - + design API - + implement library (talks to service) - + implement service (talks to plugin) - + implement sqlite plugin (talks to DB) - -Applications: -* DHT, DV -* FS / fs-libs - - - -PHASE #3: (Goal: ready for pre-release) [completion-goal: end of 2009] - -* testing -* setup -* gtk, qt GUIs -* tbench -* tracekit -* vpn - -Transports: -* UDP backend (need LIBRARY to support (de)fragmentation!) -* HTTP backend -* Testing: - + Testcases for set_quota, timeouts, disconnects, transmit_ready_cancel - + Better coverage of gnunet-service-transport (HELLO validation) - + direct test of plugins compliance to plugin API - -Databases: -* sqlite, mysql & postgres backend - - - -Minor TODO items: -* UTIL: - - crypto_hash: use libgcrypt (supports SHA-512 since 2003) - - container_bloomfilter: improve efficiency (see FIXME) -* SERVER: - - inefficient memmove +Implementable right now (but not necessarily important), with caveats +(unavailable components that will limit what can be implemented right +away), in order in which they will likely be done: +* TESTING-TESTBED [Nate] +* SETUP +* DV [Nate] +* TBENCH +* TRACEKIT +* FRAGMENTATION [Ji Lu] +* HTTP transport +* MySQL / Postgres plugins (datastore, datacache) +* UPNP [Milan] +* ARM [Safey] + +Urgent items (before announcing ng.gnunet.org): * TRANSPORT: - - transport_api: support forcing disconnects through low quotas! - (required for working F2F support!) - - API: consider having core provide deadline information for each message - (likely important for DV plugin which wants to loop back!) - - implement transport API to pretty-print transport address - + transport_api extension (API extension!) - + service-transport extension (protocol extension) - - add calls to statistics in various places - - implement gnunet-transport (transport configurator / tester) - - UPnP-based IP detection - (Note: build library always, build service when libxml2/etc. are available) - - instantly filter addresses from *other* peers that - are *equal* to our own address + port (i.e., localhost:2086). We - no longer filter those for outgoing (helps with loopback testing - and keeps the code clean), but we should filter strictly *impossible* - incoming addresses! This is for efficiency, not correctness. - - We currently are happy to take any address told to us in a WELCOME - to our set of addresses; we should have some minimal threshold-based - scheme, limiting both the total number of addresses that we accept - this way as well as requiring multiple confirmations; also, we - should possibly try to confirm that the given address works for - us ourselves (loopback-style) before adding it to the list - [SECURITY issue] - + we may be able to simplify WELCOME messages (no need to add - addresses there anymore, but may help to learn them there anyway...). - + we probably want some kind of voting/counting for learning IP addresses - (maybe including IP addresses in ads proportional to how often others - report them? we at least need some protection against >64k HELLOs!), - + provide a way to give the user a list of "learned" IP addresses and - a way to easily "veto" addresses off the list! - => If MiM attacker uses vetoed address, blacklist the specific IP for - the presumed neighbour! - - not sure current way of doing ACKs works well-enough - with unreliable transports where the ACK maybe lost; - the "is_new" check would then possibly prevent future - ACKs to be delivered, all while we're happily - receiving messages from that peer! Worse, the other - peer won't generate another ACK since it thinks we're - connected just fine... - Key questions: - + How necessary is ACKing in the first place? (alternatives?) - + Should we transmit ACKs in response to every HELLO? (would that - fully address the problem?) - - latency measurements implemented in the transport - plugins makes it only work for bi-di transports - and results in code replication - - should latency be included in the ReceiveCallback and - NotifyConnect or passed on request? - - FIXME's with latency being simply set to 0 in a few places - - Memory leak (running valgrind --trace-children=yes on test_transport_api: - ==28393== 16 bytes in 1 blocks are indirectly lost in loss record 1 of 5 - ==28393== at 0x4C2260E: malloc (vg_replace_malloc.c:207) - ==28393== by 0x52343E3: GNUNET_xmalloc_unchecked_ (common_allocation.c:62) - ==28393== by 0x5234389: GNUNET_xmalloc_ (common_allocation.c:53) - ==28393== by 0x524458A: GNUNET_NETWORK_socket_create_from_accept (network.c:289) - ==28393== by 0x524B2DA: ??? (server.c:332) - ==28393== by 0x524A4C7: ??? (scheduler.c:425) - ==28393== by 0x524A73D: GNUNET_SCHEDULER_run (scheduler.c:510) - ==28393== by 0x524FF8C: GNUNET_SERVICE_run (service.c:1326) - ==28393== by 0x405500: main (gnunet-service-transport.c:2645) - And also: - ==28393== 65,744 (65,728 direct, 16 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5 - ==28393== at 0x4C2260E: malloc (vg_replace_malloc.c:207) - ==28393== by 0x52343E3: GNUNET_xmalloc_unchecked_ (common_allocation.c:62) - ==28393== by 0x5234389: GNUNET_xmalloc_ (common_allocation.c:53) - ==28393== by 0x524473E: GNUNET_NETWORK_socket_create_from_accept (network.c:323) - (rest of trace identical) - -* SETUP: - - auto-generate "defaults.conf" using gnunet-setup from "config.scm" - - integrate all options into "config.scm" - - change config-file writing to exclude options set to default values -* ARM: - - implement exponential back-off for service restarts - - better tracking of which config changes actually need to cause process restarts by ARM. - - have way to specify dependencies between services (to manage ARM restarts better) - - client-API is inefficient since it opens a TCP connection per service that is started - (instead of re-using connections). -* CORE: - - code currently notifies clients about "encrypted" connections being up well before - we get the encrypted PONG; sometimes this may be OK (for topology killing - unwanted connnections), but of course not in general. I suspect we want - to signal on PONG and have topology hook directly into transport to - kill plaintext connections before they have a chance to become encrypted - (may require minor hack in transport API) -* PEERINFO: - - have gnunet-peerinfo print actual host addresses again - - add option to gnunet-peerinfo to modify trust value -* POSTGRES-DB: - - finish postgres implementation; simplify other SQLs using new stats -* HTTPS transport - - PolariSSL for MHD? - - https integration -* GAP improvements: - - active reply route caching design & implementation of service, - gap extension! + - nondeterministic transport testcase failures [Nate] +* CORE: + - test currently fails + - request connect not working [Christian, need transport first] +* TOPOLOGY: + - needs testing [need transport first] * HOSTLIST: - - implement advertising of hostlist URL - - implement learning of hostlist URLs - - - -=> PRE-RELEASE - -PHASE #4: [completion-goal: mid 2010] -* Documentation + - needs testing [need transport first] +* FS (basic anonymous FS only) + - test multi-peer search/download [needs core first] * new webpage + - run peer => have a 0.9.x hostlist + - improve basic documentation (configure, dependencies, what works, etc.) + - write minimal welcome/introduction to 0.9.x-development/webpage/status +=> Deploy(able) development network - - -Stuff to remember: -* Features eliminated from util: - - threading (goal: good riddance!) - - complex logging features [ectx-passing, target-kinds] (goal: good riddance!) - - complex configuration features [defaults, notifications] (goal: good riddance!) - - network traffic monitors (goal: eliminate) - - IPC semaphores (goal: d-bus? / eliminate?) - - second timers - - DNS lookup (goal: have async service; issue: still need synchronous resolution in places, current code may not be portable) - => code shrunk from 61 files to 34, 22k LOC to 15k LOC, 470k to 330k (with symbols) -* New features in util: - - scheduler - - service and program boot-strap code -* Major changes in util: - - more expressive server (replaces selector) -* Open questions: - - how to integrate scheduler with GTK event loop! - - - -Test coverage: -* UTIL : 78.7% -* HELLO : 93.7% -* ARM : 69.9% -* RESOLVER : 60.4% -* STATISTICS: 82.8% -* PEERINFO : 71.5% -* TRANSPORT : 70.9% -* CORE : 65.8% -=================== -* TOTAL : 74.9% - -Not yet tested: -* HOSTLIST : 0.0% -* TOPOLOGY : 0.0% +0.9.0pre0: +* UTIL: + - load: need way to determine network load (up/down) + - trust: need *fast* way to check/update trust in peers + (async peerinfo would not be right) + - OS: existing waitpid call is not nice (not integratable with scheduler! fix this!) + => should 'scheduler' support SIGCHILD explicitly? Always? +* STATISTICS: + - synchronous/asynchronous API (& implementation) is not nice; + => provide notification-based API + - does not seem to work with timeouts (especially if service is not running) +* ARM: + - need to get rid of synchronous API for service starts (cause all kinds of problems) + [=> eliminate for need to tell ARM about service starts most of the time!] [Safey] +* HELLO: + - need function to test "equivalency" of HELLOs; use in topology! +* Module features to implement: + - advanced FS API parts + + namespaces: fundamental namespace API + + search: SBlocks, KSBlocks, probes, notify FS-service of known results + + collection + + location URIs (publish, search, download) + + persistence support (publish, unindex, search, download) + + indexing: index-failure-cleanup + + download: management/bounding of parallel downloads (for recursive dl.) + + datastore reservation (publishing) + + gnunet-publish (URI args) + + gnunet-download (directory) + + gnunet-search (options, incl. namespace search) + + gnunet-download (many options) + + gnunet-directory (man page, options) + + gnunet-pseudonym (all of it) + + gnunet-service-fs (remove failing on-demand blocks, stats, hot-path routing, load-based routing, nitpicks) + + datastore: do active migration support here? + - implement adv. FS testcases + + getopt API + + insert: sblocks, loc uris + + download: loc uris + + search: sblocks, skblocks, loc uris + + namespaces + + collection + - implement FS performance tests (needs TESTING) + + insert + + download + + search + + unindex + - non-anonymous FS service (needs DHT) + + basic DHT integration + + CS-DHT-functions (DHT-put of LOC) + + P2P-functions (DHT-get) + - setup (RC-pre0) + + default generation + + need to settle basic design; do we want to keep guile? + - testing (RC-pre0) + + modify configuration to allow controlling connections for non-local starts + + testbed creation with topology (needs working F2F topology) [Nate] + + testbed with churn [Nate] + + implement library for distributed testing [Nate] + + implement testcases for distributed testing [Nate] + + test basic peer re-configure [Nate] + + test topology creation [Nate] + + test churn generation [Nate] + +0.9.0pre1: +* Module features to implement: + - tbench (RC-pre1) + + good to have for DV evaluation! + - DV (RC-pre1) + + write DV API + + implement DV service [Nate & CG] + + implement DV library [Nate] + + implement DV transport plugin [Nate & CG] + + implement testcases [Nate] + + implement performance tests [Nate] +* GUIs to implement: + - gtk (RC-pre1) + + how to integrate scheduler with GTK event loop! + +0.9.0pre2: +* Module features to implement: + - tracekit (RC-pre2) + + good to have for DV/DHT evaluation! + - DHT (RC-pre2) + + implement DHT service (needs DV, DATACACHE) + + implement DHT library + + implement testcases + + implement performance tests +* GUIs to implement: + - fuse (RC-pre2) +* Plugins to implement: + - UDP backend (RC-pre2) + + Fragmentation library + + actual plugin + - HTTP backend (RC-pre2) + +0.9.0pre3: +* GUIs to implement: + - qt (RC-pre3) + + see discussions @ FISL about integration with event loop! +* Determine RC bugs and fix those! + +0.9.0: +* Documentation + - update man pages + - update webpage documentation +* new webpage: + - expand bibliography + - convert documentation pages to books + - update books (especially for developers) + - create good Drupal theme for GNUnet + - make a NICE download page and figure out how to + enable developers to publish TGZs nicely + - port "contact" page + - add content type for "todo" items? + - DNS activation +* Plugins to implement: + - MySQL database backends + + datacache + + datastore + - Postgres database backends + + datacache + + datastore + - vpn +* Determine RC bugs and fix those! + + +0.9.x: +* Plugins to implement: + - SMTP transport backend + - HTTPS transport backend + + improved HTTPS support in MHD + + actual plugin +* Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009) +* Allow checking of presence of search results and/or content via command-line tools + (add options to gnunet-search / gnunet-download to limit search to local peer) +* STATISTICS: + - should use BIO instead of mmap +* PEERINFO: + - expire 'ancient' HELLOs (those without valid addresses AND that + we have not 'used' (for their public keys) in a while; need a way + to track actual 'use') + - make sue we also trigger notifications whenever HELLOs expire