X-Git-Url: https://git.librecmc.org/?a=blobdiff_plain;f=TODO;h=5cacc1603d938e00e34af32745318467bdcbde60;hb=cb1069c643a772882d701553d3670ca4a4480b24;hp=f7cfc78e9a067e6fa3c2d78ef5993c3a94a7134b;hpb=b6a89cbc1db00c0a88c20066279db79364f28a8c;p=oweals%2Fgnunet.git diff --git a/TODO b/TODO index f7cfc78e9..5cacc1603 100644 --- a/TODO +++ b/TODO @@ -1,251 +1,168 @@ -PHASE #1: (Goal: settle key design questions) - - -Util: -* improve disk API [Nils] (Nils, is this done? -Christian) -* Windows: use events instead of pipes to signal select()s [Nils] -* only connect() sockets that are ready (select()) [Nils] - [On W32, we need to select after calling socket before - doing connect etc.] - -TESTCASES WANTED: -For these functions, it would be nice if we had testcases ("make check") -that would cause them to be executed and check that they are working: -* gnunet-service-peerinfo: - - change_host_trust / flush_trust - - remove_garbage / - - discard_hosts_helper / cron_clean_data_hosts -* gnunet-service-transport: - - try_unvalidated_addresses - - lookup_address_callback - - lookup_hello_callback - - plugin_env_lookup_address - - notify_clients_disconnect - - list_validated_addresses - - cleanup_validation - - disconnect_neighbour - - handle_set_quota -* plugin_transport_tcp.c: - - tcp_plugin_cancel - - tcp_plugin_address_pretty_printer / append_port - - tcp_plugin_set_receive_quota - - delayed_done -* transport_api: - - GNUNET_TRANSPORT_set_qutoa / send_set_quota - - hello_wait_timeout - - transmit_ready - - transmit_timeout - - remove_from_any_list / remove_neighbour - - GNUNET_TRANSPORT_notify_transmit_ready_cancel -* core_api: - - timeout_request - - solicit_traffic / copy_and_free - - GNUNET_CORE_peer_configure / produce_configure_message -* gnunet-service-core: - - update_window - - find_client - - handle_client_request_configure - - set_key_retry_task - - align_and_deliver - - handle_transport_notify_disconnect -* hostlist (everything) -* topology (everything) - - - -PHASE #2: (Goal: recover basic file-sharing functionality) - -Datastores: -* implement sqlite-based sqstore/datastore service - + implement service (datastore + talks to plugin) - + implement library (talks to service) - + implement sqlite plugin (talks to DB) - + fix testcases (make them use CPS, complete their inner workings...) -* implement sqlite-based dstore services - + design API - + implement library (talks to service) - + implement service (talks to plugin) - + implement sqlite plugin (talks to DB) - -Applications: -* DHT, DV -* FS / fs-libs - - - -PHASE #3: (Goal: ready for pre-release) [completion-goal: end of 2009] - -* testing -* setup -* gtk, qt GUIs -* tbench -* tracekit -* vpn - -Transports: -* UDP backend (need LIBRARY to support (de)fragmentation!) -* HTTP backend -* Testing: - + Testcases for set_quota, timeouts, disconnects, transmit_ready_cancel - + Better coverage of gnunet-service-transport (HELLO validation) - + direct test of plugins compliance to plugin API - -Databases: -* sqlite, mysql & postgres backend - - - -Minor TODO items: -* UTIL: - - crypto_hash: use libgcrypt (supports SHA-512 since 2003) - - container_bloomfilter: improve efficiency (see FIXME) -* SERVER: - - inefficient memmove -* TRANSPORT: - - transport_api: support forcing disconnects through low quotas! - (required for working F2F support!) - - API: consider having core provide deadline information for each message - (likely important for DV plugin which wants to loop back!) - - implement transport API to pretty-print transport address - + transport_api extension (API extension!) - + service-transport extension (protocol extension) - - add calls to statistics in various places - - implement gnunet-transport (transport configurator / tester) - - UPnP-based IP detection - (Note: build library always, build service when libxml2/etc. are available) - - instantly filter addresses from *other* peers that - are *equal* to our own address + port (i.e., localhost:2086). We - no longer filter those for outgoing (helps with loopback testing - and keeps the code clean), but we should filter strictly *impossible* - incoming addresses! This is for efficiency, not correctness. - - We currently are happy to take any address told to us in a WELCOME - to our set of addresses; we should have some minimal threshold-based - scheme, limiting both the total number of addresses that we accept - this way as well as requiring multiple confirmations; also, we - should possibly try to confirm that the given address works for - us ourselves (loopback-style) before adding it to the list - [SECURITY issue] - + we may be able to simplify WELCOME messages (no need to add - addresses there anymore, but may help to learn them there anyway...). - + we probably want some kind of voting/counting for learning IP addresses - (maybe including IP addresses in ads proportional to how often others - report them? we at least need some protection against >64k HELLOs!), - + provide a way to give the user a list of "learned" IP addresses and - a way to easily "veto" addresses off the list! - => If MiM attacker uses vetoed address, blacklist the specific IP for - the presumed neighbour! - - not sure current way of doing ACKs works well-enough - with unreliable transports where the ACK maybe lost; - the "is_new" check would then possibly prevent future - ACKs to be delivered, all while we're happily - receiving messages from that peer! Worse, the other - peer won't generate another ACK since it thinks we're - connected just fine... - Key questions: - + How necessary is ACKing in the first place? (alternatives?) - + Should we transmit ACKs in response to every HELLO? (would that - fully address the problem?) - - latency measurements implemented in the transport - plugins makes it only work for bi-di transports - and results in code replication - - should latency be included in the ReceiveCallback and - NotifyConnect or passed on request? - - FIXME's with latency being simply set to 0 in a few places - - Memory leak (running valgrind --trace-children=yes on test_transport_api: - ==28393== 16 bytes in 1 blocks are indirectly lost in loss record 1 of 5 - ==28393== at 0x4C2260E: malloc (vg_replace_malloc.c:207) - ==28393== by 0x52343E3: GNUNET_xmalloc_unchecked_ (common_allocation.c:62) - ==28393== by 0x5234389: GNUNET_xmalloc_ (common_allocation.c:53) - ==28393== by 0x524458A: GNUNET_NETWORK_socket_create_from_accept (network.c:289) - ==28393== by 0x524B2DA: ??? (server.c:332) - ==28393== by 0x524A4C7: ??? (scheduler.c:425) - ==28393== by 0x524A73D: GNUNET_SCHEDULER_run (scheduler.c:510) - ==28393== by 0x524FF8C: GNUNET_SERVICE_run (service.c:1326) - ==28393== by 0x405500: main (gnunet-service-transport.c:2645) - And also: - ==28393== 65,744 (65,728 direct, 16 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5 - ==28393== at 0x4C2260E: malloc (vg_replace_malloc.c:207) - ==28393== by 0x52343E3: GNUNET_xmalloc_unchecked_ (common_allocation.c:62) - ==28393== by 0x5234389: GNUNET_xmalloc_ (common_allocation.c:53) - ==28393== by 0x524473E: GNUNET_NETWORK_socket_create_from_accept (network.c:323) - (rest of trace identical) - => SVN 8727 may fix this; should test it again! - -* SETUP: - - auto-generate "defaults.conf" using gnunet-setup from "config.scm" - - integrate all options into "config.scm" - - change config-file writing to exclude options set to default values -* ARM: - - implement exponential back-off for service restarts - - better tracking of which config changes actually need to cause process restarts by ARM. - - have way to specify dependencies between services (to manage ARM restarts better) - - client-API is inefficient since it opens a TCP connection per service that is started - (instead of re-using connections). -* CORE: - - code currently notifies clients about "encrypted" connections being up well before - we get the encrypted PONG; sometimes this may be OK (for topology killing - unwanted connnections), but of course not in general. I suspect we want - to signal on PONG and have topology hook directly into transport to - kill plaintext connections before they have a chance to become encrypted - (may require minor hack in transport API) -* PEERINFO: - - have gnunet-peerinfo print actual host addresses again - - add option to gnunet-peerinfo to modify trust value -* POSTGRES-DB: - - finish postgres implementation; simplify other SQLs using new stats -* HTTPS transport - - PolariSSL for MHD? - - https integration -* GAP improvements: - - active reply route caching design & implementation of service, - gap extension! -* HOSTLIST: - - implement advertising of hostlist URL - - implement learning of hostlist URLs - - - -=> PRE-RELEASE - -PHASE #4: [completion-goal: mid 2010] -* Documentation +Implementable right now (but not necessarily important), with caveats +(unavailable components that will limit what can be implemented right +away), in order in which they will likely be done: +* TESTING +* FS (DHT not available) +* SETUP +* DV (distributed testing not available) +* TBENCH (distributed testing not available) +* TRACEKIT (distributed testing not available) +* HTTP transport +* FRAGMENTATION +* MySQL / Postgres plugins (datastore, datacache) +* UPNP + +Urgent items (before announcing ng.gnunet.org): +* FS (basic anonymous FS only) + - need way to determine network load (up/down) + - need *fast* way to check/update trust in peers + (async peerinfo would not be right) + - implement FS service + + P2P-functions (query, response, peer-disconnect) + + CS/P2P-functions (initiate P2P, handle P2P->CS) + - implement testcases + + URI API + + getopt API + + sharing API + ~ file-information + ~ publish (insert, index) + ~ download + ~ search + ~ unindex + + directory API +* TESTING (needed for DV, DHT, Topology) + - implement library for local testing + + modify configuration to allow controlling connections for non-local starts + + CORE service does not start with valid peer ID (all zeros) -- testcase fails! + + consider changing API for peer-group termination to call continuation when done + - implement testcases for library + + get test for basic peer start to work! + + test basic peer connect + + test group start +* TEST: + - topology (needs TESTING) + - hostlist (maybe easier with TESTING?) * new webpage - - - - -Stuff to remember: -* Features eliminated from util: - - threading (goal: good riddance!) - - complex logging features [ectx-passing, target-kinds] (goal: good riddance!) - - complex configuration features [defaults, notifications] (goal: good riddance!) - - network traffic monitors (goal: eliminate) - - IPC semaphores (goal: d-bus? / eliminate?) - - second timers - - DNS lookup (goal: have async service; issue: still need synchronous resolution in places, current code may not be portable) - => code shrunk from 61 files to 34, 22k LOC to 15k LOC, 470k to 330k (with symbols) -* New features in util: - - scheduler - - service and program boot-strap code -* Major changes in util: - - more expressive server (replaces selector) -* Open questions: - - how to integrate scheduler with GTK event loop! - - - -Test coverage: -* UTIL : 78.7% -* HELLO : 93.7% -* ARM : 69.9% -* RESOLVER : 60.4% -* STATISTICS: 82.8% -* PEERINFO : 71.5% -* TRANSPORT : 70.9% -* CORE : 65.8% -=================== -* TOTAL : 74.9% - -Not yet tested: -* HOSTLIST : 0.0% -* TOPOLOGY : 0.0% + - prevent? + - migrate Mantis!? + - run peer => have a 0.9.x hostlist + - improve basic documentation (configure, dependencies, what works, etc.) + - write minimal welcome/introduction to 0.9.x-development/webpage/status +=> Deploy(able) development network + + +0.9.0pre0: +* Module features to implement: + - advanced FS API parts + + namespaces: fundamental namespace API + + search: SBlocks, KSBlocks, probes, notify FS-service of known results + + collection + + location URIs (publish, search, download) + + persistence support (publish, unindex, search, download) + + indexing: index-failure-cleanup + + download: management/bounding of parallel downloads (for recursive dl.) + + datastore reservation (publishing) + + gnunet-publish (URI args) + + gnunet-download (directory) + + gnunet-search (options, incl. namespace search) + + gnunet-download (many options) + + gnunet-directory (man page, options) + + gnunet-pseudonym (all of it) + + gnunet-service-fs (remove failing on-demand blocks) + + datastore: do active migration support here? + - implement adv. FS testcases + + insert: sblocks, loc uris + + download: loc uris + + search: sblocks, skblocks, loc uris + + namespaces + + collection + - implement FS performance tests (needs TESTING) + + insert + + download + + search + + unindex + - non-anonymous FS service (needs DHT) + + basic DHT integration + + CS-DHT-functions (DHT-put of LOC) + + P2P-functions (DHT-get) + - setup (RC-pre0) + + default generation + + need to settle basic design; do we want to keep guile? + - testing (RC-pre0) + + testbed creation with topology (needs working F2F topology) [Nate] + + testbed with churn [Nate] + + implement library for distributed testing [Nate] + + implement testcases for distributed testing [Nate] + + test basic peer re-configure [Nate] + + test topology creation [Nate] + + test churn generation [Nate] + +0.9.0pre1: +* Module features to implement: + - tbench (RC-pre1) + + good to have for DV evaluation! + - DV (RC-pre1) + + write DV API + + implement DV service [Nate & CG] + + implement DV library [Nate] + + implement DV transport plugin [Nate & CG] + + implement testcases [Nate] + + implement performance tests [Nate] +* GUIs to implement: + - gtk (RC-pre1) + + how to integrate scheduler with GTK event loop! + +0.9.0pre2: +* Module features to implement: + - tracekit (RC-pre2) + + good to have for DV/DHT evaluation! + - DHT (RC-pre2) + + implement DHT service (needs DV, DATACACHE) + + implement DHT library + + implement testcases + + implement performance tests +* GUIs to implement: + - fuse (RC-pre2) +* Plugins to implement: + - UDP backend (RC-pre2) + + Fragmentation library + + actual plugin + - HTTP backend (RC-pre2) + +0.9.0pre3: +* GUIs to implement: + - qt (RC-pre3) + + see discussions @ FISL about integration with event loop! +* Determine RC bugs and fix those! + +0.9.0: +* Documentation + - update man pages + - update webpage documentation +* new webpage: + - expand bibliography + - convert documentation pages to books + - update books (especially for developers) + - create good Drupal theme for GNUnet + - make a NICE download page and figure out how to + enable developers to publish TGZs nicely + - port "contact" page; add impressum + - add content type for "todo" items? + - DNS activation +* Plugins to implement: + - MySQL database backends + + datacache + + datastore + - Postgres database backends + + datacache + + datastore + - vpn +* Determine RC bugs and fix those! + + +0.9.x: +* Plugins to implement: + - SMTP transport backend + - HTTPS transport backend + + improved HTTPS support in MHD + + actual plugin