SEARCH!

Design and implement a decentralized search mechanism for users and tweets in the WarpNet P2P network. The solution must not rely on any central server and should combine three layers:

1) DHT-based Exact Lookup

Implement DHT-based search for exact keys:

• username → node(s) that host this user
• tweetID → node(s) that host this tweet
• optionally: hashtag → node(s) with related tweets

Requirements:

1. Define a deterministic key scheme for:

   • usernames (e.g. hash(username)),
   • tweet IDs (existing or hash(content+author+timestamp)),
   • hashtags (e.g. hash(#tag)).

2. Store in the DHT pointers to nodes, not full data (e.g. node IDs / addresses).

3. Implement DHT put/get operations to:

   • publish new/updated users and tweets,
   • look up nodes that host them.

4. Ensure lookups are logarithmic or otherwise efficient (e.g. Kademlia-style routing).

5. DHT must be eventually consistent: late joins/leaves don’t break basic lookup.

2) Local Full-Text Index per Node

Each WarpNet node must maintain a local searchable index for its own data:

• tweets it stores,
• user profiles it stores (username, display name, bio).

Requirements:

1. Add a local full-text index for:

   • tweet text,
   • user fields (username, display name, bio).

2. On new or updated data:

   • index tweets and user profiles locally,
   • remove or update outdated entries.

3. Expose a local search API on each node:

   • input: search query (string, filters),
   • output: top-N local matches (with tweet/user identifiers and basic metadata).

4. Support ranking/relevance (e.g. term frequency, simple scoring).

5. Local search must not depend on other nodes being online.

3) Gossip / Query Propagation with TTL

Implement distributed search across multiple nodes using limited query propagation.

Requirements:

1. Define a search query message format:

   • query text,
   • query type (user/tweet/both),
   • TTL (max hops),
   • unique query ID (for deduplication),
   • optional limits (top-N per node).

2. When a node receives a search query:

   • run local full-text search,
   • return local results to the originator (or defined aggregator),
   • decrement TTL and forward the query only to selected neighbors,
   • avoid reprocessing the same query ID twice.

3. Implement TTL-based limiting (e.g. TTL=2–3) to prevent flooding the whole network.

4. Handle slow/unreachable nodes:

   • aggregate partial results within a timeout,
   • do not block the UI waiting for every node to respond.

5. Implement client-side aggregation:

   • merge results from multiple nodes,
   • de-duplicate tweets/users by ID,
   • globally re-rank or at least group by relevance.

6. Accept that results are partial and eventually consistent (no global completeness guarantee).

General Acceptance Criteria

1. Users can:

   • search by username (exact or prefix),
   • search by tweet text (keywords).

2. Exact lookups (username, tweet ID) work via DHT and return at least one node that hosts the target.

3. Each node maintains and uses a local full-text index for its own data.

4. Text search queries are propagated with a limited TTL and return aggregated results from multiple nodes.

5. The system continues working with partial visibility when some nodes are offline or slow.

6. No central search server or global index is required; everything runs on top of the P2P network.