−
100%
+
fit
scroll
# Cross-Mesh Sift — Specification ##
Overview `/api/things` endpoint enhancement to
automatically aggregate Things from all
accessible Boxes across mesh nodes. Replaces
single-node querying with unified multi-node
aggregation, returning deduplicated results
with precedence control. ## Architecture ###
Design Principle Portal is the mesh center. It
knows all nodes and their Boxes. By default,
aggregate all accessible Boxes unless a
parameter narrows the scope. This makes
cross-node discovery the baseline behavior.
### Query Flow 1. **Client Request** →
`/api/things?being=hex&box_order=hexlet,hexperiment,hexstone`
2. **Portal Sift** → Load mesh topology +
accessible Boxes 3. **Fan-Out** → Issue
parallel `/api/things` requests to each node
4. **Merge** → Combine results by box_order
precedence, deduplicate by document_id 5.
**Pagination** → Merge cursors, sort all
results, return next_cursor 6. **Response**
→ unified Things array with origin_box
metadata per item ### Parameters **Existing
(pass-through to all nodes):** - `being` —
Being name/handle (required for being-specific
queries) - `ring` — Ring UUIDs
(comma-separated) - `sapphire` — Thing type
filter - `limit` — Results per page (1-100,
default 20) - `cursor` — Pagination token
(RFC3339 timestamp) - `start_day` / `end_day`
— Sacred-day range - `exclude` — Excluded
types (comma-separated) - `visibility` —
public|private **New (control aggregation):**
- `box_order` — Comma-separated node names
defining precedence for deduplication -
Example: `?box_order=hexlet,hexperiment` means
if doc_id appears in both, use hexlet's
version - Default: all nodes in mesh
constellation order (typically: hexlet first)
- Single node: `?box_order=hexperiment`
queries only that node's Boxes - `boxes` —
Specific Box names to include
(comma-separated) - Example:
`?boxes=hexperiment,migration-moons-2026-05`
narrows to those Boxes only - Default: all
accessible Boxes across all nodes - Overrides
box_order (box_order still controls
precedence) ## Implementation Strategy ### Gap
cms-001: Mesh Topology & Box Discovery
**What:** Load Portal's mesh constellation and
query Box metadata from each node. **Output:**
Mapping of node_name → [Box{uuid, name,
visibility, thing_count}] **Implementation:**
- In `api_things_sift`: Check `?box_order` and
`?boxes` parameters - Query local Boxes +
remote node Boxes via HTTP - Cache Box
metadata (TTL: 5 min) - Handle node offline
gracefully (skip, return available results,
warn in response) ### Gap cms-002: Parallel
Node Fetching **What:** Issue concurrent
`/api/things` requests to each node, passing
filters through. **Input:** Deduplicated
params (being, ring, sapphire, etc.) +
box_order precedence **Output:**
Vec<(node_name, Results)> — tagged results
from each source **Implementation:** - Use
tokio::join_all for parallel HTTP requests -
Construct query string on each node (preserve
being, ring, sapphire, etc.) - Each node
returns its local Things matching filters -
Collect all results with origin node metadata
### Gap cms-003: Merge & Deduplicate by
Precedence **What:** Combine Things from
multiple nodes, keeping first version by
box_order. **Algorithm:** 1. Parse box_order:
`["hexlet", "hexperiment", "hexstone"]` 2.
Flatten all Things from all nodes into a
single vec 3. Group by `document_id` 4. For
each group: keep Thing from first node in
box_order 5. Tag each Thing: `{ ...,
origin_box: "hexlet", origin_node: "hexlet" }`
6. De-duplicate: if doc_id exists in 2 nodes,
discard non-primary copy **Example:** ``` Node
hexlet: doc_A (v1), doc_B (v1) Node
hexperiment: doc_A (v2), doc_C (v1) box_order:
hexlet, hexperiment Result: doc_A (v1 from
hexlet), doc_B (v1 from hexlet), doc_C (v1
from hexperiment) ``` ### Gap cms-004:
Pagination Across Sources **What:** Merge
paginated results from multiple nodes into a
single cursor-based stream. **Algorithm:** 1.
Collect all Things from all nodes (all pages,
respect limit) 2. Sort by `created_at DESC,
document_id DESC` (consistent ordering) 3.
Slice [:limit] to return page 4. Return
`next_cursor = things[limit-1].created_at` if
more exist 5. Client next request:
`?cursor=<next_cursor>` refetches all nodes,
continues **Note:** This is O(N) across all
sources per page. For high-volume feeds,
consider: - Limit box_order to reduce fan-out
- Cache recent aggregations - Implement
server-side cursor tracking per node
combination ## Response Format ```json {
"sacred": true, "being": { "name": "hex",
"id": "...", "handle": "hex" }, "count": 42,
"things": [ { "document_id": "...", "type":
"app.bsky.feed.post", "title": "...", "body":
"...", "gem_values": { ... }, "created_at":
"2026-05-30T...", "origin_box": "hexperiment",
"origin_node": "hexperiment" }, ... ],
"next_cursor": "2026-05-30T12:34:56Z",
"aggregation_info": { "nodes_queried":
["hexlet", "hexperiment"], "boxes_merged": 3,
"deduped_count": 5 } } ``` ## Error Handling -
**Node offline:** Return available results +
warning: `"warning": "hexperiment offline,
results from hexlet + hexstone only"` - **Box
not found:** Skip silently (Box may have been
deleted) - **Auth failure on remote node:**
Return public-only results from that node -
**Malformed box_order:** Treat as default (all
nodes) ## Rollout **Phase 1:** Implement
cms-001 + cms-002 (mesh topology + fetching)
**Phase 2:** Implement cms-003 (deduplication
by precedence) **Phase 3:** Implement cms-004
(pagination) **Phase 4:** Test with
being-profile.js on hexperiment querying
hexlet + hexperiment Boxes **Phase 5:** Deploy
to all nodes ## Testing - Unit: Deduplication
logic (same doc_id in 2 nodes) - Integration:
Query hexperiment Box from hexlet via
`/api/things?boxes=hexperiment` - E2E:
being-profile.js on hexperiment loads AT
Things from hexperiment Box on hexlet
△