HPPR Replication

© R.A.Sol

Repository replication uses command 🖧EXCHANGE. It synchronizes packets while preserving ACL decisions.

🖧EXCHANGE

🖧EXCHANGE has three phases:

1. announce
2. transfer
3. finalize

Phase 1: Announce

Requester sends up to 1024 LF-separated lines:

NEED <versioned-coord>
HAVE <versioned-coord>

Responder returns FIN or PENDING followed by status lines:

SEND <coord>
DENY <coord>
RECV <coord>
HAVE <coord>

Status meanings:

• SEND: responder will send this packet
• DENY: denied by ACL or policy
• RECV: responder wants this packet from requester
• HAVE: responder already has this packet

Phase 2: Transfer

Both peers stream raw packets concurrently in both directions.

Implementations MUST perform bidirectional I/O concurrently. Unidirectional blocking can deadlock the exchange.

Packet boundaries are parsed using Data-Length.

Phase 3: Finalize

Responder reports final per-coordinate results:

FIN
OK <coord>
FAIL <coord> <type> <msg>

ACL Rules in Phase 1

For NEED <coord>:

1. check read
2. if readable and exists: SEND
3. else: DENY

For HAVE <coord>:

1. check read
2. if denied: DENY
3. if allowed and exists: HAVE
4. if missing, check write
5. if writable: RECV
6. else: DENY

Blob-Addressed 🖧EXCHANGE

🖧EXCHANGE also accepts blob hash addressing:

NEED ////B.<hash>.H3
HAVE ////B.<hash>.H3

Rules:

• only B. hashes are valid
• P. and S. in blob-addressed mode are FATAL invalid requests
• no ACL checks in this mode
• any authenticated session may request blob hash sync

Decision rules:

• NEED: blob exists => SEND, else DENY
• HAVE: blob exists => HAVE, else RECV

Transfer still uses full raw blob packets.

🖧STREAM_IN

🖧STREAM_IN ingests live trailer-format Seal segments.

Request:

• App: 🖧STREAM_IN
• data: <coordinate-prefix>

Response:

• sealed OK

After OK, connection enters relay mode and accepts trailer segments.

Per segment:

1. relay bytes to matching 🖧STREAM_OUT subscribers
2. detect segment close marker
3. verify hash and signature
4. store packet
5. emit 🖧WATCH event

ACL: write checked per segment at store time.

Disconnect behavior:

• publisher disconnect ends stream
• no explicit end marker required
• partial in-flight segment is discarded
• subscriber sockets are closed

No FATAL is injected into relay stream. Subscribers detect truncation via trailer parsing.

Size rule:

• each segment must respect 32 MiB blob limit

Continuity recommendation:

• include Previous-Segment: <hash> after first segment

🖧STREAM_OUT

🖧STREAM_OUT subscribes to live trailer bytes by prefix.

Request:

• App: 🖧STREAM_OUT
• data: <coordinate-prefix>

Behavior:

• enters dedicated relay mode immediately
• no OK response packet
• when no active publisher exists for the prefix, subscriber waits for one
• wait uses a fixed server timeout
• on wait timeout, repo writes FATAL NOT_FOUND wait timeout and closes
• forwards bytes from matching active publishers

Late joiners receive bytes buffered from current in-flight segment and then continue live.

ACL: read checked at subscription time.

On publisher disconnect, repo closes subscriber sockets. Mid-segment disconnect yields partial trailer data followed by EOF.

Completed segments remain available via 🖧GET and 🖧EXCHANGE.

Relay Chaining

A downstream repo can relay a live stream by:

1. subscribing upstream with 🖧STREAM_OUT
2. publishing downstream with 🖧STREAM_IN