perf(api-gateway): keep-alive connection pooling for outbound S3 storage requests #9

Open
opened 2026-11-11 06:50:05 +00:00 by crapougnax · 0 comments
Owner

Summary

When proxying media streams (\Strategy: STREAMING), ^Gpi-gateway\ performs an outbound HTTP/HTTPS request to the storage provider (\ etch(storageUrl)).

In production/staging and on-premise topologies, renegotiating TCP handshakes and TLS sessions on every storage request adds significant overhead:

  • Measuring TTFB shows that 1,100 to 1,200 ms is spent negotiating connection and waiting for the storage provider to send the initial headers.
  • Cold sockets to remote S3 endpoints (or even on-premise MinIO/Silo instances) incur unnecessary connection setup latency if persistent Keep-Alive connections are not maintained.

Proposed Solution

  • Ensure outbound storage requests from ^Gpi-gateway\ reuse a warm connection pool via persistent HTTP/HTTPS Keep-Alive.
  • Verify and configure Bun's outbound fetch dispatcher / agent with:
    • Persistent socket keep-alive (\keepAlive: true, \keepAliveTimeout >= 60s).
    • Tuned max idle sockets per host.
    • DNS cache / caching resolver to prevent repetitive DNS lookups against the storage endpoint.
  • Add diagnostic metrics or benchmark logs comparing cold vs warm TTFB to the storage backend.

Expected Impact

  • Eliminates repeated TLS and TCP handshake overhead on outbound storage streaming.
  • Critical preparation for the on-premise deployment with colocalized S3 (MinIO/Silo), ensuring sub-50ms TTFB for local streaming.

Migrated from GitHub mirror: https://github.com/Quatrain/CoreApps/issues/14

## Summary When proxying media streams (\Strategy: STREAMING\), \^Gpi-gateway\ performs an outbound HTTP/HTTPS request to the storage provider (\ etch(storageUrl)\). In production/staging and on-premise topologies, renegotiating TCP handshakes and TLS sessions on every storage request adds significant overhead: - Measuring TTFB shows that **1,100 to 1,200 ms** is spent negotiating connection and waiting for the storage provider to send the initial headers. - Cold sockets to remote S3 endpoints (or even on-premise MinIO/Silo instances) incur unnecessary connection setup latency if persistent Keep-Alive connections are not maintained. ## Proposed Solution - Ensure outbound storage requests from \^Gpi-gateway\ reuse a warm connection pool via persistent HTTP/HTTPS Keep-Alive. - Verify and configure Bun's outbound fetch dispatcher / agent with: - Persistent socket keep-alive (\keepAlive: true\, \keepAliveTimeout >= 60s\). - Tuned max idle sockets per host. - DNS cache / caching resolver to prevent repetitive DNS lookups against the storage endpoint. - Add diagnostic metrics or benchmark logs comparing cold vs warm TTFB to the storage backend. ## Expected Impact - Eliminates repeated TLS and TCP handshake overhead on outbound storage streaming. - Critical preparation for the on-premise deployment with colocalized S3 (MinIO/Silo), ensuring sub-50ms TTFB for local streaming. --- _Migrated from GitHub mirror: https://github.com/Quatrain/CoreApps/issues/14_
Sign in to join this conversation.
No labels
enhancement
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
quatrain/core-apps#9
No description provided.