Proxying inference requests in 6ms with Pingora, Envoy, and Spanner
4 points by birdculture 8 hours ago | 1 comments
  • prios8 8 hours ago |
    Really interesting approach. How do retries work if a stream drops halfway through?