A gRPC Deadline Should Shrink as Work Moves Downstream
An AI request's latency budget should follow every downstream gRPC call. Deadline propagation limits waiting, while application cancellation stops work already in motion.
Topic · 5 stories
5 stories tagged cancellation on AI Tamer, newest first.
An AI request's latency budget should follow every downstream gRPC call. Deadline propagation limits waiting, while application cancellation stops work already in motion.
AbortSignal.reason and throwIfAborted help distinguish cancellation from ordinary failure; combine caller cancellation with a timeout for bounded AI requests.
Python asyncio.shield protects an inner task from cancellation propagated by its waiter. The caller still receives CancelledError and must account for work that continues.
When one task in a Python TaskGroup fails, its siblings receive cancellation requests. The group waits for them and reports the failures together.
A losing tokio::select! branch is dropped. This explainer traces interrupted I/O and shows how to review each await before retrying an operation.