inference performance / TOKEN WIKI
The total system throughput of
Streaming output / Streaming
In one sentence
streaming output allows applications to receive content incrementally before the complete result is generated, often used for conversations and real-time interactions.
Why it matters
It improves perceived waiting but does not automatically reduce total token usage or total computational cost.
Practical considerations
handles connection interruption, timeout, end mark and final usage to ensure that retries do not generate repeated business actions. Is the
FAQ
streaming output cheaper?
billing depends on the supplier and actual usage, and the streaming itself does not represent a discount. What is
