Fixed an issue where the buffering layer processes could use a lot of CPU when inflight window is full.