I thought the point of a sliding window, especially a fixed one, is to add the next element and subtract the oldest one from a running sum, to achieve O(1) running time except the start cost. This is what an efficient constant-time median filter in image processing would do, such as Perreault and Hébert's Median Filtering in Constant Time, and Weiss' Fast Median and Bilateral Filtering do.
Edit: I can't see the visualizer explaining this part.