VideoToolbox emits keyframes early, and my HLS segments came out twice as long

Kevin McMahon · August 25, 2026

I have a tvOS app, AmbientCast, that takes a 3D printer's camera feed, composites a status overlay onto it, re-encodes it with VideoToolbox and serves it as HLS to a local player. Segments are cut on keyframes, and the encoder is configured for a keyframe every second, so segments should be a second long.

On a real camera, one segment a minute came out at about 1.97 seconds. Fourteen consecutive minutes, fourteen double-length segments. On the app's synthetic demo feed — same build, same encoder settings, same minute — the longest segment was 1.01 seconds and it never happened once.

The measurement

A minute-by-minute probe on a physical Apple TV 4K (3rd generation) running tvOS 27, against a Bambu Lab P2S camera at 1920×1080 delivering around 28 fps. Segment count, mean length, and the longest segment in each window:

54 segments in 60.8s   mean 1.13s   longest 1.96s
56 segments in 60.1s   mean 1.07s   longest 1.98s
56 segments in 60.7s   mean 1.08s   longest 1.96s
56 segments in 60.9s   mean 1.09s   longest 1.97s
53 segments in 60.3s   mean 1.14s   longest 1.97s
57 segments in 61.3s   mean 1.07s   longest 1.98s
53 segments in 60.1s   mean 1.13s   longest 1.96s
52 segments in 61.2s   mean 1.18s   longest 1.99s
57 segments in 61.0s   mean 1.07s   longest 1.98s
60 segments in 62.0s   mean 1.03s   longest 2.00s
52 segments in 60.1s   mean 1.16s   longest 2.03s
57 segments in 60.3s   mean 1.06s   longest 1.97s
54 segments in 60.1s   mean 1.11s   longest 1.98s
55 segments in 60.2s   mean 1.09s   longest 1.96s

The ~2× outlier is what I noticed, but the mean is the more honest number: it ran 3 to 18 percent long the whole time. Whatever was happening was happening constantly, and one segment a minute was just the worst case of it.

The fix that only fixed one side

There was already a tolerance in the segment-cut test, and I had put it there. Segments used to come out in a coin-flip mixture of 1.00 and 2.00 seconds, decided by sub-millisecond timing, because the test was a bare elapsed >= 1.0 against keyframes arriving a hair under a second. So I widened it by one frame:

// keyframeArrivalTolerance = 1.0 / 30.0
if isKeyframe && elapsed >= targetDuration - keyframeArrivalTolerance {
    closeSegment()
}

That worked, and the name I gave the constant is what cost me the second bug. Tolerance and arrival describe absorbing jitter — a keyframe turning up slightly late. I had reasoned that the encoder property is an upper bound, so keyframes arrive at one second or a hair under, and a hair under is exactly what the tolerance absorbed. Having named the constant after one side of the distribution, I stopped thinking about the other one.

A keyframe arriving at 0.95 seconds misses that test by 17 milliseconds. No cut happens, and the segment stays open until the next keyframe — which, because the encoder's own interval restarted from the early keyframe, is about 1.95 seconds in.

What the documentation says, and where

Apple's page for kVTCompressionPropertyKey_MaxKeyFrameIntervalDuration, which is the property I was setting, says this in full:

The maximum duration from one key frame to the next in seconds.

Its discussion adds that the default is 0, meaning no limit, that it is useful when the frame rate is variable, that it can be combined with the frame-count version — and then, in a single cross-reference, "See kVTCompressionPropertyKey_MaxKeyFrameInterval for more discussion of key frames."

That link is where the answer was. The frame-count property's page says:

Video encoders are allowed to generate key frames more frequently if doing so results in more efficient compression.

So the behavior is documented — on a page about a different property, in a paragraph you reach by following a one-line "for more discussion" pointer from the page you were actually reading. I had read the page for the property I was setting, decided it told me what I needed, and not followed the link.

Efficient compression is exactly what a camera pointed at a moving toolhead under changing light provokes. Scene changes are constant, and the encoder obligingly spends a keyframe on each one. It is a ceiling, not a schedule.

Which clock? That part really isn't documented

There is a second question the documentation does not answer anywhere: duration in seconds of what. Wall-clock seconds, or seconds of the presentation timestamps you hand the encoder? Those diverge the moment your input frame rate isn't what you assumed, or a backpressure guard drops frames.

This is answerable in about forty lines and does not need the app, a device, or a simulator. Push frames through a real VTCompressionSession on a Mac as fast as the CPU allows, with the timestamps advancing at 1/30 of a second per frame, and see where the keyframes land:

fed 150 frames (5.00s of PTS at 30fps) in 0.130s of wall clock -> 38x real time
keyframes at input frame indices: [0, 30, 60, 90, 120]
keyframe gaps, in frames: [30, 30, 30, 30]
=> PTS-BASED

At 38 times real time the cadence did not move. The property counts the timestamps you supply and knows nothing about wall clock. That matters if, like me, you derived those timestamps from a frame counter and a nominal frame rate while the segmenter measured elapsed time some other way: the producer of keyframes was counting frames and the consumer of keyframes was counting seconds, and they agreed only while the camera delivered exactly the rate I had assumed.

Reach for a harness like that whenever the question is "what does this Apple API actually do" rather than "what does my app do with it." Minutes, against a build-install-observe cycle for the same answer.

Why the test rig couldn't see any of this

Every automated check I have kept passing the whole time. The app's demo mode is immune twice over, and both reasons are properties of synthetic video rather than of my code: a smooth generated animation has no scene changes to provoke an early keyframe, and its timestamps sit on an exact frame grid, so keyframes land at exactly 1.000 seconds and satisfy any test you write. The same is true of an ffmpeg test pattern, which is what my other rig used.

Smooth, synthetic, exactly-timed sources hide a whole class of encoder behavior. If the encoder's decisions are load-bearing for you, at some point you have to point it at real content.

The fix

State the rule as a floor rather than as a target minus an allowance. Cut on the first keyframe at or after a minimum segment duration:

// minimumSegmentDuration = 0.75
if isKeyframe && elapsed >= minimumSegmentDuration {
    closeSegment()
}

An early keyframe now closes a slightly short segment instead of leaving a double-length one. Short costs nothing here — the playlist's EXT-X-TARGETDURATION is 3 and the served window is measured in seconds — where a double-length segment eats into the window's depth.

Measured on the same camera during the same print, ten minutes apart, one variable changed. Before: 52–60 segments a minute, mean 1.03–1.18s, longest 1.96–2.03s. After:

62 segments in 60.4s   mean 0.97s   longest 1.02s
62 segments in 60.9s   mean 0.98s   longest 1.01s
61 segments in 60.2s   mean 0.99s   longest 1.06s
61 segments in 60.1s   mean 0.99s   longest 1.02s
61 segments in 60.1s   mean 0.99s   longest 1.02s

It also moved something I wasn't aiming at. The player's seekable margin — how much buffered video sits ahead of it — had been oscillating around zero on this camera, reading +1.1, -0.2, +0.4, -0.5, -0.8, -3.1 and so on. Afterwards it sat at a steady +2.1 to +2.7 seconds, with the playhead holding the same distance behind live. A window measured in seconds only holds the depth it intends if the segments are the length it assumes, and one double-length segment a minute had been quietly eating it. I am attributing that to this change rather than to conditions on the usual terms — same camera, same print, ten minutes apart, one variable moved — which is as clean as an A/B gets on live hardware and is still not a controlled experiment.

What I'd tell you to check first

If your keyframe-aligned segments are the wrong length and you're setting a max keyframe interval:

• Assume the encoder will beat your deadline. The interval is a ceiling. Scene changes and rate-control decisions both spend keyframes early, and real camera content supplies plenty of both.
• Check which clock your timestamps are on against which clock your segmenter is on. The encoder is on presentation timestamps. If your segmenter is on wall clock and your timestamps came from a frame counter, those agree only while the input rate is what you assumed.
• Stop asking and start telling. The version that ends this class of bug is kVTEncodeFrameOptionKey_ForceKeyFrame, driven from the segmenter's own clock, so the two cannot disagree at all. I widened a tolerance twice before writing that sentence down.
• Read the neighbouring documentation page. Half of this was published the whole time, one cross-reference away from the page I was on.

And the one I'd keep even if you never touch VideoToolbox: name a constant after the property it enforces, not after the failure it absorbs. keyframeArrivalTolerance described one tail of a two-sided distribution, and for as long as it was called that, the other tail was not something I thought to look for.