Picture in Picture kept freezing, and it was the audio session

I have an app, AmbientCast, that puts a 3D printer's camera feed into a Picture in Picture window on an Apple TV, so you can keep an eye on a print in the corner of the screen while something else plays. For weeks that window kept freezing — the picture stopped, the app behind it was fine, and whatever was playing in the foreground stalled alongside it. Killing my app was the only thing that fixed either, at the time.

The symptom was entirely in video. The cause was the audio session.

Everything I replaced, and what it changed

The PiP window is fed by a local pipeline: the camera feed is decoded, a status overlay is composited onto it, the result is re-encoded and served as HLS from a small server inside the app, and a player reads that back to give the system something to put in the window. That is a lot of parts, and over several evenings on a physical Apple TV 4K (3rd generation) on tvOS 27 I replaced most of them. Each of these was a real change, installed and measured on hardware, and none of them moved the freeze:

Thirteen things. Every one of them on the video side. That is not an accident of what I happened to try — the symptom was a frozen video window, so every hypothesis I formed was about video, and I went on forming them for weeks.

The trigger was a transition, not a state

The first thing that actually moved was noticing that the freeze only ever followed a source switch — changing which printer the dashboard is showing. Every experiment above had tested the pipeline in steady state, and steady state was fine. Once I tested across the switch instead, it reproduced on demand:

PiP up, no switch        9.5 minutes, zero freezes
after a source switch    froze at 46s
after a source switch    froze at 68s
after a source switch    froze at 43s

One thing there cost me several trials before I caught it: the degraded state survives killing the app. The first launch after a reboot ran clean, and every launch after a source switch froze in about the same 45 seconds regardless of which process it was. So any trial run after a reproduction is measuring a poisoned machine, not your change. Reboot between trials. And note that devicectl device process launch --terminate-existing is a SIGKILL that runs no cleanup at all, which means it bypasses the very teardown path I was trying to test.

Why a silent video app holds an audio session at all

Picture in Picture is not a video feature as far as the system is concerned. It is part of the same capability as background audio and AirPlay: to get it you declare the audio background mode and configure a .playback audio session, which is the setup Apple's Configuring your app for media playback describes.

ELI5: what's an audio session?

A device has one set of speakers and many apps that might want them. An audio session is your app's declaration of how it intends to use audio — whether it should keep playing when the screen locks, and crucially whether it plays alongside other apps or takes over from them. The system arbitrates between everyone's declarations.

ELI5: what's a background mode?

iOS and tvOS normally freeze an app the moment it leaves the screen. A background mode is a permission you declare in the app's configuration saying you need to keep doing a specific kind of work anyway. They are narrow, App Review looks at them, and "audio" is the one that covers Picture in Picture.

Three things in my app existed for no reason other than keeping PiP alive:

ELI5: why would a silent app ship an audio track?

The video here is generated on the fly and has no sound. But the parts of the system that hand out Picture in Picture windows expect a normal piece of media, and a normal piece of media has an audio track — without one the player never finished loading the stream, so there was nothing to put in a window. So the stream carries one containing silence — real audio data that happens to be inaudible.

Put together: as far as the rest of the system was concerned, my app was an app playing audio, through a session that interrupts everyone else's. And a source switch tears that whole stack down and builds a new one, which re-claims the session in the middle of whatever else is playing.

The one-line change I had spent weeks not making

// before
try AVAudioSession.sharedInstance()
    .setCategory(.playback, mode: .moviePlayback)

// after
try AVAudioSession.sharedInstance()
    .setCategory(.playback, mode: .moviePlayback, options: [.mixWithOthers])

The reason it took weeks is the reason it is worth writing down. Every one of those three deliberate decisions exists to keep Picture in Picture working, and PiP is the feature this app exists for. .mixWithOthers looked exactly like the sort of change that would quietly persuade the system I wasn't really playing audio and stop handing me a window — and losing PiP to fix an interruption would have been the worst available trade. I could not find that question answered anywhere: not in the AVFoundation documentation, not in the forums, not in anyone's write-up.

So I built the variant and looked at it. With the mixable session, on the same hardware:

audio: session category .playback + .mixWithOthers
pip:   didStartPictureInPicture
pip:   didStartPictureInPicture
hls:   61 segments, served 101 requests

PiP started on demand, twice, with the pipeline running at its normal rate. That is one app on one device, and the device is an Apple TV — the setCategory call is shared with my iOS target, but I have not re-measured PiP on an iPhone since the change. It is not a guarantee from Apple either — but it is one more data point than I could find anywhere, so I will state it plainly: adding .mixWithOthers did not cost me Picture in Picture on tvOS.

What it fixed

Before the change, the window froze within about a minute of every source switch, without exception. After it, on the same Apple TV with PiP up and a video app playing alongside: 36 minutes clean after one source switch and 4 more after a second, zero freezes, and the local server serving its usual 101 to 103 requests a minute throughout. The app playing in the foreground dropped 0 of 13,148 frames.

The documentation said so the whole time

From Apple's page on the .playback category:

By default, using this category implies that your app’s audio is nonmixable—activating your session will interrupt any other audio sessions which are also nonmixable.

That sentence has been there the whole time, and I had read the page. What I never did was connect it to a video app, because my app makes no sound that anyone can hear. The track in those segments is silence. The system does not care whether anyone can hear it.

What I'd tell you to check first

If your symptom crosses into another process, it is a message about something shared. Another app stalling, or recovering when you kill yours, is not a report about your video objects — none of mine ever helped. Audio is usually the only system-wide claim a media app holds, and it was the last thing I looked at rather than the first.

The visible subsystem recruits every hypothesis. A frozen video window generated thirteen experiments about video. Ask early what else the feature depends on, and if the answer includes a capability you declare in UIBackgroundModes, start there.

Read back what the session actually is, at the moment of the symptom. Not what you set at launch — plenty of things reset the category behind you, including AVKit, capture sessions and third-party SDKs, and the last writer wins:

let session = AVAudioSession.sharedInstance()
print("audio: \(session.category.rawValue) opts \(session.categoryOptions.rawValue)")

And test the transition, not the steady state. A pipeline that is healthy while it runs tells you nothing about what happens when it is torn down and rebuilt. Mine was healthy for 9.5 minutes at a stretch and broken 46 seconds after I touched it.