Apple has begun beta testing a new suite of artificial intelligence features for the Apple Watch, collectively branded as "Audio Intelligence." The rollout started with developer access through the iOS 27.3 update, following the features' debut at Apple's Surprise and Shine hardware event held in September. The package includes four opt-in capabilities designed for Apple Watch Series 12 users, covering sound detection, music recognition, real-time conversation replay, and automated note-taking from spoken discussions.

Two of the four features are relatively straightforward in scope. The first uses sound recognition to help users who are deaf or hard of hearing by alerting them to important environmental audio cues - such as emergency sirens, alarm systems, or doorbells - directly through their watch. The second connects to Apple's Shazam platform to identify songs playing in the surrounding environment, surfacing the track title and artist name on the watch display.

Conversation-Aware Features Raise Privacy Questions

The remaining two features are likely to attract more scrutiny, given the broader public debate around AI-powered ambient listening. Both Live Rewind and Siri Recaps involve the Apple Watch processing nearby human speech, placing Apple in territory that has proven contentious for other technology companies. Products like Meta's AI-integrated smart glasses have faced significant criticism over passive audio capture, and Apple appears to have anticipated similar concerns by building several layers of transparency and user control into its implementation.

Live Rewind is designed for moments when a user misses part of what someone has said. By double-pressing the digital crown on the Apple Watch, the wearer can retrieve a text snippet of the last 15 seconds of conversation, displayed directly on the watch face. The intended use cases include situations like receiving verbal directions, hearing a list of daily specials at a restaurant, or catching the name of a contact mentioned in passing. If the user takes no action, the snippet disappears permanently within 30 seconds. Alternatively, it can be saved to the standalone Siri app that shipped alongside iOS 27, making it accessible later for reference.

Apple has also built in an "Ask Siri" prompt that appears beneath the transcribed snippet, allowing users to follow up on anything they may not have understood - whether that is an unfamiliar term, a product name, or a food ingredient mentioned in passing. When the double-press gesture is performed, the watch emits an audible chime and displays a visual animation. Apple says this combination of gesture, sound, and on-screen indicator is intended to make it clear to anyone nearby that something has been captured.

Siri Recaps: Ambient Summarization With Guardrails

Siri Recaps takes the ambient audio processing concept further by generating structured summaries of conversations over a longer period. Apple has been deliberate in clarifying what the feature does not do: it does not identify individual speakers, retain raw audio recordings, or produce verbatim transcripts. Instead, it generates a brief written summary accompanied by bullet points that highlight key information from the conversation - such as important dates or deadlines, pricing details, or other factual elements that a user might want to remember.

Apple has framed the feature as a digital substitute for jotting down notes by hand, particularly in situations where doing so is impractical - such as during a hands-on training session, a work meeting, or a parent-teacher conference where sustained attention to the speaker takes priority. The company has also noted that Siri Recaps will not activate when the user is listening to music through headphones or has noise cancellation enabled.

At the conclusion of a session during which recaps have been generated, users receive a prompt to review what was captured and decide whether to keep any of it. Notes can be saved to Apple's Notes app or exported to the Journal app for more personal entries. If no action is taken, the recaps are automatically deleted within seven days. This review prompt also appears when the user first unlocks their Apple Watch each morning, serving as a reminder to process any outstanding summaries.

Users have several options for managing when Siri Recaps runs. It can be enabled manually on demand, or configured to operate on a set schedule - for example, only during work hours - while remaining off at night. Even when a schedule is active, the feature can be disabled at any moment from the Control Center. Apple has also stated that Siri Recaps is built to automatically filter out sensitive categories of information, including government-issued identification numbers, financial data, authentication credentials, certain personal identifiers, and any content that could be associated with self-harm.

Hardware and Privacy Architecture Behind the Features

Audio Intelligence on the Apple Watch Series 12 and Apple Watch Ultra 4 draws on a combination of on-device hardware, software processing, and cloud infrastructure. The S11 chip inside the watch, along with its built-in microphone, works in conjunction with Apple Intelligence models running on the paired iPhone and Apple's Private Cloud Compute (PCC) service.

Central to the privacy design is the watch's always-on processor, which includes a secure enclave - a hardware-isolated memory space that is inaccessible to the broader operating system. A dedicated audio buffer within this enclave handles the raw audio signal in a manner that prevents it from being accessed by any external party, including Apple itself. The audio is only passed to the paired iPhone through an encrypted, secure pairing channel between the two devices.

The processing pipeline works as follows: lightweight AI models running on the S11 chip first determine whether the wearer is participating in an active conversation. If so, the relevant audio is transferred to the secure enclave on the iPhone, where on-device processing converts it into condensed text - essentially the substance of the conversation with filler words and verbal noise stripped out. This condensed text is then encrypted and sent to Apple's Private Cloud Compute, where the actual recap or summary is generated and returned to the device. Apple says that once the recap is delivered, both the summarization data and any intermediate outputs used to create it are immediately deleted from PCC servers.

The resulting recaps stored on the user's iPhone are accessible only to that user. Apple does not retain access to them. When synced across multiple Apple devices, the recaps remain encrypted throughout the transfer. The same general pipeline - transferring audio from the watch to the iPhone for processing - also underpins the Live Rewind feature and its 15-second text snippets.

Battery Impact and Rollout Timeline

Apple has stated that none of the Audio Intelligence features are expected to have a measurable impact on the battery life of either the Apple Watch or the paired iPhone, though the company has not provided specific technical data to support that claim at this stage of testing.

The features are currently available to registered developers through the iOS 27.3 beta. Apple plans to extend access to a broader public beta cohort before eventually making the full Audio Intelligence feature set available to all compatible users in a general software release.