We’ve long suspected that our devices are listening to us, but what about when we invite them to do it? That’s what testing Audio Intelligence has felt like on the Apple Watch. But despite how disturbing it may sound to have your smartwatch listening and taking notes of your entire day, the reality has been a lot more mundane.
Along with the general sensor, processor, and battery upgrades, the Apple Watch Series 12 and Apple Watch Ultra 4 introduced Audio Intelligence, a new set of features that allows the watch to listen in on conversations.
Siri Recap generates summaries of what you’ve discussed throughout the day, while Live Rewind gives you a transcript of the last 15 seconds of audio in case you missed something.
It’s similar functionality as AI-powered recording pins like Plaud or Amazon’s Bee, but within the context of your smartwatch and without the additional cost or subscription.
And it’s completely optional. You have to first opt-in to the features to activate them, then choose to turn the feature you want on whenever you want to start using it. Then you can leave it on all day, set it on a schedule or toggle it manually whenever you need it. If you’re like me and forget that it’s on, you get reminders that it’s still listening or to check your recaps.
The fact that you have so much control over whether you use it and how much helped ease the creepy factor for me, but there’s still everyone else to consider.
Beyond a barely audible chime when I double-click the Digital Crown to activate Live Rewind, there’s no way to alert others that the feature is in use and that whoever’s in earshot is inadvertently getting roped into my little experiment as well. Other wearables like smart glasses have an indicator light on the front to alert spectators when they’re recording video.
Apple has built-in privacy safeguards for the audio, how it’s processed and then deleted. I’ve broken all that down in a separate Audio Intelligence FAQ. This is about what it’s like to use it in the wild.
I’ve been testing Audio Intelligence in the WatchOS 27.3 (and iOS) public beta for two weeks. Because this isn’t the final version (Apple says it will release publicly later this year), consider this a sneak preview.
Live Rewind: Late to the trigger
Let’s get Live Rewind out of the way because there’s not a lot to say about it. Whether from a lack of muscle memory (or usefulness), I’ve had to go out of my way to remember the feature exists.
I figured it’d come in handy for those “come again?” moments during conversations, but the reality is that it’s just easier to ask someone to repeat what they said than remember to double-press the Digital Crown at just the right moment. I can see it being more useful when you’re listening to a presentation or other event.
Fifteen seconds go by a lot faster than I expect and by the time I realize something might be worth revisiting, the person has moved on with the conversation. I either missed what I intended to get or just captured the silence after.
Siri Recap: Good at meetings, bad at life
Siri Recap has been more useful, though it’s still hit-or-miss. The feature gives you high-level summaries of what it heard — it doesn’t provide a transcript to review, and it doesn’t identify who said what. It shines brightest during dedicated meetings when there’s a single conversation to follow rather than a jumble of unrelated discussions.
One success story: I was moderating the chat for our iPhone Duo live folding test on YouTube while Matthew Murray, CNET’s lab director, answered viewer questions on the stream.
I thought I’d caught the gist of what he was saying, but the typing must’ve distracted me more than I realized because I was pleasantly surprised to learn some interesting facts that I’d missed in the Siri Recap, like “Opening a phone 20 times daily over 10 years equals 73,000 folds” or that the “machine runs approximately 22,560 cycles every 24 hours”.
But Siri’s batting average wasn’t always that good, which made it challenging to trust the notes. During a conversation with another mom about car-pool logistics, Siri Recap pulled out action items, like who I was picking up and at what time, which I appreciated, but the information wasn’t accurate. Not only had Siri gotten the names of the kids wrong, it also had me picking up the wrong kid. Clearly, the mental load of figuring out after-school logistics is challenging even for Siri.
Siri can’t handle the messy conversations (and TV shows)
Where things get interesting (and occasionally hilarious) is how Siri discerns what constitutes a conversation and how many topics belong in a single recap.
Siri is generally good at accurately labeling what conversations are about: “Parent-teacher conferences,” “Car-pool logistics,” “Folding phone durability testing.” But rather than keeping them separate, as it should, it sometimes conflates two or even three completely unrelated conversations into a single recap.
I encountered bullet points summarizing a serious conversation with a friend about a medical condition alongside notes about a chaotic (albeit funny) exploding apple cider incident with my kids. The two conversations happened within the same two-hour window, but they were by no means back-to-back.
Siri’s version:
- Details about medical condition
- Entire case of apple cider ruined after repeated warnings to stop shaking
- No more apple cider will be brought into the house
- Any remaining stickiness on the floor results in no iPad on the plane
It’s a good thing Siri Recap leaves out tone and volume cause that last bullet would’ve been in ALL CAPS.
What it did decide was worthy of a standalone recap was a discussion about wedding plans during an episode of Nobody Wants This on Netflix. To be fair, this would’ve absolutely warranted a recap if I’d been discussing my own wedding plans; however, the fact that the “conversation” happened at midnight should’ve tipped off Siri that this was a late-night binge session and not an actual conversation needing to be recapped.
Rather than coming away better informed about the important conversations I’d had throughout the day, I came away feeling validated by the sheer chaos of it all. The constant whiplash of being pulled in a dozen different directions — bouncing between work, kids, gossip and the occasional sticky beverage crisis — was real and now immortalized (until the recap automatically expires if you don’t choose to save it).
A case for manually enabling Siri Recap during meetings
For now, while novel, neither Siri Recap nor Live Rewind is reason enough to buy (or avoid) the Apple Watch Series 12 or Ultra 4. They’re two of our top-rated smartwatches for their fitness, health and other smart features. Audio Intelligence is either an added bonus (it’s free) or something you can ignore altogether.
But if you do decide to venture into this brave new world of AI notetakers, my suggestion would be to use Siri Recap intentionally rather than leaving it running all day.
Turn it on before a work meeting, a doctor’s appointment or another conversation where you might want a few notes to jog your memory later. Just make sure everyone involved knows it’s enabled and is comfortable with it, and that you’re not relying on it for crucial information.
If the stakes are high, have a backup — whether that’s your own notes or a recording — lest you risk coming home with the wrong kid.
Vanessa Hand Orellana
