Apple unveils Siri AI: a contextual assistant built to act across apps

For some time now, we have been covering the key moves in the race for dominance in general-purpose AI. It is a market featuring names like Gemini, ChatGPT, Claude, Grok… in which Apple, however, had been falling behind. Now, though, it seems the company led by Tim Cook has decided to shift its strategy, unveiling its proposal for users of the Apple brand. The new Apple Intelligence has arrived and, with it, the new Siri AI.

Apple has introduced the next generation of Apple Intelligence, powered by “a new groundbreaking architecture” that promises to help create smarter, more useful apps, with new features ranging from intelligent photo editing and tools to personalize browsing in Safari, to the ability to strengthen security protections with Passwords and an entirely new version of Image Playground, among others.

And with them, Siri AI arrives.

What the new Siri AI is and how it works

To understand the real impact of this announcement, we first need to clarify the system architecture. Apple Intelligence acts as the invisible engine, based on the latest Foundation Models custom-developed in collaboration with partners such as Google and… yes, its Gemini models. One of the most visible faces of this development is Siri AI, which is no longer just a voice assistant but rather the interface for an intelligent agent capable of seeing your screen, understanding your personal context, and carrying out complex actions by connecting data across multiple apps.

In addition, Siri AI arrives alongside an exclusive app so users can pick up past conversations across any of their devices, expands its visual intelligence capabilities, and integrates writing tools. As for the purely conversational side, this new Siri AI offers “more expressive” voices and improvements in dictation accuracy across the system. Users can therefore adjust Siri’s level of expressiveness and speaking pace to suit their preferences.

To give a few examples:

• If you are looking at a photo of a plate of food, you can activate Siri AI and say, “How many calories does this have?” Siri AI will look at the screen, and Apple Intelligence will process the image to give you the answer.
• You can ask Siri AI something like: “Send last weekend’s photos to Juan”. Siri AI will use Apple Intelligence’s capabilities to search your gallery, identify who Juan is in your messages, and automatically send them.
• Siri AI will no longer just answer generic internet questions, but will use the knowledge Apple Intelligence has about your day-to-day life (your emails, messages, notes, and calendar) to help you solve specific problems in your life.
• If you receive a message about a meal with friends, you can ask Siri for ideas on what to bring and add a recipe to the Notes app

As Craig Federighi explained, Apple’s senior vice president of Software Engineering, “with access to broad world knowledge to deliver up-to-date answers on virtually any topic, in addition to screen analysis and personal context understanding, Siri AI can help users take action in apps more naturally than ever before.”

New smart features

But Apple’s applied AI updates are not limited to the new Siri.

One of the most striking applications of this shift can be found in photography. The Photos app will introduce groundbreaking features such as Spatial Reframing, a system inherited from Apple Vision Pro’s spatial technology that makes it possible to modify an image’s perspective after it has been captured, with AI generating the remaining background. This is joined by the Extend tool, designed to widen frames or correct horizons without cropping the subject, along with a more precise version of the Clean Up tool. That said, every edited or generated image will include an invisible watermark through SynthID.

Productivity and web browsing in Safari will also get a boost. Thanks to Apple Intelligence, the browser can now automatically classify and group dozens of tabs by specific topics. In addition, the new Notify Me feature allows users to ask Safari to monitor web pages, alerting them in real time if, for example, there are relevant price changes in products they are interested in. Finally, with the update to the Passwords app, the system can browse autonomously and securely in the background to replace weak passwords with high-security alternatives in a single tap.

As is often the case with this company, one of the major differentiators in Apple’s AI approach compared with OpenAI or Google lies in its commitment to privacy. Through Private Cloud Compute, any data processing that cannot be carried out directly on the device will be sent to ultra-secure servers. Apple guarantees that this data is not stored and cannot be accessed by anyone.

Apple faces a regulatory issue in the EU

Buuuuut not everything is quite so rosy. Despite the enthusiasm with which these new features have been received, there are a few things you should know, especially if you live in Europe. The public beta will arrive this summer and the official launch will take place in the fall, but strict local regulations will mean that Siri AI will not initially be available on iOS, iPadOS, and watchOS devices within the European Union.

European users will be able to use it on Mac and Apple Vision Pro as long as they set a supported language, while Apple continues negotiating to unlock access on its mobile devices.

Image: Apple and Gemini