After the announcement of the iPhone 16 models, the collaboration between Apple and OpenAI is starting to make more sense to me.
Leading up to WWDC 24 in June, rumors circulated about a deal between Apple and OpenAI. Some believed Apple would either replace Siri completely or power it with ChatGPT. However, during WWDC 2024, Apple officially announced they've been developing their own fleet of GenAI tools for various on-device AI applications, including improving Siri.
Apple introduced "Apple Intelligence" as their own AI implementation. For the most part, it would run completely on-device for Apple Silicon-powered devices. More intensive AI requests would be offloaded to "Private Cloud Compute" - essentially Apple's own silicon-based servers dedicated to handling heavier AI tasks while staying within Apple's privacy-focused ecosystem.
Towards the end of WWDC, almost like it was an afterthought, Apple revealed the heavily speculated partnership with OpenAI. Essentially, it would allow users to have Siri access ChatGPT for "world knowledge" - information that Apple Intelligence might not have. Initially, this seemed puzzling. Why would Apple need this when they're developing their own AI? Couldn't OpenAI just use the Siri API for integration like every other developer?
The announcement of this partnership sparked concerns in the online community. Many worried about Apple potentially sharing users' private data with OpenAI, seemingly forgetting the prior 2 hours Apple spent emphasizing privacy. While these concerns are understandable given the sensitive nature of AI and data, they overlook Apple's consistent stance on privacy and the rest of the event.
Apple has repeatedly demonstrated their commitment to user privacy. Their approach sets an industry standard that other major players should emulate. In the months following WWDC, Apple detailed their AI strategy, emphasizing that processing would happen on-device or via their secure private cloud. Users could access ChatGPT through Siri without compromising their data or privacy.
Still, the benefits of the OpenAI partnership didn't make sense, maybe because Apple Intelligence is already delayed, and it seems unlikely that the new Siri would quickly catch up to ChatGPT's capabilities.
Then came the iPhone 16 announcement. Not only will all new models support Apple Intelligence, but they will also feature a new "Camera Control" button. This smart button can detect pressure, mispresses, and swipes, acting differently in various scenarios. So another question arises: why did Apple decide that now is the time to add a dedicated camera button after 15+ years of iPhone?
The answer to both questions lies in a newly revealed AI feature exclusive to iPhone 16: Visual Intelligence. This groundbreaking capability allows users to interact with their surroundings through the camera, similar to Google Lens but deeply integrated into the iPhone's ecosystem. With Visual Intelligence, users can use the camera to pass data directly to AI without the need to prompt. It seems like the sky is the limit with the potential use cases here; some examples are the ability to identify dog breeds, add event details to their calendar by scanning posters, get restaurant information by pointing at storefronts, or even shop for products they see in the real world. This feature combines on-device intelligence with Apple's cloud services to analyze images without storing them, maintaining user privacy.
The partnership with OpenAI enhances this capability. By leveraging ChatGPT's advanced multimodal abilities, Apple can offer a more comprehensive Visual Intelligence experience with ChatGPT's "world knowledge" while continuing to develop their own technology. This collaboration allows Apple to provide cutting-edge AI features on their hardware while maintaining their commitment to privacy.
Users can trigger the camera control with this button and interact directly with Apple Intelligence and ChatGPT (and others) about their surroundings. This offers a new way to interface with AI beyond just text or voice. A button to quickly give AI eyes... I'm very excited about this (and also a bit scared).
To me, the OpenAI collaboration finally makes sense. The Camera Control button isn't just for photos or videos; it's a gateway to AI vision. By partnering with OpenAI, Apple can offer a cutting-edge Visual Intelligence experience while they continue developing their own technology. It's a win-win: Apple provides an advanced feature on some of the best hardware in the world, and users get the most powerful visual AI within the familiar and privacy-centric iOS environment.
This iPhone event felt more like a preview of Apple's AI future than a traditional product launch (even though Apple Intelligence won't even be available at launch). While the changes to the iPhone might seem subtle, the AI capabilities paired with a tangible hardware experience represent a significant leap forward in integrating AI into our daily lives. Apple's approach remains characteristically privacy-focused, tightly integrated with hardware, and strategically partnered to fill its capability gaps.
The collaboration with OpenAI, especially for Visual Intelligence, is more nuanced than it first appeared. It allows Apple to offer advanced AI features while maintaining their privacy-first approach and buying time to develop comparable technologies. I believe that for better or for worse, this new method of calling AI could redefine how we interact with our devices and the world.
The Apple way of doing AI might be fashionably late, but it has the potential to strike a unique balance between innovation and privacy. In the end, I believe in the Apple way, most of the time. Once we experience Apple Intelligence, there might be no going back.
Comments