A recent blog post by developer Jeff Johnson has shed light on a feature in Apple’s Photos app for iOS 18 called “Enhanced Visual Search.” This feature allows your iPhone to send photo data to Apple to enable identification of landmarks within your photo library. However, it’s worth noting that the setting is enabled by default and was not prominently announced by Apple or widely covered in discussions about the latest iPhone operating system.
What is Enhanced Visual Search?
Enhanced Visual Search functions similarly to Apple’s existing “Visual Look Up” feature, which leverages machine learning to identify objects such as plants and animals in your photos. To use the feature, users can swipe up on a photo containing a landmark or building and select “Look Up Landmark.” The app then attempts to identify the location or structure in the image.
In practice, Enhanced Visual Search makes it easier to recognize and search for specific landmarks captured in your photos, streamlining organization and exploration within your photo library.
How It Works
According to Apple’s description under the toggle for Enhanced Visual Search, the feature “privately matches places in your photos.” However, Johnson points out that the explanation is somewhat ambiguous. The process reportedly involves your phone generating vector embeddings for parts of an image containing a landmark. These embeddings, along with several “fake queries,” are sent to Apple’s servers for analysis. Apple then returns a set of potential matches, from which the phone selects the most accurate result.
While this approach is intended to maintain privacy by obfuscating user data, Johnson raises concerns about whether it truly upholds Apple’s privacy standards.
Privacy Concerns
Johnson referenced Apple’s famous “What happens on your iPhone, stays on your iPhone” slogan, which debuted on a billboard at CES 2019, to highlight potential discrepancies between the company’s claims and this feature’s functionality. He argues that the mere act of sending data from your device to Apple undermines the concept of complete privacy.
He explained:
“If something happens entirely on my computer, then it’s private. But if my computer sends data to the manufacturer, it’s not private—or at least not entirely private. A software bug alone could compromise user data, making privacy claims questionable.”
This observation suggests that even if Apple has no malicious intent or security flaws in its system, the potential for unintended vulnerabilities exists, raising concerns about the feature’s implications for user privacy.
Apple’s Response
As of now, Apple has not commented on Johnson’s blog post or the concerns raised regarding Enhanced Visual Search. The lack of transparency around the feature has left some users questioning how much control they truly have over their data.
Locating the Enhanced Visual Search Setting
If you want to review or adjust this setting, it can be found under Settings > Apps > Photos on iOS and iPadOS devices. On a Mac, the option is located under the “Search” heading within Photos > Settings.
Implications for Users
The introduction of Enhanced Visual Search exemplifies how Apple continues to expand its use of artificial intelligence to improve user experiences. Features like this provide convenience, such as quickly identifying landmarks or organizing photos more effectively. However, it also highlights the delicate balance between functionality and privacy.
Apple has built its brand around user privacy, often touting it as a cornerstone of its product offerings. Yet, as Johnson points out, features requiring data exchange—even with safeguards in place—challenge the company’s promises of keeping all data securely on the device.
While Enhanced Visual Search offers an intriguing way to use AI for photo organization and landmark identification, it’s not without controversy. Users should be aware of its default activation and consider whether the convenience outweighs potential privacy concerns. By reviewing their settings, users can make more informed decisions about how their data is used and shared.
Ultimately, this feature serves as a reminder of the trade-offs involved in leveraging advanced AI-driven tools. As privacy becomes an increasingly critical concern, users will likely scrutinize such innovations to ensure they align with their expectations and comfort levels regarding data security.