Apple on Thursday announced that iOS 15 and iPadOS 15 will include a feature that uses a new application of cryptography to help limit the spread of Child Sexual Abuse Material (CSAM) online while designing for user privacy. CSAM detection will help Apple provide valuable information to law enforcement on collections of CSAM in iCloud Photos.
Devices will download a database of known CSAM image hashes (the database is unreadable to users) and will do an on-device comparison to a user’s photos, flagging them for CSAM material before uploading them to iCloud Photos.
While CSAM image scanning is not an optional feature and will happen automatically, but Apple confirmed to MacRumors that it cannot detect known CSAM images if the iCloud Photos feature is turned off.
Apple will identify a known CSAM photo on the device and then flag it when it’s uploaded to iCloud Photos with an attached voucher. Once a certain number of flagged photos have been uploaded to iCloud Photos, Apple will interpret the vouchers and do a manual review. If CSAM content is found, the user’s account will be disabled and the National Center for Missing and Exploited Children will be notified.
Since Apple is scanning iCloud Photos for the CSAM flags, the feature will not work with iCloud Photos disabled. Apple also confirmed that it cannot detect known CSAM images in iCloud Backups if iCloud Photos is disabled on a user’s device.
While Apple is scanning only for hashes of known child sexual abuse materials and is not broadly inspecting a user’s photo library, the announcement has raised concerns among privacy experts who worry that the technology could be used in the future to detect other types of content, which may have political or safety implications. However, users that have privacy concerns can turn off the iCloud Photos option.