Apple has shared details of “Applebot,” a long-rumored web crawler first noticed last November. The company provided some details of the crawler in a recently updated support document.
According to Apple, Applebot is a crawler that primarily serves Siri and Spotlight. Both services offer web search capabilities, though until now it was thought that Apple routed queries through an existing search engine such as Google or Bing.
Applebot’s operation is similar to that of Google’s Googlebot crawler, and will follow Googlebot instructions if no Applebot-specific instructions are found. If website operators have any questions or concerns about the crawler, they can contact Apple’s network operations center.
It isn’t clear just how long the Applebot has been operating. However one developer spotted the crawler in November 2014, after noticing a number of hits from a crawler in the Apple-assigned 17.0.0.0/8 block of addresses.
