Apple has revealed a slew of new innovations for its hardware with a strong push for accessibility, features that will be implemented via software updates later in 2022.
New software features developed using machine learning include door detection for users who are blind or low-vision, and a real-time captioning system for those who are deaf or hearing impaired, and Apple Watch Mirroring for those with physical and visual impairments, movement disorders, and smartwatch control via iPhone Ability.
Let’s take a deeper look at these features, starting with door detection, which, as the name suggests, allows iPhone and iPad users to locate doors when they arrive at a new place.
The feature uses lidar, so requires a device with a lidar scanner (iPhone 12 and iPhone 13 – Pro and Pro Max – phones, and various iPad Pro models), built into the Magnifier app. It can determine whether a door is open or closed, if the latter, how it opened, and the user’s distance from the door, plus it can read any signs or characters on the door.
The magnifying glass will gain a new detection mode that will play the role of door detection and will also provide features like person detection and image description.
People who are deaf or hard of hearing can access Live Captions on iPhone, iPad, and Mac computers, allowing captions to be generated on-device for everything from video chats to watching streaming content. In FaceTime, captions are automatically attributed to the person in the call, while on Mac users have the option to type in a reply and have them read it aloud in real time.
Note on device support that only supports Macs with an Apple chip, or you need an iPhone 11 or later, or for an iPad, models with an A12 Bionic chip. Initially, Live Captions will debut in beta form (and thus still in testing — Apple observes that the accuracy of captions “may vary”) only in English.
The last major accessibility revelation Apple made here was the introduction of Apple Watch Mirroring, which allows people to use their iPhone to control their watch. In other words, users can benefit from smartphone accessibility features such as voice control and switch controls when interacting with the Apple Watch, voice commands to turn on the watch, head tracking, and more.
The new Quick Actions for Apple Watch also allow users to control with simple gestures, such as answering (or ending) a call with a double-pinch gesture.
Note that you need an Apple Watch Series 6 or newer to benefit from the mirroring feature.
>>>>>>>>>>>Apple battery