Apple unveils new accessibility features that rely heavily on artificial intelligence

We’ve all used or tried some of the accessibility features built into iOS, whether it’s flashing a flash when a call comes in, or double or triple tapping on the back of the iPhone to bring up an app or setting. But these are the tip of a giant iceberg, creating accessibility settings.

A few days before World Accessibility Awareness Day, which will be held on Thursday, May 19, Apple has just announced a series of new features for people with disabilities. These won’t be launched until a little later in the year, but the California giant still details a few of them in a press release.

(Right) Take the door

The first of these is called door detection and is for people who are blind or partially sighted. Its purpose is to allow the user to find the right door when he comes across an environment which is unknown to him. Thus, the iPhone on which the function is enabled will indicate how far the user is from the door, and will be able to describe it by contacting a possible number or symbol without even indicating whether it is open. On or off. The iPhone door will also specify the type of handle it is equipped with and will indicate to the user whether to push or pull to open it.

To provide this information, the iPhone (or iPad) uses lidar and camera modules, then the data is processed locally by a machine learning algorithm.

This function will be enabled in the detection mode in the Magnifier application. This will be added to the People Detection function, already available, and the image description. To make it easier for the visually impaired to navigate the outside world, Apple Maps will add sound and haptic rendering for voiceover, specifically to let users know which way to go when exiting the subway, for example.

On-the-fly captions for iPhone, iPad and Mac

Another big innovation announced by Apple is Live Captions, a function that lets you caption any content, including audio tracks. This can be a way to find out what your interlocutors are saying during a facetime call, during a video conference, on a social media application or while you’re watching a streaming video – even if most platforms now offer their own solutions. On Mac, during chat, it will be possible to type an answer on the keyboard and read it “aloud” by the machine.

This information is always managed locally and does not go online through Apple or partner’s servers.

The only small regret is that the live caption will not be available in beta until later this year, and at first only in English. You’ll need an iPhone 11 or newer, an A12 Bionic SoC or an iPad or an Apple Silicon Mac.

Also watch the video:

A mirrored clock.

Also, Apple has unveiled a new function that allows a motor user with a motor problem to display his watch screen on his iPhone. He can then control his watch by voice from his smartphone, in particular, or through a device to monitor head movements.

According to Apple, this is a way to allow people to access the watch’s improved health functions, such as heart rate monitoring, or blood oxygen levels.

Also, Cupertino giant engineers will launch new fast action on the watch. This will make it possible to answer a call or hang up, turn off a notification, or perform a double pinch on the screen to read a song. A function that fans can find among all users.

Finally, Apple has listed other innovations that are less important, but which could change the lives of people with disabilities. Buddy controller This will allow one player to request help from another player to combine instructions from two different controllers for a single player game.

Once requested, it will also be possible to adjust Siri’s timeout so that the assistant waits longer or shorter. In the end, Word recognitionFeatured in the past, the goal is to alert the user when the doorbell rings or the fire alarm goes off.

It is quite possible that Tim Cook’s teams will return to these announcements during the inaugural conference of WWDC 2022 on June 6, which will be held primarily online.

Source: Apples

Leave a Comment