On Monday, June 10, 2024, Apple held its annual Worldwide Developers Conference keynote address. Leading up to this event, there was considerable speculation regarding Apple’s potential foray into Artificial Intelligence (AI), as well as anticipated updates to Siri, Apple’s virtual assistant.
This article does not cover every update. For the complete keynote address, visit the Apple WWDC 2024 Keynote on YouTube.
Catch up on Accessibility in SwiftUI
Before we discuss the Apple Keynote, lets take a look at the accessibility session for developers at this year's WWDC. You can watch this session here and watch many other developer sessions here.
This session was led by Tommy, an engineer on Apple’s Accessibility Team. He began by stating, “Accessibility is a fundamental part of any great app, allowing everyone to create, connect, and play with the experience you build.” He demonstrated how accessibility features are integrated by creating an app for visiting different beaches.
Tommy explained, “SwiftUI creates Accessibility Elements as one of its primary outputs. Accessibility elements are the fundamental building blocks that assistive technology programs such as VoiceOver, Voice Control, and Switch Control use to present and interact with content in your app.” He emphasized that assistive technologies like VoiceOver interact with apps solely through their elements, meaning only content from views included in an element will be accessible.
He demonstrated his app, showing which controls were accessible and how to fix those that were not. Since his app uses Drag and Drop, he also showed how to make this feature accessible.
Tommy encouraged developers to use assistive technology like VoiceOver to test their apps, advising, “Developers should investigate the Accessibility APIs to make their app better.” Apple provides extensive written accessibility information for developers. The Vision section on accessibility is available here. The Vision section covers VoiceOver and screen magnification, and the “Supporting VoiceOver in Your App” section includes links and examples for making apps accessible. On the main Vision page, there is a VoiceOver Maze Challenge, where developers must navigate a maze using only VoiceOver.
Apple Intelligence
Tim Cook commenced the keynote with the announcement of Apple’s new AI, stating, “Introducing Apple Intelligence: the new personal intelligence system that makes your most personal products even more useful and delightful.”
Cook then passed the presentation to Craig Federighi, Apple’s Senior Vice President for Software Engineering. Federighi elaborated, “With iOS 18, iPadOS 18, and macOS Sequoia, we are embarking on a new journey to bring you intelligence that understands you. Apple Intelligence is the personal intelligence system that places powerful generative models at the core of your iPhone, iPad, and Mac.”
He continued, “It leverages your personal context to provide intelligence that’s most helpful and relevant for you. It safeguards your privacy at every step and is deeply integrated into our platforms and the apps you rely on to communicate, work, and express yourself.”
Federighi then detailed some of the features of Apple Intelligence. The iPhone will prioritize notifications, offer new writing tools accessible system-wide, and introduce creative features for images such as photos, emojis, and GIFs. Emails can be categorized by topics such as personal, newsletters, and marketing. Apple Intelligence processes relevant data to assist you. Phone calls can be recorded, with both a transcript and the recording available. Participants on the call will be notified when recording begins.
Apple Intelligence will be available across various apps including Messages, Mail, and Photos. Additionally, Apple has partnered with OpenAI to integrate ChatGPT 4.0 into Apple Intelligence, available on supported iPhones, iPads, and Macs at no extra cost. If you ask Siri a question, you will be given the option to use ChatGPT, enabling inquiries about documents, photos, and more. ChatGPT is also integrated into the writing tools and will be released with iOS 18, iPadOS 18, and macOS Sequoia later this year.
When a request is made, Apple Intelligence will determine whether it can be processed on your device or if greater computational capacity is required. Only the pertinent data will use Private Cloud Compute, sending the data to Apple’s private silicon servers without storing your data on the server.
Siri
You will no longer need to say “Siri” or “Hey Siri” to ask another question related to your topic. Siri will become more contextual and personal, allowing you to type a question or switch between typing and talking seamlessly.
Siri will guide you through tasks on your device, even if you don't know the exact terminology. Simply describe what you want to do, and Siri will provide the solution.
Additional features are set to arrive next year, including on-screen awareness, which enables Siri to interact with items on your screen. For example, you can ask Siri to add a new address to a contact card or perform tasks within apps, such as showing a specific photo. Siri will understand and act on requests based on your personal context.
Writing
Apple Intelligence will offer a suite of writing tools. For instance, the Rewrite tool can provide alternative versions of your text or change its tone. You can specify how you want your information presented, such as writing a party invitation as a poem.
The Proofread tool checks grammar, word choice, and sentence structure, allowing you to review and accept changes. For lengthy documents, the summarize feature provides a brief overview. These tools will be available throughout the operating system and in third-party apps.
Apple Intelligence also focuses on reducing interruptions, ensuring you receive priority notifications. Moreover, Apple is extending its tools to developers. Federighi explained, “We’ve updated our SDKs [Software Development Kits] with new APIs [Application Programming Interfaces] and frameworks.” He added, “We’re investing deeply in our App Intents framework to connect the vast world of apps with Apple Intelligence.”
Currently, Apple Intelligence is compatible with iPhone 15 Pro, iPhone 15 Pro Max, M1 or later iPads, and M1 or later Macs.
iOS 18
Home Screen and Control Center
Your home screen no longer needs to follow a grid layout; you can place apps wherever you prefer and change icon colors. The Control Center will feature different screens, including a Control Gallery with various options. Developers can create custom controls for the Control Center, which you can position and size as needed, especially beneficial for those with low vision. On the lock screen, the flashlight and camera controls can be replaced with other options.
Apps and Privacy
iOS 18 introduces the option to lock apps, requiring authentication via Face ID, Touch ID, or Passcode to access them. Information within locked apps will remain private and not appear elsewhere in the system. Apps can also be hidden in a locked Hidden Apps folder. You can control which contacts an app can access, and apps requesting Bluetooth access will not automatically connect to all Bluetooth-enabled devices.
Messages
In iOS 18, you can use any emoji or sticker for Tapback responses. Messages can be scheduled for later delivery, and text formatting options will include bold, italic, underlined, and strike-through. New text effects will be available, and suggestions will offer more options. Additionally, texts can be sent via satellite with the iPhone 14 and later.
Devices with Apple Intelligence will benefit from various sorting options and the ability to send emails by topic as a digest. You can reassign emails to different categories if you disagree with the AI's sorting.
Maps
Maps will feature detailed topographical maps, including all National Parks, and provide spoken turn-by-turn directions. You can also create your own hiking routes.
Wallet
The new Tap to Cash option allows users to transfer money by holding their phones next to each other.
Game Mode
Game Mode on the iPhone minimizes background activity to enhance your gaming experience.
Photos
The Photos app has been reorganized for easier navigation. A new Filter button allows you to narrow searches to specific content types. AI will create collections based on trips or frequently appearing individuals in your photos. It will also simplify the deletion of unwanted photos such as screenshots.
iPadOS 18
Users will receive the same features as in iOS 18, along with some iPad-specific enhancements, such as a new floating tab bar that can move into the sidebar. Favorite tabs can be customized within the tab bar. Browsing documents will be more intuitive, and you can now tap and draw to illustrate tasks for others. If necessary, you can request permission to control their iPad or iPhone.
The Calculator app is coming to the iPad. When using the Apple Pencil to write an equation, the iPad will solve it upon entering the equals sign. The calculator also functions as a scientific calculator.
watchOS 11
You will be able to set your activity rings by day and pause them to preserve your streak. The new training load features provide a rating from 1 to 10 on your workout activities, utilizing metrics such as heart rate, pace, age, and weight. The watch employs a specially designed algorithm to estimate your effort rating, which you can review on the Summary page in the app. Your effort rating and duration contribute to calculating your training load, allowing you to monitor your workout trends.
The Summary tab in the Fitness+ app can be customized. The Vitals app tracks metrics such as heart rate and temperature overnight, making recommendations if necessary. Smart Stack will offer more widgets, and the Check-in feature will be available on Apple Watch. The watch can now use live activities to track events like flights and sports scores.
macOS Sequoia
The new continuity feature, iPhone mirroring, allows you to control your iPhone from the Mac, which is particularly useful when your iPhone is not nearby. You can use the keyboard and trackpad, view iPhone notifications on the Mac, and play iPhone audio through Mac speakers. The iPhone will remain locked.
Presenter Preview shows what you are going to share before sharing it. You can now replace your background with built-in backgrounds or your photos.
Passwords
Passwords will have their own dedicated app, available on Mac, iPhone, iPad, Vision Pro, and the iCloud for Windows app. Passwords will sync across devices and be available in the app when using autofill.
Safari
While browsing a website, Safari can highlight significant content and display it. It can also create a table of contents and summary in Reader mode.
Vision ProSpatial photos can be created from 2D images, and new gestures have been added. If you are watching an Apple TV+ program, you can quickly get information about the actors and characters.
tvOSA new feature of tvOS, enhanced dialogue, provides greater vocal clarity when activated. tvOS will support 21:9 projectors and more speakers. The new Insight feature on Apple TV+ gives you information about what is happening on the screen.
The Bottom Line
In addition to the general improvements and new features you would expect in an iOS update, Apple's inclusion of generative AI throughout has the possibility to be a significant step forward both in regard to productivity as well as accessibility. The planned "on-screen awareness" feature for Siri might have an impact on app accessibility in particular. Stay tuned for the fall, when we will describe how these features function in action as well as detail the accessibility specific additions in the final release of iOS 18.