Qualcomm's 'Holy Grail': Generative AI Is Coming to Phones Soon
The company wants its next-gen Snapdragon chips to use AI for more than just improving camera shots.
Generative AI like ChatGPT and Midjourney have dazzled imaginations and disrupted industries, but their debut has mostly been limited to browser windows on desktop computers. Next year, you'll be able to make use of generative AI on the go once premium phones launch with Qualcomm's top-tier chips inside.
Phones have used AI for years to touch up photos and improve autocorrect, but generative AI tools could bring the next level of enhancements to the mobile experience. Qualcomm is building generative AI into its next generation of premium chips, which are set to debut at its annual Qualcomm Summit in Hawaii in late October.
Summit attendees will get to experience firsthand what generative AI will bring to phones, but Qualcomm senior vice president of product management Ziad Asghar described to CNET why users should get excited for on-device AI. For one, having access to a user's data -- driving patterns, restaurant searches, photos and more -- all in one place will make solutions generated by AI in your phone much more customized and helpful than general responses from cloud-based generative AI.
"I think that's going to be the holy grail," Asghar said. "That's the true promise that makes us really excited about where this technology can go."
There are other advantages to having generative AI on-device. Most importantly, queries and personal data searched are kept private and not relayed through a distant server. Using local AI is also faster than waiting for cloud computation, and it can work while traveling on airplanes or in other areas that lack cell service.
But an on-device solution also makes business and efficiency sense. As machine learning models have gotten more complex (from hundreds of thousands of parameters to billions, Asghar said), it's more expensive to run servers answering queries, as Qualcomm explained in a white paper published last month. Back in April, OpenAI was estimated to spend around $700,000 per day getting ChatGPT to answer prompts, and that cost prediction was based on the older GPT-3 model, not the newer GPT-4 that is more complex and likely to be costlier to maintain at scale. Instead of needing an entire server farm, Qualcomm's solution is to have a device's existing silicon brain do all the thinking needed -- at no extra cost.
"Running AI on your phone is effectively free -- you paid for the computing power up front," Techsponential analyst Avi Greengart told CNET over email.
Greengart saw Qualcomm's on-device generative AI in action when the chipmaker had it on display at Mobile World Congress in February, using a Snapdragon 8 Gen 2-powered Android phone to run the image generating software Stable Diffusion. Despite being an early demo, he found it "tremendously exciting."
What on-device generative AI provides users
Qualcomm has ideas for what people could do with phone-based generative AI, improving everything from productivity tasks to watching entertainment to creating content.
As the Stable Diffusion demo showcased, on-device generative AI could allow people to tweak images on command, like asking it to change the background to put you in front of the Venice canals, Asghar said. Or they could have it generate a completely new image -- but that's just the beginning, as text and visual large learning models could work in succession to flow from an idea to a ready output.
Using multiple models, Asghar said, a user could have their speech translated by automatic speech recognition into text that is then fed into an image generator. Take that a step further and have your phone render a person's face, which uses generative AI to make realistic mouth movements and text-to-speech to speak back to you, and boom, you've got a generative AI-powered virtual assistant you can have full conversations with.
This specific example could be powered in part by third-party AI, like Facebook parent company Meta's recently launched large language model Llama 2 in partnership with Microsoft as well as Qualcomm.
"[Llama 2] will allow customers, partners and developers to build use cases, such as intelligent virtual assistants, productivity applications, content creation tools, entertainment and more," Qualcomm said in a press release at the time. "These new on-device AI experiences, powered by Snapdragon, can work in areas with no connectivity or even in airplane mode."
2023-08-25 19:50:56
