Is the Apple GPT in your pocket? This may become a reality faster.
Apple artificial intelligence (AI) researchers recently published a paper on the preprint website arXiv, which mentioned an innovative "flash utilization" technology that can deploy large language models (LLMs) on iPhones and other memory limited Apple devices, which is almost a major breakthrough.
Memory constraints
LLM based chatbots (such as ChatGPT, Claude, etc.) rely heavily on data and memory, requiring a large amount of data to be processed simultaneously, often requiring a large amount of memory to run.
Therefore, running LLM is a challenge for devices such as iPhones with limited DRAM (generally referring to memory) capacity.
Usually, the standard method for computing data is to load the data from flash memory into DRAM, and then perform data inference in DRAM.
DRAM with high performance can increase data processing speed by millions of times, but the downside is its capacity. Running on DRAM severely limits the maximum model size that can be run.
To address this issue, Apple researchers have developed a new technology that uses larger capacity flash memory to store data from artificial intelligence models, which can then be transferred to DRAM memory for processing when needed.
Storing AI on flash memory
In a new research paper titled "LLM in Flash: Efficient Large Language Model Reasoning in Limited Memory", the author points out that flash memory in mobile devices is more abundant than traditional memory used to run LLM.
This method cleverly bypasses capacity limitations. The paper proposes two key technologies to minimize data transmission and maximize flash processing capabilities:
One of them is called "windowing" technology, which is equivalent to a recycling method. AI models do not need to load new data every time, but instead reuse some already processed data. This reduces the need for continuous memory acquisition, making the process faster and smoother.
The second is called "Row Column Bundling" technology. This technology is achieved by grouping data more effectively, that is, by setting the order of accessing data blocks based on the data characteristics of flash memory, which can read data from flash memory faster and accelerate the ability of artificial intelligence to understand and generate language.
According to this paper, the combination of these methods enables the running capacity of artificial intelligence models to reach twice the available memory of iPhones. This means that under this method, the inference speed in the CPU has increased by 4-5 times compared to traditional loading methods, and the inference speed in the GPU has increased by an astonishing 20-25 times.
The author of the paper wrote, "This breakthrough is particularly important for deploying advanced LLMs in resource limited environments, thereby expanding their applicability and accessibility."
Apple's AI Strategy
The breakthrough in artificial intelligence efficiency has opened up new possibilities for future iPhones, such as more advanced Siri features, real-time language translation, complex AI driven photography, and augmented reality capabilities.
The new technology in the paper also lays the foundation for iPhone to run complex artificial intelligence assistants and chatbots on devices, and it is said that Apple is already developing this technology.
Apple's work in generative artificial intelligence may eventually be integrated into its voice assistant Siri. Apple introduced its large-scale language model work to employees at the February Artificial Intelligence Summit this year. According to previous media reports, Apple's goal is to launch an intelligent version of Siri that is deeply integrated with artificial intelligence.
There are also rumors that Apple plans to add artificial intelligence to as many Apple applications as possible.
In addition, according to reports, Apple is also developing its own generative artificial intelligence model, "Ajax," which runs on 200 billion parameters to compete with OpenAI's GPT-4 model.
Internally known as "Apple GPT," Ajax aims to unify the entire Apple machine learning development, highlighting Apple's broader strategy of integrating artificial intelligence deeper into the Apple ecosystem.
According to the latest report, Ajax is considered more powerful than the early ChatGPT 3.5. However, the new model GPT-4 launched by OpenAI in September 2023 may have surpassed the capabilities of Ajax.
Fruit Chain analyst Jeff Pu has pointed out that Apple will launch some kind of generative artificial intelligence feature on iPhones and iPads around the end of 2024, which will be included in iOS 18. Pu also stated that Apple will build hundreds of artificial intelligence servers in 2023, and there will be more in 2024.