If you follow the latest developments in artificial intelligence, you’ll find that bigger models and more data are the keys to success. Jiang Daxin, the founder of Stepfun, a Shanghai-based AI start-up, believes in the power of scaling laws in large language model (LLM) development. Despite challenges like lower investment and a lack of advanced chips in China, Jiang remains optimistic.
Jiang, who used to work at Microsoft, shared his thoughts at the World Artificial Intelligence Conference (WAIC) in Shanghai. He predicts that LLMs will eventually reach hundreds of trillions of parameters, greatly enhancing their capabilities.
Scaling laws are all about the relationship between an AI model’s performance and its number of parameters. Generally, larger models perform better, especially with more data and excellent computational resources, although the improvements can slow down after a certain point. Big tech companies invest heavily in advanced technology, particularly Nvidia’s H100 chips, to maximize performance.
Jiang highlighted this trend in his talk. “The advancements in OpenAI’s GPT series, which powers ChatGPT, and the massive investments in supercomputing centers by companies like Amazon, Microsoft, and Meta show that scaling laws work,” he said on Saturday. However, he cautioned that the availability of data, skilled personnel, and concerns about return on investment could affect the pace of these advancements.
Since OpenAI launched ChatGPT in late 2022, Chinese tech giants and start-ups have been eager to develop their LLMs. China has over 200 AI models, including Alibaba’s Tongyi Qianwen and Baidu’s Ernie. Alibaba owns the South China Morning Post, which reported this news. Yet, many Chinese AI firms struggle to match the spending power of their US counterparts and focus instead on revenue-generating applications.
Founded in April 2023, Stepfun has been dedicated to developing fundamental models. At WAIC, the company launched Step-2, a trillion-parameter LLM, along with the Step-1.5V multimodal model and the Step-1X image generation model.
Jiang also emphasized the importance of multimodality in creating a comprehensive AI. Multimodal models can process visual and other data types to develop internal representations of the external world. He explained that Stepfun aims to combine generative and comprehension abilities in a single model.
Stepfun also offers consumer-facing products, such as Yuewen, a ChatGPT-like personal assistant, and Maopaoya, an AI companion that can take on various character personalities.
“Last year, global AI investments reached US$22.4 billion, with 70 to 80 percent going to companies developing large models,” said Alex Zhou Zhifeng, managing partner at Qiming Venture Partners, at another WAIC side event. Qiming was an early investor in Stepfun.
Zhou noted that more investments in AI applications are expected soon, partly due to decreasing token costs. In AI, a token is a basic data unit processed by algorithms.
Peng Wensheng, an economist at China International Capital, added that China’s AI model market is projected to reach about 5.2 trillion yuan (US$715.1 billion) by 2030. The size of the industrial AI market is expected to be around 9.4 trillion yuan.
This optimistic outlook suggests a bright future for AI development in China, driven by the potential of scaling laws and innovative models like those from Stepfun.
- Scaling Laws and AI: Larger AI models generally perform better, although improvements slow down past a certain size.
- Stepfun’s Innovations: The company launched the Step-2 trillion-parameter LLM, the Step-1.5V multimodal model, and the Step-1X image generation model.
- Investment Trends: Global AI investments reached US$22.4 billion in the previous year, with significant funding going towards large model development.
- Future Projections: China’s AI model market is projected to reach 5.2 trillion yuan by 2030, with the industrial AI market expected to be around 9.4 trillion yuan.
AI Investment and Development in China
| Key Metrics | 2022 | 2030 (Projected) |
|---|---|---|
| AI Investments (Global) | US$22.4 billion | N/A |
| China’s AI Model Market | N/A | 5.2 trillion yuan |
| China’s Industrial AI Market | N/A | 9.4 trillion yuan |
Jiang Daxin’s insights at the WAIC underline the transformative potential of AI scaling laws and large language models. Despite facing challenges such as limited investment and technological constraints, Chinese AI firms continue to innovate and develop cutting-edge models. Stepfun’s ambitious projects exemplify the forward-looking approach necessary to compete on the global stage.
The company’s recent releases, including the Step-2 trillion-parameter LLM, Step-1.5V multimodal model, and Step-1X image generation model, showcase its commitment to pushing the boundaries of AI capabilities. By integrating multimodal processing and combining generative and comprehension abilities, Stepfun aims to create more versatile and powerful AI systems.
Jiang’s emphasis on the importance of scaling laws reflects a broader industry trend. Major tech companies like OpenAI, Amazon, Microsoft, and Meta invest heavily in advanced AI technologies, driving significant advancements. However, the availability of data, skilled personnel, and return on investment concerns remain critical factors influencing the pace of these developments.
As AI continues to evolve, the focus on scaling laws and the development of large language models will likely remain central to the industry’s progress. The combination of innovative models, increased investment, and a growing market indicates a promising future for AI in China and beyond.
Featured Image courtesy of DALL-E by ChatGPT