ETRI Launches Development of Groundbreaking 100B-Scale AI Model

ETRI, South Korea’s premier government-funded research institute, has initiated the development of a large-scale artificial intelligence model, targeting a groundbreaking **100 billion parameters**. Selected as one of five national research teams by the **NC AI Consortium**, ETRI began its full-scale AI research in August 2023, in alignment with a project promoted by the **Ministry of Science and ICT**.

This selection not only cements ETRI’s role as a national research leader but also enhances its representation within the global AI research community. The project, titled “Development of a Scalable Multimodal Generative Foundation Model for Industrial AI Transition,” aims to create advanced AI technologies applicable across various industrial sectors, including manufacturing, healthcare, education, and culture.

Advancing Industrial AI through Multimodal Technologies

ETRI’s initiative focuses on enhancing multimodal AI technology, which is capable of integrating and processing diverse data types such as text, audio, and images. This approach is expected to accelerate AI innovation in key industries by developing models that can operate effectively in real-world settings.

Currently, ETRI has established a robust research infrastructure, with data support from the **National Information Society Agency (NIA)** and GPU resources from the **National IT Industry Promotion Agency (NIPA)**. This foundational work is facilitating the smooth pre-training of the **100B-scale model**, aimed at bolstering South Korea’s technological independence in the field of ultra-large AI models.

ETRI’s project represents a critical national strategy to enhance the country’s AI competitiveness. It seeks to develop **Large Language Models (LLMs)** and multimodal AI technologies that can serve multiple sectors, including industry, public services, and academia.

Leveraging Existing Technologies for Enhanced AI Models

The **Intelligence Information Research Division** at ETRI is integrating technologies from previous national projects into this new research initiative. The **Language Intelligence Research Section** is applying insights from the language model “Eagle,” which has been designed to enhance understanding and inference in the Korean language. This model is expected to transition into a multimodal foundation model post-2026.

Additionally, the **Embodied Integrated Intelligence Research Section** is focused on developing multimodal AI models that prioritize audio and video applications. This effort builds on significant research advancements, including technologies for assessing degenerative brain functions and multi-speaker conversation modeling. The objective is to expand these developments into general-purpose foundational models.

The **Visual Intelligence Research Section** is concurrently enhancing the performance of vision-language fusion generation and ensuring AI safety. This is based on technologies developed through the text-to-image generation model “KOALA” and the visual language question-answering model “Ko-LLaVA.”

Oh-woog Kwon, Assistant Vice President of ETRI’s Intelligence Information Research Division and project manager, stated, “As the only government-funded research institute to represent the nation, ETRI has overcome initial resource challenges and is steadily pre-training the 100B-scale model. Our collaboration with NC AI aims to lead the transformation of industrial AI by developing a scalable and highly reliable multimodal foundation model.”

ETRI is leveraging this project to advance its technology roadmap, which connects model development, large-scale pre-training, and industrial implementation. With support from NIA and NIPA, the outcomes of this initiative are poised to mark a significant milestone in South Korea’s quest for independent and cutting-edge AI capabilities.

As the project progresses, ETRI is anticipated to play a crucial role in establishing South Korea as a global leader in AI technology, driving innovation within the AI ecosystem and facilitating the intelligent transformation of various industries nationwide.