EXAONE 4.5 Technical Report

核心问题

How can an effective open-weight vision-language model be developed for industrial applications by integrating visual processing capabilities into a large language model architecture?

核心方法

{'approach': 'The approach trains a 1.2B-parameter vision encoder from scratch and integrates it with the EXAONE 4.0 32B language model using a two-stage pre-training pipeline for cross-modal alignment. The model employs Grouped Query Attention, 2D RoPE for vision encoding, and embeds context extension into supervised fine-tuning to achieve 256K token context length. Multi-stage offline preference optimization and reinforcement learning enhance reasoning capabilities across text and vision domains.', 'key_components': ['A 1.2B-parameter vision encoder was trained from scratch to avoid performance degradation from visual token reduction.', 'Grouped Query Attention (GQA) is employed in both vision encoder and language model for computational efficiency.', '2D RoPE is used for vision encoding while 1D RoPE is maintained for language processing to optimize cross-modal performance.', 'The K-EXAONE tokenizer provides enhanced multilingual support, particularly for Korean language processing.', 'The license agreement governs use of the EXAONE AI Model between Licensee and LG Management Development Institute Co., Ltd.', 'Users agree to the terms by downloading, installing, copying, or using the Model.', 'The Agreement constitutes a binding legal contract between Licensee and Licensor.', 'Users who do not agree to all terms must not download, install, copy, or use the Model.'], 'section_ids': ['sec_2', 'sec_20']}

论点验证

可复现性评估

较低可复现性 (0%)

局限性（作者自述）

论文中未明确列出局限性。