Haiyue Song: LLM Researcher and Engineer Profile

Bridging Worlds: The AI Engineering and Multilingual Research of Haiyue Song

In the rapidly evolving landscape of artificial intelligence, breakthroughs often depend on researchers who can bridge distinct linguistic, structural, and computational domains. Haiyue Song (宋海越) is one such contemporary voice, making distinct technical waves in the fields of Large Language Models (LLMs), machine translation, and natural language processing (NLP).

Blending a rigorous academic foundation with cutting-edge industry application, Song’s work tackles some of the most stubborn structural and multilingual bottlenecks in modern AI.

Roots and Academic Foundation

The path of a researcher is often mirrored in their background. Born with a name that poetically translates to “crossing the sea”—where Hai (海) means sea and Yue (越) means to cross—Song’s academic journey has literally spanned borders and disciplines:

• The Formative Years: Song completed his Bachelor of Computer Science and Technology at Shanghai Jiao Tong University, concurrently pursuing a minor in Japanese. This early immersion in linguistics and computation laid the groundwork for an international career centered on multilingual systems.
• The Kyoto Era: Relocating to Japan, Song earned both his Master’s in Intelligence Science and Technology and his Ph.D. in Informatics from Kyoto University, under the guidance of renowned scholars like Prof. Sadao Kurohashi. During this time, he also served as a Japan Society for the Promotion of Science (JSPS) Research Fellow, cementing his expertise in text segmentation and machine translation architectures.

Core Contributions to NLP and Machine Translation

Before shifting deeper into the realm of generative AI and LLMs, Song built a prolific publication record addressing how machines process words, subwords, and diverse writing systems.

Key themes in his early research included:

• Subword Segmentation and Tokenization: Recognizing that standard tokenizers often fail low-resource or morphologically complex languages, Song introduced innovative methods like SelfSeg and SubMerge to optimize subword regularizations and align cross-granularity structures in neural machine translation.
• Low-Resource Adaptations: From winning top spots at translation challenges (such as IWSLT) to building benchmarks for African and Asian languages, his work ensures that the benefits of modern language tech are not restricted solely to high-resource languages like English.

Pushing Boundaries in the Era of Large Language Models

As the AI industry transitioned from traditional translation models to giant generative architectures, Song’s research evolved in tandem. After serving as a technical researcher at Japan’s National Institute of Information and Communications Technology (NICT), he stepped into the industry as an LLM Post-Train Engineer at Preferred Networks.

At Preferred Networks, Song focuses heavily on the post-training lifecycle of foundational models—such as PLaMo, a Japanese LLM built completely from scratch. His cutting-edge contributions target several critical pillars of modern AI development:

1. Continual Pre-Training & Mid-Training: Finding efficient ways to update and expand model knowledge without catastrophic forgetting. His research explores advanced methodologies like optimal distribution vector merging (OptiMer) to enhance pre-training phases.
2. Structured Document Translation & Format RL: Moving beyond simple sentence-level translation, Song’s work utilizes Reinforcement Learning (such as FormatRL) to help models accurately handle complex document structures like XML and HTML tags, bridging the gap between raw text generation and software documentation requirements.
3. Agent Harnesses and Long-Context Capabilities: Tailoring LLMs to act as intelligent, reliable agents capable of reasoning over extended contexts and executing multi-step tasks.

Looking Ahead

Haiyue Song’s trajectory exemplifies the modern AI scientist: deeply rooted in academic theory yet actively engineering practical, production-ready systems. Whether he is optimizing how a neural network processes byte-level metrics or refining reinforcement learning loops for structured data, his work continues to quietly shape how machines understand, translate, and bridge human languages across the globe.

Did You Know?
Outside of his core engineering and research duties, Song remains deeply engaged in the global academic community—serving as an area chair, workshop co-organizer, and program committee member for top-tier NLP venues like EMNLP and ACL.