Jiaju Lin

[Project] PandaReading: A Chatbot that Facilitates Constructive Children-Parents Interaction in Reading

Jiaju Lin , Feiwen Xiao, Zhaohui Li

Under Construction

Keywords: AI4Edu, LLM, Children-Parents Interaction, Reading Comprehension

▼ Abstract:

We are developing a chatbot to improve the children-parents interaction quality when they are reading a story book together.

[Project] Emotional Town: A Simulated Society driven by Language Modesl as An Immersive Environment for Social Emotional Training

Jiaju Lin , Feiwen Xiao

Under Construction

Keywords: AI4Edu, LLM Agents, Social Emotional Learning

▼ Abstract:

We are developing a simulated town based on AgentSims as a learning platform for social emotional skills like emotion recognition and management.

ByteComposer: a Human-like Melody Composition Method based on Language Model Agent

Xia Liang, Jiaju Lin , Xingjian Du

Keywords: MultiModal Learning, Machine Music Composition, LMM Agent

▼ Abstract:

We propose ByteComposer, an agent framework emulating a human's creative pipeline in four separate steps : "Conception Analysis - Draft Composition - Self-Evaluation and Modification - Aesthetic Selection". This framework seamlessly blends the interactive and knowledge-understanding features of LLMs with existing symbolic music generation models, thereby achieving a melody composition agent comparable to human creators. We conduct extensive experiments on GPT4 and several open-source large language models, which substantiate our framework's effectiveness.

AgentSims: An Open-Source Sandbox for Large Language Model Evaluation

Jiaju Lin , Haoran Zhao, Aochi Zhang, Yiting Wu, Huqiuyue Ping, Qin Chen

Keywords: LLM-driven Agents, LLM evaluation

▼ Abstract:

We suggest that task-based evaluation, where LLM agents complete tasks in a simulated environment, is a one-for-all solution to solve above problems. We present AgentSims, an easy-to-use infrastructure for researchers from all disciplines to test the specific capacities they are interested in. Researchers can build their evaluation tasks by adding agents and buildings on an interactive GUI or deploy and test new support mechanisms

Joint Music and Language Attention Models for Zero-shot Music Tagging

Xingjian Du, Zhesong Yu, Jiaju Lin, Bilei Zhu, Qiuqiang Kong

Keywords: MultiModal Learning, Music Tagging, Dataset

▼ Abstract:

In this work, we propose a zero-shot music tagging system modeled by a joint music and language attention (JMLA) model to address the open-set music tagging problem. The JMLA model consists of an audio encoder modeled by a pretrained masked autoencoder and a decoder modeled by a Falcon7B. We introduce preceiver resampler to convert arbitrary length audio into fixed length embeddings.

Educhat: A large-scale language model-based chatbot system for intelligent education

Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He, Xipeng Qiu

Keywords: LLM, Education Instructions

▼ Abstract:

EduChat (https://www.educhat.top/) is a large-scale language model (LLM)-based chatbot system in the education domain. Its goal is to support personalized, fair, and compassionate intelligent education, serving teachers, students, and parents.

Cup: Curriculum learning based prompt tuning for implicit event argument extraction

Jiaju Lin , Qin Chen, Jie Zhou, Jian Jin, Liang He

Keywords: Event Extraction, Prompt Tuning, Curriculum Learning

▼ Abstract:

we propose a Curriculum learning based Prompt tuning (CUP) approach, which resolves implicit EAE by four learning stages. The stages are defined according to the relations with the trigger node in a semantic graph, which well captures the long-range dependency between arguments and the trigger. In addition, we integrate a prompt-based encoder-decoder model to elicit related knowledge from pre-trained language models (PLMs) in each stage, where the prompt templates are adapted with the learning progress to enhance the reasoning for arguments.

Jiaju Lin , Qin Chen

Keywords: Event Extraction, Prompt Learning

▼ Abstract:

One of the first work that integrates prompt learning into event extraction. We present a novel prompt-based approach, which elicits both the independent and joint knowledge about different events for event argument extraction.