Bio
𝒯𝒾𝒶𝓃 𝒯𝒶𝓃 𝐵𝓊𝒹𝒹𝒽𝒶, 𝐻𝑜𝓃𝑔 𝒦𝑜𝓃𝑔
I am a Ph.D. student in Computer Engineering at Boston University, under the supervision of Prof. Vijaya Kolachalama. Prior to BU, I received my M.Phil. degree (2023) in Computer Science from City University of Hong Kong, supervised by Prof. Shiqi Wang. In 2017, I attended a summer school at University of California, Irvine.
𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗜𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝘀: In my career life, I keep seeking for a sweet spot between engaging in exciting research and open-sourcing impactful software and AI systems:
Multimodal Large Language Models (MLLMs)
Large Language Models (LLMs) [1]
Retrieval-augmented Generation (RAG) [1]
Generative AI [1]
Computer Vision [1]
AI4Science [1, 2, 3]
Research Focus Illustration
Research Presentations and Resources
𝐖𝐡𝐚𝐭 𝐢𝐬 𝐭𝐡𝐞 𝐦𝐨𝐬𝐭 𝐬𝐮𝐫𝐩𝐫𝐢𝐬𝐢𝐧𝐠 𝐟𝐢𝐧𝐝𝐢𝐧𝐠 𝐢𝐧 𝐲𝐨𝐮𝐫 𝐫𝐞𝐬𝐞𝐚𝐫𝐜𝐡?
𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 - Artificial Intelligence (AI)
AI In the 2020s And Beyond
𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 - Graph Neural Network (GNN)
Graph Convolutional Neural Networks
Attention-based BiLSTM-GCN
Dynamic Graph Convolutional Neural Networks
𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 - Natural Language Processing (NLP)
Foundation Models for Sequential Decision Making
Factual Associations in LLMs
Graph Matching
Sub-word BPE Algorithm for NMT
Concept Matching for Medical Terms
𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 - Computer Vision (CV)
Image Quality Assessment and Perceptual Optimization
Deep Learning Models Compression and Acceleration
3D Human Pose Estimation and Human Body Reconstruction
YOLO Object Detection
𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 - Tutorials and Useful Coding Scripts
Usage of Cloud Server and Setting-up
Python Environment Setting-up
TensorFlow for Deep Learning
Crypto Currency Return and Price Prediction with Machine Learning
Big Data Parallel Processing by PySpark and Horovod Distributed Deep Learning
Download Papers from Sci-Hub via Unix Shell
Retrieve and Download Google Scholar Citation Papers
Homepage Shows Real-time Google Scholar Citations through PHP
Dynamic (Ajax) Web Crawler in Python
Deploy Machine Learning and Deep Learning Models with Flask and Docker as Web Applications
Convert PowerPoint (PPT) to PDF with Animations
Download YouTube Videos with Unlimited Speed via youtube-dl
Compile Hadoop, Install Redis, Flink, Kafka, ZooKepper, and Spark on macOS
Hadoop HDFS Data
Hive Configuration on macOS
Spark Configuration on macOS
Hadoop Configuration on Linux
Hive Configuration on Linux
Storm Configuration on Linux
Compiling Kaldi ASR on macOS
Server SSH and SCP Scripts
Shell Tmux Usage
Go Programming Scripts
Julia Programming Scripts
Python Conda Scripts
Python PyPI Package Building Scripts
npm Scripts
R Packages
SQL Programming Scripts
CSV Shell Scripts
macOS Homebrew Scripts
Install LaTeX on Linux
LaTeX Package Installation on macOS and EPS to PDF
LaTeX Useful Newcommands
LaTeX IEEE Conference Useful Scripts
LaTeX IEEE Trans Useful Scripts
Matlab GPU Acceleration Tricks
Run Multiple Python Scripts via Shell sh
Build PyTorch (CPU) from Source on macOS
PyTorch Control the Usage of CPU Resources
Build TensorFlow (CPU) from Source on macOS
Build TensorFlow (CPU) from Source on Linux
Build TensorFlow (GPU) from Source on Linux
Docker Basic Commands
Vim Basic Commands
Git Basic Commands
M.Phil. Thesis (LaTeX), City University of Hong Kong
𝐄𝐱𝐜𝐞𝐥𝐥𝐞𝐧𝐭 𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧
On the Inductive Bias of Language Modeling | Oral Presentation (Tatsunori B. Hashimoto)
𝐓𝐞𝐜𝐡𝐧𝐢𝐜𝐚𝐥 𝐁𝐨𝐨𝐤𝐬
Foundation Models for Natural Language Processing: Pre-trained Language Models Integrating Media (Gerhard Paaß and Sven Giesselbach)
The Path to Artificial General Intelligence: Insights from Adversarial LLM Dialogue (Edward Y. Chang)
Pattern Recognition and Machine Learning (Christopher Bishop)
Reinforcement Learning: An Introduction (Richard S. Sutton and Andrew G. Barto)
Digital Image Processing (Rafael C. Gonzalez and Richard E. Woods)
Multiple View Geometry in Computer Vision (Richard Hartley and Andrew Zisserman)
Graph Representation Learning (William L. Hamilton)
Computer Systems: A Programmer's Perspective (Randal E. Bryant and David R. O'Hallaron)
Computer Organization and Design: The Hardware/Software Interface (David A. Patterson and John L. Hennessy)
𝐀𝐜𝐚𝐝𝐞𝐦𝐢𝐜 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬
Writing and Technical Presentations by Prof. Ayse Coskun
The Most Common Habits from more than 200 English Papers written by Graduate Chinese Engineering Students by Felicia Brittman
How to Write Good Research Articles by Prof. Xiaohua Jia
业余做研究的经验 by Dr. Yuandong Tian
What is Research and How to do it? by Prof. Yi Ma
How to Write a Good CVPR Submission? by Prof. Bill Freeman
Tips on Writing Papers with Mathematical Content by Prof. John N. Tsitsiklis
How to Be a Responsible Reviewer by Prof. Jiebo Luo
How to Use the IEEEtran LaTeX Class
IEEE Editing Mathematics Guide (Standard)
IEEE Math Typesetting Guide (LaTeX)
IEEE Formula Comma and Period
IEEE Reference Guide (2021)
IEEE Citation Examples
Science and Engineering Journal Abbreviations
Standard abbreviations used in the IEEE Reference list
IEEE Editorial Style Manual (2021)
LaTeX Mathematical Symbols
Typesetting Subtleties by Prof. Vivek Goyal
Writing Mathematical Formulas in Markdown
InCites Journal Citation Reports (2021)
中国计算机学会推荐国际学术会议和期刊目录 (2019)
中国科学院SCI分区表 (2019)
中国科学院国际期刊预警名单
Python编码风格与规范 @ Tencent
C++编码风格与规范 @ Tencent
JavaScript编码风格与规范 @ Tencent
Go编码风格与规范 @ Tencent
CVPR Paper, Supplementary Materials, and Rebuttal LaTeX Templates
PowerPoint (PPT) and LaTeX Demos of Academic Posters
National Science Foundation (NSF) Proposal & Award Policies & Procedures Guide (PAPPG)
National Science Foundation (NSF) Biographical Sketch
NSF's Proposal Preparation & Submission Guidelines
ACL Fellow Nomination Form
CityU AP Promotion
RICE AP Offer
MIT EECS Courses
MIT EECS Organization and Labs (2019)
Artificial Intelligence Index Report by Stanford
𝐑𝐞𝐜𝐨𝐦𝐦𝐞𝐧𝐝𝐞𝐝 𝐑𝐞𝐚𝐝𝐢𝐧𝐠𝐬
文章千古事, 得失寸心知 by Prof. Song-Chun Zhu
Statement of Purpose by Dr. Kai-Fu Lee
Dartmouth Summer Project on Artificial Intelligence (1956)
The Summer Vision Project at MIT (1966)
Turing Test (1950) by Alan Mathison Turing
The Bitter Lesson from 70 years of AI research by Richard Sutton
Academic Achievements of Prof. Richard Yi-Da Xu
Brief Bio of Prof. Richard Yi-Da Xu
Prompt Template Engineering Experiment by Prof. Edward Chang
心灵之旅 by Prof. Edward Chang
生命是什么? by Prof. Edward Chang
The Ph.D. Grind by Prof. Philip J. Guo
You and Your Research by Prof. Richard Hamming
You’ve Got to Find What You Love by Steve Jobs
A Brief History of Computational Vision by Prof. Song-Chun Zhu
Artificial Intelligence by Prof. Song-Chun Zhu
上海交通大学学生生存手册
李琳山教授个人数位典藏
Randy Pausch Last Lecture: Achieving Your Childhood Dreams
Contact
𝗖𝗼𝗻𝘁𝗮𝗰𝘁
Shuyue Jia (Bruce Jia)
Department of ECE, Boston University
Add: Kolachalama Lab, 14/F, Center for Computing & Data Sciences,
Boston University, 665 Commonwealth Ave., Boston, MA 02215
Tel: 6176851479
Email: brucejia@bu.edu; shuyuej@ieee.org
𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗧𝗲𝗮𝗺
Email:
brucejia@bu.edu;
shuyuej@ieee.org
Resume
Scholar
🤗 HuggingFace
GitHub
News
-
Nov 2024 Our PodGPT preprint is available online! It is an audio-augmented Large Language Model (LLM) for STEMM research and education.
-
Oct 2024 We have launched PodRAG on our PodGPT platform with the advanced 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) techniques! It is designed to provide 𝘁𝗵𝗲 𝗺𝗼𝘀𝘁 𝗮𝗰𝗰𝘂𝗿𝗮𝘁𝗲 𝗮𝗻𝗱 𝘂𝗽-𝘁𝗼-𝗱𝗮𝘁𝗲 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻 for medical education and research! Please try it out if you are interested!
-
Aug 2024 Open Source and Keep Updating Awesome Large Vision-Language Model (LVLM/MM-LLM), a curated list of Large Vision-Language Model
, and Awesome Mixture of Experts (MoE), a curated list of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)
.
Welcome to contribute and work together!
-
Aug 2024 Release and maintain a collection of 🤗 Quantized Large Language Models for public usage, offering AI solutions with reduced computational requirements.
-
July 2024 Open Source PodGPT Library, a library for benchmarking multilingual medical Large Language Models (Medical LLMs)
.
-
July 2024 Our MedPodGPT preprint is available online! It is an audio-augmented Large Language Model (LLM) for medical research and education.
-
June 2024 Our AI Platform, PodGPT, is now accessible to the general public. It is an online platform for deploying our latest multimodal foundation models for education and research.
-
June 2024 Our paper MedSyn is accepted by IEEE T-MI.
-
May 2024 𝗣𝗵.𝗗. 𝗖𝗮𝗻𝗱𝗶𝗱𝗮𝗰𝘆 𝗥𝗲𝗽𝗼𝗿𝘁: Preference Alignment via Reinforcement Learning from Human Feedback and 𝗣𝗿𝗲𝘀𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻: Slides.
-
Jan 2024 🔥 We are releasing 🤗 GSM8K-Consistency, a benchmark database for analyzing the consistency of Arithmetic Reasoning on GSM8K.
-
Dec 2023 Open Source 🔨 PromptCraft and its published PyPI Package, a prompt perturbation toolkit from the character, word, and sentence levels for prompt robustness analysis
.
-
Oct 2023 Open Source Awesome Semantic Textual Similarity, a curated list of Semantic/Sentence Textual Similarity (STS) in Large Language Models (LLMs) and Natural Language Processing (NLP)
.
-
Oct 2023 A Presentation of Sentence Textual Similarity: Model Evolution Overview.
-
Oct 2023 Open Source Awesome LLM Self-Consistency, a curated paper and presentation list of self-consistency in Large Language Models (LLMs)
.
-
Oct 2023 A Presentation of Prompt Perturbation and Robustness Evaluation.
-
Sept 2023 A Presentation of Prompt-based Learning and Robustness Evaluation.
-
Sept 2023 A Presentation of Self-Consistency Benefits Large Language Models.
-
Sept 2023 Our paper Deep Transfer Learning is accepted by IEEE T-IM.
-
May 2023 𝗠.𝗣𝗵𝗶𝗹.𝗧𝗵𝗲𝘀𝗶𝘀: No-reference Image Quality Assessment via Non-local Modeling and 𝗗𝗲𝗳𝗲𝗻𝘀𝗲 𝗦𝗹𝗶𝗱𝗲𝘀: Image Quality Assessment and Perceptual Optimization: A Non-local Modeling Approach.
-
Mar 2023 A Presentation of Foundation Models for Sequential Decision Making.
-
Jan 2023 A Presentation of IQA Regression and EEG Classification.
-
Nov 2022 A Research Proposal of Video Panoptic Segmentation (VPS).
-
Aug 2022 Our paper GCNs-Net is accepted by IEEE T-NNLS
.
-
Aug 2022 Our paper NLNet is accepted by IEEE MMSP. Source codes are available on GitHub
, and the trained models are available on 🤗 HuggingFace for real-life IQA inference.
-
Dec 2021 Our paper BiLSTM-GCNs is accepted by Frontiers in Bioengineering and Biotechnology.
-
Apr 2020 Open Source EEG-DL, a Deep Learning (DL) library written by TensorFlow for EEG Signals Classification
.
-
Feb 2020 Our paper ESI-CNNs is accepted by Journal of Neural Engineering
.
Publications
Topic 1 - Generative AI and Foundation Models
-
PodGPT: An Audio-augmented Large Language Model for Research and Education
Shuyue Jia, Subhrangshu Bit, Edward Searls, Meagan V. Lauber, Lindsey A. Claus, Pengrui Fan, Varuna H. Jasodanand, Divya Veerapaneni, William M. Wang, Rhoda Au, Vijaya B. Kolachalama
medRxiv Preprint
See More
Here, we introduce PodGPT, an audio-augmented large language model (LLM) tailored for research and education. The process began by leveraging publicly available generative AI auto-regressive language models across various scales. These models underwent continuous pre-training on a curated dataset of English CC-BY podcasts produced by scientific journals and clinical experts, as well as content from The New England Journal of Medicine (NEJM). The podcast corpus comprised over 3,700 hours of audio, covering diverse topics in science, research, and medicine, visually summarized in the accompanying word cloud. The next phase involved developing the software infrastructure, which included an inference engine for model deployment, a messaging queue, database integration, retrieval augmented generation (RAG) implementation, API microservices, and a responsive human-machine interface. This highly performant and robust system enabled users with internet access to engage seamlessly with current research and educational material via an adaptive chatbot. The chatbot supported multi-turn conversations across various languages, empowering users to access and interact with STEMM knowledge in a dynamic and accessible manner.
-
MedPodGPT: A Multilingual Audio-augmented Large Language Model for Medical Research and Education
Shuyue Jia, Subhrangshu Bit, Edward Searls, Lindsey A. Claus, Pengrui Fan, Varuna H. Jasodanand, Meagan V. Lauber, Divya Veerapaneni, William M. Wang, Rhoda Au, Vijaya B. Kolachalama
medRxiv Preprint
See More
Here, we introduce MedPodGPT, an audio-augmented large language model (LLM) designed for medical research and education. Medical podcasts offer audio content rich in specialized terminology, diverse medical topics, and expert dialogues, helping the medical community stay current with the latest information. Integrating this content into LLMs can enhance their ability to provide up-to-date clinical information.
-
MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images
Yanwu Xu, Li Sun, Wei Peng, Shuyue Jia, Katelyn Morrison, Adam Perer, Afrooz Zandifar, Shyam Visweswaran, Motahhare Eslami, Kayhan Batmanghelich
IEEE Transactions on Medical Imaging (IEEE T-MI)
Topic 2 - Computer Vision
-
No-reference Image Quality Assessment via Non-local Dependency Modeling
Shuyue Jia, Dingquan Li, Shiqi Wang
IEEE 24th International Workshop on Multimedia Signal Processing (IEEE MMSP)
See More
A no-reference image quality assessment method based on non-local features learned by a graph neural network (GNN). The proposed quality assessment framework is rooted in the view that the human visual system perceives image quality with long-dependency constructed among different regions, inspiring us to explore the non-local interactions in quality prediction.
-
Learning From Mixed Datasets: A Monotonic Image Quality Assessment Model
Zhaopeng Feng, Keyang Zhang, Shuyue Jia, Baoliang Chen, Shiqi Wang
IET Electronics Letters
Topic 3 - Neuroscience
-
GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals
Shuyue Jia, Yimin Hou, Xiangmin Lun, Yan Shi, Yang Li, Rui Zeng, Jinglei Lv
IEEE Transactions on Neural Networks and Learning Systems (IEEE T-NNLS)
See More
Traditional works classify EEG signals without considering the topological relationship among electrodes. Thus, a graph convolutional neural network is presented while cooperating with the functional topological relationship of electrodes.
-
Deep Feature Mining via Attention-based BiLSTM-GCN for Human Motor Imagery Recognition
Yimin Hou, Shuyue Jia (Corresponding Author), Xiangmin Lun, Shu Zhang, Jinglei Lv
Frontiers in Bioengineering and Biotechnology
See More
This paper presents a novel deep learning approach designed toward both remarkably accurate and responsive motor imagery (MI) recognition based on scalp EEG. Bidirectional long short-term memory (BiLSTM) with the attention mechanism is employed, and the graph convolutional neural network (GCN) promotes the decoding performance by cooperating with the topological structure of features.
-
A Novel Approach of Decoding EEG Four-Class Motor Imagery Tasks via Scout ESI and CNN
Yimin Hou, Lu Zhou, Shuyue Jia, Xiangmin Lun
Journal of Neural Engineering
See More
We presented a novel approach that could potentially improve the current stroke rehabilitation strategies by implementing a deep learning approach for an Electroencephalogram (EEG) based on MI Brain-Computer Interface System.
-
Constructed 6 convolutional layers, 2 max-pooling layers, and 3 FC layers CNNs for four-class motor imagery classification through TensorFlow, with 50% dropout (spatial dropout after every Conv layer and regular dropout for FC layers) – 11.44% accuracy improvement, batch normalization (BN) – 10.15% improvement, and Short-cut Connection – 1.76% improvement to prevent overfitting, and achieved SOTA results: 94.50% accuracy on scout R5, 94.54% at subject level, and 96% for left fist prediction.
-
Took charge of DNNs design, including methods comparisons, such as MLPs, CNNs, RNNs, and LSTMs, classification results calculations, and programming. 10 and 14 subjects’ data were utilized (19,320 and 27,048 samples in the experiments)
-
Benchmark Dataset: EEG Motor Movement/Imagery Dataset.
-
Attention-based Graph ResNet for Motor Intent Detection from Raw EEG Signals
Shuyue Jia, Yimin Hou, Yan Shi, Yang Li
arXiv preprint arXiv:2007.13484
Topic 4 - Intelligent Technologies
-
PMU Measurements based Short-term Voltage Stability Assessment of Power Systems via Deep Transfer Learning
Yang Li, Shitu Zhang, Yuanzheng Li, Jiting Cao, Shuyue Jia
IEEE Transactions on Instrumentation and Measurement (IEEE T-IM)
-
Improving Performance: A Collaborative Strategy for the Multi-data Fusion of Electronic Nose and Hyperspectral to Track the Quality Difference of Rice
Yan Shi, Hangcheng Yuan, Chenao Xiong, Qi Zhang, Shuyue Jia, Jingjing Liu, Hong Men
Sensors and Actuators B: Chemical
Academic Services
- Journal Reviewer of
IEEE Transactions on Multimedia (IEEE T-MM)
IEEE Transactions on Circuits and Systems for Video Technology (IEEE T-CSVT)
IEEE Transactions on Neural Networks and Learning Systems (IEEE T-NNLS)
IEEE Transactions on Industrial Informatics (IEEE T-II)
IEEE Journal of Biomedical and Health Informatics (IEEE JBHI)
IEEE Open Journal of the Industrial Electronics Society (IEEE OJIES)
IEEE Open Journal of the Computer Society (IEEE OJCS)
IEEE MultiMedia (MM)
IEEE Sensors Journal
Journal of Medical Internet Research
- Conference Reviewer of
The International Conference on Learning Representations (ICLR) 2025
- Committee Member of Local Conference Committee, IEEE Boston Section
- Member of IEEE, ACM, ACL, AAAI, and AAAS
Selected Awards
-
CityU Top 5 Runner, City University of Hong Kong
Athlete
-
Outstanding Athlete, Northeast Electric Power University
Athlete
-
3000-meter Steeplechase, The 45th Northeast Electric Power University Games
The 7th Place in college
-
2015 National High School Math League, China
Second Prize
-
The 32nd Chinese Physics Olympiad (CPhO), China
Third Prize
International Marathon Athlete Activities
-
2024 Boston Half
01:43:13 (906 / 6524)
-
2021 Standard Chartered Hong Kong Marathon
01:38:14 (318 / 6000)
-
2017 National Marathon Championships (Jilin City Station)
01:47:36 (148 / 5000)