Student Research Opportunities
Contact: Xuhao Chen
We are looking for undergraduate students to join our research projects. If you are interested in working in machine learning, computer systems and architectures, feel free to reach out.
The AI revolution is transforming various industries and having a significant impact on society. However, AI is computationally expensive and hard to scale, which poses a great challenge in computer system design. Our lab is broadly interested in computer systems, architectures, high performance computing for AI and machine learning.
We look at system conferences [OSDI, SOSP], [ASPLOS], [ISCA], [VLDB], [SIGMOD] [SC], [PPoPP].
Interesting MLSys topics: Transformer (Attention), Mixture-of-Experts, Vector Similarity Search, Deep Recommendation Models, Graph Machine Learning, Graph Sampling, Graph Algorithms, Robotics, Large Language Models, GPU Acceleration, Model Serving, Graph Databases, Graph Transformer, Diffusion, Generative AI, Reinforcement Learning. Below are some ongoing research projects.
Scalable Vector Database [Link]
Recent advances in deep learning models map almost all types of data (e.g., images, videos, documents) into high-dimension vectors. Queries on high-dimensional vectors enable complex semantic-analysis that was previously difficult if not impossible, thus they become the cornerstone for many important online services like search, eCommerce, and recommendation systems.
In this project we aim to build a massive-scale Vector Database on the multi-CPU and multi-GPU platform. In a Vector Database, the major operation is to search the k closest vectors to a given query vector, known as k-Nearest-Neighbor (kNN) search. Due to massive data scale, Approximate Nearest-Neighbor (ANN) search is used in practice instead. One of the most promising ANN approaches is the graph-based approach, which first constructs a proximity graph on the dataset, connecting pairs of vectors that are close to each other, then performs a graph traversal on the proximity graph for each query to find the closest vectors to a query vector. In this project we will build a vector database using graph-based ANN search algorithm that supports billion-scale datasets.
Qualifications:
- Strong programming skills in C/C++/Python language
- Experience with design and analysis of algorithms, e.g., MIT 6.1220 (previously 6.046)
- Experience with performance engineering is a plus, e.g., MIT 6.1060 (previously 6.172)
- GPU/CUDA programming is a plus
This project has the potential to become an MEng thesis. Please send your CV to cxh@mit.edu if you are interested. In the email please also specify when you want to start, and if you look for direct funding, course credit or volunteering.
Graph AI System [Link]
Deep Learning is good at capturing hidden patterns of Euclidean data (images, text, videos). But what about applications where data is generated from non-Euclidean domains, represented as graphs with complex relationships and interdependencies between objects? That’s where Graph AI or Graph ML come in. Handling the complexity of graph data and graph algorithms requires innovations in every layer of the computer system, including both software and hardware.
In this project we will design and build efficient graph AI systems to support scalable graph AI computing. In particular, we will build software frameworks for Graph AI and ML, e.g., graph neural networks (GNN), graph pattern mining (GPM) and graph sampling, and hardware accelerators that further enhance system efficiency and scalability.
Qualifications:
- Strong programming skills in C/C++/Python language
- Experience with design and analysis of algorithms, e.g., MIT 6.1220 (previously 6.046)
- Basic understanding of computer architecture, e.g., MIT 6.1910 (previously 6.004)
- Experience with performance engineering is a plus, e.g., MIT 6.1060 (previously 6.172)
- GPU/CUDA programming is a plus
- Familiarity with deep learning frameworks is a plus, e.g., PyTorch
This project has the potential to become an MEng thesis. Please send your CV to cxh@mit.edu if you are interested. In the email please also specify when you want to start.
Please specify in the email if you look for direct funding, course credit or volunteering.
Graph AI for Financial Security [Link]
The advent of cryptocurrency introduced by Bitcoin ignited an explosion of technological and entrepreneurial interest in payment processing. Dampening this excitement was Bitcoin’s bad reputation. Many criminals used Bitcoin’s pseudonymity to hide in plain sight, conducting ransomware attacks and operating dark marketplaces for the exchange of illegal goods and services.
This project offers a golden opportunity to apply machine learning for financial forensics. The data of Bitcoin transactions naturally forms a financial transaction graph, in which we can apply graph machine learning and graph pattern mining techniques to automatically detect illegal activities. We will explore the identification and clustering of frequent subgraphs to uncover money laundering patterns, and conduct link predictions on the wallets (nodes) to unveil the malicious actor behind the scene.
Qualifications:
- Strong programming skills in Python and C/C++ language
- Background and prior experience with design and analysis of algorithms (e.g., 6.046)
- Some familiarity with deep learning frameworks such as PyTorch
This project has the potential to become an MEng thesis. We have 6-A program opportunities available. Please send your CV to cxh@mit.edu if you are interested. In the email please also specify when you want to start.
Please specify in the email if you look for direct funding, course credit or volunteering.
Scalable Graph Transformer [Link]
Transformer [1], the engine behind ChatGPT, has yielded competitive results in many domains such as natural language processing (NLP) [2] and computer vision (CV) [3] due to its unique attention mechanism where each element in the input data pays attention to every other element.
Recently Graph Transformer [4,5,6] has been proposed by introducing the attention mechanism into graph machine learning. As opposed to graph neural networks (GNNs) and graph convolutional networks (GCNs) where nodes use neighborhood aggregation (a.k.a. message passing) mechanisms to perform inference on graph data, graph transformers use the self-attention mechanism that can capture long-range interactions between nodes. However, current graph transformers do not scale to large graphs because of the expensive computation and memory consumption. Thus, this project aims to design a graph transformer with hierarchical multi-head attention (MHA) to improve node classification for graphs on the order of 100k-100M nodes.
Qualifications:
- Strong programming skills in Python and C/C++ language
- Background and prior experience with design and analysis of algorithms (e.g., 6.046)
- Some familiarity with deep learning frameworks such as PyTorch
This project has the potential to become an MEng thesis. We have 6-A program opportunities available. Please send your CV to cxh@mit.edu if you are interested. In the email please also specify when you want to start.
Please specify in the email if you look for direct funding, course credit or volunteering.