About

I’m Andrei, a 1st year PhD student at University of Montreal and Mila supervised by Irina Rish, working on mechanistic interpretability and continual learning of language models for scientific texts. Previously, I was a Masters’ student in Jackie Cheung’s group at McGill University and Mila, with my thesis on the topic of augmenting language model pretraining with lexical semantics. I also have collaborations with Renee Seiber’s group (applying NLP to social media to support the information needs of crisis managers and affected people during extreme weather events) and Yaoyao Fiona Zhao’s group (creating a scientific information extraction system leveraging LLMs to support researchers in literature reviews).

My current research interests center on how language models and machine learning more broadly can effectively support scientists in their research and enable scientific progress. I’m currently approaching this from three related angles: (1) mechanistic interpretability to gain insights into how LLMs handle scientific knowledge; (2) inductive biases to improve LLM robustness on scientific domains; and (3) HCI to better understand how LLMs can support scientists in practice. If this interests you, please reach out!

prof_pic.png
mirandrom+ghp@pm.me

Selected Publications

2023

  1. Balaur: Language Model Pretraining with Lexical Semantics
    Andrei Mircea, and Jackie Chi Kit Cheung
    In Findings of the Association for Computational Linguistics: EMNLP 2023, 2023

2021

  1. Discourse-Aware Unsupervised Summarization for Long Scientific Documents
    Yue Dong*, Andrei Mircea*, and Jackie Chi Kit Cheung
    In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021
  2. Bridging the gap between supervised classification and unsupervised topic modelling for social-media assisted crisis management
    Mikael Brunila*, Rosie Zhao*, Andrei Mircea*, and 2 more authors
    In Proceedings of the Second Workshop on Domain Adaptation for NLP, 2021

2020

  1. Real-time Classification, Geolocation and Interactive Visualization of COVID-19 Information Shared on Social Media to Better Understand Global Developments
    Andrei Mircea
    In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020, 2020
  2. Using deep learning and social network analysis to understand and manage extreme flooding
    Andrei Romascanu*, Hannah Ker*, Renee Sieber, and 6 more authors
    Journal of Contingencies and Crisis Management, 2020

News

Sep 2023 🎉 Started a PhD with Irina Rish at the University of Montreal and Mila
Jun 2023 🧑‍🔬 Summer research with the Additive Design and Manufacturing Lab at McGill under the supervision of Yaoyao Fiona Zhao, building a human-centered scientific information extraction system with large language models and Next.js
May 2023 🎓 Completed my M.Sc. in Computer Science supervised by Jackie Cheung at McGill University and Mila. Thesis: Language model pretraining with lexical semantics.