Maksym Andriushchenko
Email   Twitter/X   Google Scholar   GitHub   CV
Short bio. I’m a fifth-year PhD student in computer science at EPFL 🇨🇠advised by Nicolas Flammarion. My research is supported by the Google and Open Phil AI PhD Fellowships. I did my MSc at Saarland University and the University of Tübingen, and interned at Adobe Research.
Research interests. My primary research goal is to understand robustness and generalization in deep learning. Toward this goal, I’ve worked on adversarial robustness, out-of-distribution generalization, implicit regularization, and sharpness-aware minimization. These days, I’m focusing almost entirely on various aspects of large language models, such as optimization, alignment, memorization, and robustness. My complete publication list is available here.
On Ukraine. Since I’m from Ukraine, I’m often asked about the situation in my country and how one can help. The most effective way is to donate to local Ukrainian organization helping on the ground, e.g., see this list which includes both trusted military and humanitarian organizations. You can also host displaced scholars and students from Ukraine, e.g., see the #ScienceForUkraine project where I’m involved as a volunteer. You can also help simply by spreading the word about the war and going to demonstrations in your city. It’s very important that we don’t normalize annexations of territories, numerous war crimes, mass deportations, and nuclear threats. Otherwise, we’ll end up in a world we don’t really want to be in.
selected publications
M. Andriushchenko, F. Croce, N. Flammarion. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks (arXiv, Apr 2024)
H. Zhao, M. Andriushchenko, F. Croce, N. Flammarion. Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning (arXiv, Feb 2024)
M. Andriushchenko, A. Varre, L. Pillaud-Vivien, N. Flammarion. SGD with Large Step Sizes Learns Sparse Features (ICML 2023)
M. Andriushchenko, N. Flammarion. Towards Understanding Sharpness-Aware Minimization (ICML 2022)
F. Croce*, M. Andriushchenko*, V. Sehwag*, E. Debenedetti*, N. Flammarion, M. Chiang, P. Mittal, M. Hein. RobustBench: a standardized adversarial robustness benchmark (NeurIPS 2021 Datasets and Benchmarks Track, Best Paper Honorable Mention Prize at ICLR’21 Workshop on Security and Safety in ML Systems)
M. Andriushchenko, N. Flammarion. Understanding and Improving Fast Adversarial training (NeurIPS 2020)
M. Andriushchenko*, F. Croce*, N. Flammarion, M. Hein. Square Attack: a query-efficient black-box adversarial attack via random search (ECCV 2020)
news
| Apr 2, 2024 | Our new paper Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks is available online (see a Twitter/X thread for a summary). We show how to jailbreak basically all leading safety-aligned LLMs with ≈100% success rate. |
|---|---|
| Mar 28, 2024 | Our new benchmark JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models is available online (see a Twitter/X thread for a summary). We prioritize reproducibility, support adaptive attacks, and test-time defenses. |
| Feb 15, 2024 | A talk at the Math Machine Learning seminar MPI MIS + UCLA about our paper A modern look at the relationship between sharpness and generalization. Slides: pdf, pptx. |
| Feb 7, 2024 | Our new paper Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning is available online (see a Twitter/X thread for a summary). We need simple methods to better understand alignment.  |
| Jan 16, 2024 | Layer-wise Linear Mode Connectivity got accepted to ICLR 2024. See (some of) you in Vienna! |
| Jan 5, 2024 | A talk at the Deep Learning: Classics and Trends (organized by ML Collective) about our recent work Why Do We Need Weight Decay in Modern Deep Learning? (slides) |
| Dec 21, 2023 | A new short paper Adversarial Attacks on GPT-4 via Simple Random Search on how we can leverage logprobs for a black-box attack on the latest GPT-4-turbo (see a Twitter/X thread for a summary).  |
| Dec 10, 2023 | Going to NeurIPS’23 in New Orleans. Feel free to ping me if you want to chat! |
| Nov 14, 2023 | A talk at the Deep Learning and Optimization Seminar (organized by faculties from Westlake University, City University of Hong Kong, Peking University) about our recent work Why Do We Need Weight Decay in Modern Deep Learning? |
| Nov 9, 2023 | A talk at the University of Tübingen about our recent work Why Do We Need Weight Decay in Modern Deep Learning? |
| Oct 30, 2023 | A talk at the Efficient ML Reading Group (organized by TU Graz) about our recent work Why Do We Need Weight Decay in Modern Deep Learning? |
| Oct 23, 2023 | Excited to have participated in red teaming of OpenAI models as an external expert! I hope my findings will help improving the safety of their models/services. |
| Oct 9, 2023 | Our new paper Why Do We Need Weight Decay in Modern Deep Learning? is available online. Also check out our new preprint on layer-wise linear mode connectivity.  |
| Sep 21, 2023 | Both Sharpness-Aware Minimization Leads to Low-Rank Features and Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings got accepted to NeurIPS 2023! See y’all in New Orleans! 🎶🎷 |
| Aug 23, 2023 | A talk at the ELLIS Mathematics of Deep Learning reading group about our ICML 2023 paper SGD with Large Step Sizes Learns Sparse Features. Slides: pdf, pptx. |
| Jul 23, 2023 | Going to ICML 2023 in Hawaii to present SGD with Large Step Sizes Learns Sparse Features and A Modern Look at the Relationship Between Sharpness and Generalization at the main track and Sharpness-Aware Minimization Leads to Low-Rank Features at a workshop. Feel free to ping me if you want to chat! |
| Jul 21, 2023 | A talk at the Tatsu’s lab group meeting at Stanford about our ICML 2023 paper A modern look at the relationship between sharpness and generalization. Slides: pdf, pptx. |
| Jun 5, 2023 | A talk at the Efficient ML Reading Group (organized by TU Graz) about our ICML 2023 paper A modern look at the relationship between sharpness and generalization. Slides: pdf, pptx. |
| May 30, 2023 | A talk at a mini-symposium of the 93rd Annual Meeting of the International Association of Applied Mathematics and Mechanics about our ICML 2022 and ICML 2023 papers on robustness/flatness in the parameter space. |
| May 26, 2023 | Our new paper Sharpness-Aware Minimization Leads to Low-Rank Features is available online! We investigate the low-rank effect of SAM which occurs in a variety of settings (regression, classification, contrastive learning) and architectures (MLPs, CNNs, Transformers).  |
| May 5, 2023 | A talk at the Amazon Research Reading Group about our ICML 2023 paper A modern look at the relationship between sharpness and generalization. Slides: pdf, pptx. |
| Apr 25, 2023 | Both SGD with large step sizes learns sparse features and A modern look at the relationship between sharpness and generalization got accepted to ICML 2023! See you in Hawaii! 🌴 |
| Apr 12, 2023 | A talk at the Deep Learning and Optimization Seminar (organized by faculties from Westlake University, City University of Hong Kong, and Peking University) about our paper SGD with large step sizes learns sparse features. Slides: pdf, pptx. |
| Mar 13, 2023 | A talk at the OOD Robustness + Generalization Reading Group at CMU about our paper A modern look at the relationship between sharpness and generalization. Slides: pdf, pptx. |
| Feb 15, 2023 | Our new paper A modern look at the relationship between sharpness and generalization is available online! Do flatter minima generalize better? Well, not really.  |
| Dec 9, 2022 | A talk at the University of Luxembourg about our work with Adobe: ARIA: Adversarially Robust Image Attribution for Content Provenance. |
| Dec 1, 2022 | A talk in the ML and Simulation Science Lab of the University of Stuttgart about RobustBench and SGD with large step sizes learns sparse features. |
| Nov 28, 2022 | Going to NeurIPS’22 in New Orleans. Feel free to ping me if you want to chat! |
| Oct 28, 2022 | A talk at the ELLIS Mathematics of Deep Learning reading group about our ICML’22 paper Towards Understanding Sharpness-Aware Minimization. Slides: pdf, pptx. |
| Oct 12, 2022 | Our paper SGD with large step sizes learns sparse features is available online! TL;DR: loss stabilization achieved via SGD with large step sizes leads to a hidden dynamics that promotes sparse feature learning. Also see this twitter thread for a quick summary of the main ideas.  |
| Oct 7, 2022 | Recognized as one of the top reviewers at NeurIPS’22. Yay! 🎉 |
| Sep 7, 2022 | A talk at Machine Learning Security Seminar hosted by University of Cagliari about our paper ARIA: Adversarially Robust Image Attribution for Content Provenance (available on youtube). |
| Sep 1, 2022 | Truly excited to be selected for the Google PhD fellowship and OpenPhil AI fellowship! |
| Jun 13, 2022 | Our paper Towards Understanding Sharpness-Aware Minimization got accepted to ICML’22!  |
| Apr 1, 2022 | Our paper ARIA: Adversarially Robust Image Attribution for Content Provenance is accepted to the CVPR’22 Workshop on Media Forensics. One of (a few?) applications where \(\ell_p\) adversarial robustness is well-motivated from the security point of view.  |
| Mar 25, 2021 | A talk at the NLP club of Grammarly about our paper On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (available on youtube). |