Research Projects
Replications, reports, and original research that explore trustworthy, efficient, and accessible machine learning.
Sparse Representations Improve Adversarial Robustness of Neural Network Classifiers
Authors: Killian Steunou, Théo Druilhe, Sigurd Saue
arXiv Preprint GitHub RepositoryAbstract: Deep neural networks perform remarkably well on image classification tasks but remain fragile under adversarial perturbations. We revisit linear dimensionality reduction as a data-adapted defense and compare standard Principal Component Analysis (PCA) with Sparse PCA (SPCA) as front-end feature extractors. The project combines theory and practice: we derive robustness certificates for linear heads in both L2 and Lâ threat models and show that sparsity tightens Lipschitz bounds for non-linear heads, thereby lowering input sensitivity. Empirically, SPCA paired with a lightweight classifier degrades more gracefully than PCA under strong white-box and black-box attacks while keeping competitive clean accuracy.
Score-Based Generative Neural Networks for Large-Scale Optimal Transport
Authors: Max Daniels, Tyler Maunu, Paul Hand
Original Paper Download Report GitHub RepositoryAbstract: We reproduced the hybrid approach that injects score-based generative models into regularized optimal transport to handle large-scale datasets. Classical Sinkhorn-style regularization accelerates computation but tends to blur transport maps. By parameterizing dual variables with neural networks and sampling through Langevin dynamics (the SCONES framework), the method recovers sharper couplings without exploding compute budgets. Our experiments on synthetic distributions confirm the reported gains over barycentric projection baselines and highlight how regularization strength and sampler noise control the quality of the recovered maps.
Test Time Training with Masked Autoencoders
Authors: Yossi Gandelsman, Yu Sun, Xinlei Chen, Alexei A Efros
Original Paper Download Report GitHub RepositoryAbstract: Test-time training (TTT) adapts models on the fly to fight distribution shift. We evaluated TTT-MAE, which pairs Masked Autoencoders with TTT, on the ImageNet-C benchmark. The method consistently improved robustness across all corruption types and highlighted an interesting failure mode: the reconstruction and classification objectives can decouple if the masking ratio is poorly tuned. We also explored an online variant that keeps encoder weights between samples and observed steady cumulative improvements, suggesting that lightweight adaptation during deployment is a practical path forward.
Are Generative Classifiers More Robust to Adversarial Attacks?
Authors: Yingzhen Li, John Bradshaw, Yash Sharma
Original Paper Download Report GitHub RepositoryAbstract: We revisited the question âAre generative classifiers more robust to adversarial attacks?â by reimplementing the original MNIST experiments and extending them to the German Traffic Sign Recognition Benchmark. Despite the theoretical appeal of generative models, our empirical evaluation under black-box attacks could not establish a clear robustness advantage over carefully tuned discriminative baselines, reinforcing how nuanced adversarial robustness claims can be.
Toxic Gas Characterization
Authors: Killian Steunou
Abstract: Toxic gas characterization is hampered by sensor drift caused by humidity shifts between training and deployment. We benchmarked a range of strategiesâfrom Random Forests and XGBoost to a custom RAMTNet multi-task networkâand combined them with adversarial domain adaptation. Simulating test humidity inside the training split and using a two-stage classification/regression pipeline delivered the best trade-off, reaching a weighted RMSE of 0.154 (vs. the 0.157 baseline) while remaining stable across humidity regimes.
Download Report GitHub RepositoryConvergence of SGD for Training Neural Networks with Sliced Wasserstein Losses
Authors: Eloi Tanguy
Original Paper Download Report GitHub RepositoryAbstract: We verified the convergence guarantees presented for training neural networks with sliced Wasserstein losses by recreating the experiments on both 2D distributions and Fashion-MNIST. The study also covers Noise Projected SGD, an alternative optimizer that projects gradient noise onto the tangent space to stabilize updates. Our reproduction confirms the empirical convergence rates claimed in the paper and highlights scenarios where NPSGD enjoys faster progress than vanilla SGD.
An End-to-End Transformer Model for 3D Object Detection
Authors: Ishan Misra, Rohit Girdhar, Armand Joulin
Original Paper Download ReportAbstract: 3DETR extends DETRâs transformer architecture to 3D point clouds with minimal inductive bias and without the need for hand-crafted proposals or pretraining. We reproduced the method, evaluated it on SUN RGB-D, and studied how query count impacts performance. We also prototyped an RGB-enhanced variant that consumes colorized point clouds. The experiments confirmed that a streamlined transformer pipeline can remain competitive with heavily engineered 3D detectors.