Chawin Sitawarin

The following bio is AI-generated by AlphaArXiv.

I'm a ~~Research Scientist at UC Berkeley~~ postdoc at Meta (Central Applied Science) focusing on machine learning security and robustness. My research interests span adversarial machine learning, prompt injection attacks, and automated security analysis of ML systems. I work closely with Professor David Wagner and collaborate with researchers from various institutions including Google Research.

My recent work has focused on developing novel attacks and defenses for large language models, particularly in the area of prompt injection attacks. I led the development of PAL, a proxy-guided black-box attack on LLMs, and StruQ, a defense against prompt injection using structured queries. I've also made contributions to improving adversarial robustness of computer vision models through techniques like part-based models and random transformation defenses.

Prior to Berkeley, I worked on problems in autonomous vehicle security at Princeton University, where I demonstrated vulnerabilities in traffic sign recognition systems. I received my B.S. in Electrical Engineering from Princeton University, where I also conducted research in nanophotonics and inverse design optimization. My work has been published at top security and machine learning venues including ICLR, ICML, CCS, and NeurIPS.

Publications

Vulnerability Detection with Code Language Models: How Far Are We?

Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, Yizheng Chen

arXiv.org 2024

ABS HTML PDF

PAL: Proxy-Guided Black-Box Attack on Large Language Models

Chawin Sitawarin, Norman Mu, David Wagner, Alexandre Araujo

arXiv.org 2024

ABS HTML PDF

StruQ: Defending Against Prompt Injection with Structured Queries

Sizhe Chen, Julien Piet, Chawin Sitawarin, David Wagner

arXiv.org 2024

ABS HTML PDF

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, David Wagner

European Symposium on Research in Computer Security 2023

Chawin Sitawarin

Publications

Vulnerability Detection with Code Language Models: How Far Are We?

PAL: Proxy-Guided Black-Box Attack on Large Language Models

StruQ: Defending Against Prompt Injection with Structured Queries

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

Mark My Words: Analyzing and Evaluating Language Model Watermarks

Defending Against Transfer Attacks From Public Models

OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift

SPDER: Semiperiodic Damping-Enabled Object Representation

REAP: A Large-Scale Realistic Adversarial Patch Benchmark

Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems

Part-Based Models Improve Adversarial Robustness

Demystifying the Adversarial Robustness of Random Transformation Defenses

Adversarial Examples for k-Nearest Neighbor Classifiers Based on Higher-Order Voronoi Diagrams

SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothing

Improving Adversarial Robustness Through Progressive Hardening

Minimum-Norm Adversarial Examples on KNN and KNN based Models

Analyzing the Robustness of Open-World Machine Learning

Defending Against Adversarial Examples with K-Nearest Neighbor

Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples

Inverse-designed photonic fibers and metasurfaces for nonlinear frequency conversion [Invited]: publisher’s note

On the Robustness of Deep K-Nearest Neighbors

Not All Pixels are Born Equal: An Analysis of Evasion Attacks under Locality Constraints

DARTS: Deceiving Autonomous Cars with Toxic Signs

Rogue Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logos

Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars

Inverse designed photonic fibers and metasurfaces for nonlinear frequency conversion

Enhancing robustness of machine learning systems via data transformations

Inverse-designed nonlinear nanophotonic structures: Enhanced frequency conversion at the nano scale

Short: Certifiably Robust Perception Against Adversarial Patch Attacks: A Survey

OODRobustBench: benchmarking and analyzing adversarial robustness under distribution shift

Improving the Accuracy-Robustness Trade-Off for Dual-Domain Adversarial Training

M ITIGATING A DVERSARIAL T RAINING I NSTABILITY WITH B ATCH N ORMALIZATION

1 DARTS : Deceiving Autonomous Cars with Toxic Signs

0 DARTS : Deceiving Autonomous Cars with Toxic Signs