Language. Vision. A little curiosity.

Joel Ruben Antony Moniz

Staff Machine Learning Engineer · Tech Lead · DoorDash

I work on language and multimodal learning, with interests in assistants, efficient learning, and grounded AI.

Explore my research

Joel Ruben Antony Moniz wearing a blue suit outdoors

01 Selected work

A few things I’ve worked on.

From understanding a conversation to navigating the world. A selection of my research, old and new.

ReALM examples encode conversational turns and on-screen entities for reference resolution.

SIGDIAL2024

ReALM: Reference Resolution as Language Modeling

Resolving references across conversation and on-screen content by expressing them as a language modeling problem.

A sequence of Milo guiding a handler across a bridge in a park.

Preprint2026

Milo, a Fully Autonomous Indoor/Outdoor Robotic Guide Dog

An autonomous robotic guide dog that helps people navigate indoor and outdoor environments.

The SEA-VL pipeline combines crowdsourcing, web crawling, and generation to collect culturally grounded data.

ACL2025

Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia

Exploring how to build vision-language data that reflects the cultures and languages of Southeast Asia.

Green and pink heatmaps show correlations among hallucination evaluation metrics.

Findings of EMNLP2025

Evaluating Evaluation Metrics – The Mirage of Hallucination Detection

Examining how well automatic hallucination metrics agree with human judgments—and with one another.

Five stages of the SynthDST synthetic-dialogue generation pipeline.

EACL2024

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking

Generating synthetic dialogues to improve dialogue state tracking when labeled examples are scarce.

MARRS architecture connects query rewriting with mention detection and resolution.

CRAC2023

MARRS: Multimodal Reference Resolution System

Bringing conversational, visual, and background context together in an on-device reference resolution system.

The 5IDER architecture uses a shared encoder and task-specific edit predictors to rewrite queries.

Interspeech2023

5IDER: Unified Query Rewriting for Steering, Intent Carryover, Disfluencies, Entity Carryover and Repair

A unified editing approach for five ways people revise, extend, and correct requests to voice assistants.

Classification performance curves compare annotation acquisition strategies over successive rounds.

ACL2022

On Efficiently Acquiring Annotations for Multilingual Models

Choosing where to spend an annotation budget to learn more effectively across languages.

CREAD jointly detects mentions, resolves references, decides whether to rewrite, and generates a new query.

NAACL2021

CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues

Jointly resolving references and rewriting incomplete follow-up requests into self-contained queries.

Three colored point distributions illustrate the difficulty of matching non-isometric embedding spaces.

ACL2019

Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces

Learning bilingual dictionaries with a little supervision, even when embedding spaces have different shapes.

The DepthNet architecture and face-rotation examples show how predicted depth changes facial pose.

NeurIPS2018

Unsupervised Depth Estimation, 3D Face Rotation and Replacement

Learning depth from facial keypoints for 3D face rotation and replacement without ground-truth depth supervision.

A nested LSTM cell places an inner memory cell inside the outer recurrent cell.

ACML2017

Nested LSTMs

Giving recurrent networks an inner memory cell to learn how to manage longer-term information.

Keep exploring

There’s more
where that came from.

Find the full collection of papers, collaborations, and new work on Google Scholar.

More research

02 Beyond the papers

I like…

Research