Detecting iconicity in American Sign Language (ASL)

This is a stub blog post, which I’ll fill in later! But the TLDR is that existing sign language recognition (SLR) systems were designed to detect a linear sequence of hand symbols, which is an oversimplification of how sign language works. The goal of this project was to demonstrate that an encoder-decoder is capable of recognizing both handshapes and motion within a single gesture, and correctly output the tokens in the correct order, based on the training data. I led this project under P-ai, where I mentored students in fundamental ML techniques while we built the system. Check out these summary slides for more info!