Manual gestures occur on a continuum from co-speech gesticulations to conventionalized emblems to language signs. Our goal in the present study was to understand the
neural bases of the processing of gestures along such a continuum. We studied four types of gestures, varying along linguistic and semantic dimensions: linguistic and meaningful
American Sign Language (ASL), non-meaningful pseudo-ASL, meaningful emblematic, and nonlinguistic, non-meaningful made-up gestures. Pre-lingually
deaf, native signers of ASL participated in the
fMRI study and performed two tasks while viewing videos of the gestures: a visuo-spatial (identity) discrimination task and a category discrimination task. We found that the categorization task activated left
ventral middle and
inferior frontal gyrus, among other regions, to a greater extent compared to the
visual discrimination task, supporting the idea of semantic-level processing of the gestures. The reverse contrast resulted in enhanced activity of bilateral
intraparietal sulcus, supporting the idea of featural-level processing (analogous to phonological-level processing of speech
sounds) of the gestures. Regardless of the task, we found that
brain activation patterns for the nonlinguistic, non-meaningful gestures were the most different compared to the ASL gestures. The
activation patterns for the emblems were most similar to those of the ASL gestures and those of the pseudo-ASL were most similar to the nonlinguistic, non-meaningful gestures. The
fMRI results provide partial support for the conceptualization of different gestures as belonging to a continuum and the
variance in the
fMRI results was best explained by differences in the processing of gestures along the semantic dimension.
