introduction
Synaesthesia is a phenomenon in which one sensory modality is stimulated by input, but another simultaneously receives input, generating the impulse to “hear colors, feel sounds, and taste shapes.” This cross-modal activation is automatic and unconscious, perceptual rather than cognitive. Some synaesthetes assign personalities to letters or numbers, while others view musical tones with specific hues. Approximately 4.4% of the global population has some form of synaesthesia, and the most common form of synaesthesia is grapheme-color synaesthesia (GCS), where patients map colors onto individual letters or numbers. A grapheme is the smallest meaningful contrastive unit in a language. This synaesthesia can extend to the entire word followed by the letter as well –– for example, if they see the letter ‘A’ as red, they might also perceive the word ‘ant’ as red. Some associations are more common than others, but typically, synaesthetes maintain their own color-grapheme vocabularies. This suggests that synaesthesia and language are intrinsically interlinked.
Synaesthetic extra-sensory information has been shown to be incorporated into lexical identity. This means that when we retrieve the word’s meaning, we also retrieve the color alongside it. One study administered the Stroop Color and Word Test to synaesthetes and tested them on their recall of incongruently colored digits (compared to their own color associations) compared to congruently colored digits or black-and-white digits. The Stroop Color and Word Test provides color words that are visually a different color –– for example, the word red but in blue ink. Participants consistently demonstrated poorer recall for the incongruently colored digits compared to the other two cases. These results suggest that synaesthetic associations influence one’s memory of language (Smilek et al., 2002). When it came to both memory encoding and memory retrieval, it made a difference to the participant on whether the stimulus presented matched the color-grapheme association they had in their mind.
This table shows which types of synaesthesia are proportionally the most prevalent in the U.S.
The above table reflects the distribution of types of synaesthetes. Grapheme-color synaesthetes form 68.8% of synaesthetes polled, with phoneme-color synaesthetes forming another 10.6%. It should be noted that there is overlap because 40% of individuals exhibit multiple types of synaesthesia. Overall, however, the association of language with color is not uncommon amongst those who experience this phenomenon (Ward and Cytowic, 2006).
(Interestingly, numbers, musicla notes and occasionally, punctuation, Braille, and musical symbols can automatically evoke color sensations.) However, 68.8% of individuals with synaesthesia associate graphemes with colors (Ward and Cytowic, 2006). This is a vast majority, and suggests something unique about language processing and synaesthesia. Where does synaesthesia come from, and how does it affect our processing and acquisition of language? Let’s take a look at some of the root causes of synaesthesia, and consider what it means to form a ‘lexical entry’ for a word.
the biological basis for synaesthesia
Although synaesthesia appears to be something abstract and cognitive, tracing it through family lineages suggests that it is heritable. This was reported as early as the late 1800s. In the last few decades, modern studies have suggested a ‘synaesthesia gene’ passed down from parents to children.
Scientists David Brang and V.S. Ramachandran posit that increased connectivity between regions is responsible for synaesthesia, and this increased connectivity is influenced by a deficit in axonal pruning during fetal development due to an overexpression of a gene. Specifically, an overexpression of serotonin 2A receptor coding on chromosome 13 leads to an high receptor density on neurons in various regions of the brain. This means that the same input can lead to polarization. Let’s take a step back and review what some of these neuroscientific terms mean.
overview of neuroscientific terms
This is a diagram of a neuron – which can be thought of as the most basic communicative block of the human brain. They are known as the ‘grey matter’ of the brain because their axons are unmyelinated, giving them a grey color; this color refers to the bodies of the neurons, which are generally located in the external layer of the cortex, which is what forms the largest portion of the brain inside of the skull, and what we traditionally visualize when we think of the brain.
What connects the gray matter is ‘white matter’ in the interior of the brain. To give you a better sense of these proportions, here’s a diagram:
The white matter is composed of many bundles of axons which connect the neurons. The axons are like telephone lines, and electrical pulses pass through the axons so the neurons can communicate with one another. This communication occurs in simple binary: either they send enough electricity to activate the neuron at the end of the ‘wire,’ or they don’t. Activating is called depolarization, and a lack of activation is called polarization.
This is a close-up image of an axon ending contacting a neuron at the end of the ‘telephone line.’ At the junction between the two cells (the horizontal gap between the blue bulbs), the axon from the neuron on top is releasing neurotransmitters, and the dendrite (the receiving branch) of the following neuron has five receptors that are opening to allow these neurotransmitters through. Neurons can modulate their receptor density in live time. This means that the same amount of neurotransmitter input can polarize or depolarize the neuron, depending on the neuron’s receptor density.
back to synaesthesia
Returning to synaesthesia after that brief review of neuroscience, we can remember that these scientists are proposing that synaesthesia’s biological origin occurs in the overexpression of a receptor on chromosome 13. This leads to a deficiency in the pruning of axon-dendrite connections. At these junctions, the density of receptors directly influences how easily one neuron can depolarize the neuron it is connected to: the higher the density of receptors, the more easily it can depolarize the connected neuron, even with the same amount of input. During fetal development, there should be cells or enzymes responsible for pruning the receptors at these connective junctions, but due to the malfunction in the gene, there is less pruning, leading to an excessively high density of receptors in the axon-dendrite connections in axons specifically between sensory modalities.
This increase of receptor density only occurs for connections between, for example, visual cortex or auditory cortex, each responsible primarily for processing data from one sensory modality (Brang and Ramachandran, 2011).
So, now we know that synaesthesia may be due to hyper-connectivity between modality-specific regions in the brain. Does this mean that language-based synaesthesia is neurally based in hyperconnectivity and a greater density of white matter between language-specific regions and, for example, the visual cortex? Let’s look at what happens in the brains of synaesthetes.
Upon performing fMRI screening in a group of individuals who exhibited color-grapheme synaesthesia (associating graphemes with colors), scientists discovered that achromatic graphemes activate grapheme regions traditionally used for language in the brain. (There are language-specific regions that are activated only upon language processing, not when non-linguistic stimuli are applied, and that are not activated when visual areas of the brain are activated. This is called a double dissociation) – but they also activated color region V4.
(As a brief aside, visual information from the retinas are passed in complex ways towards V1 at the occipital, or backmost, part of the brain, which is the primary hub for processing visual data. V1 then works in fascinatingly complicated ways to distribute different ‘aspects’ of visual data to other ‘hubs’ of processing, which parse this data further: there are two pathways shown below in the diagram.)
This diagram shows the pathways that visual information takes. Two of the primary pathways are the dorsal pathway (the uppermost one, from V1 -> V2 -> MT) and the ventral pathway (the bottom one, from V1 -> V2 -> V4). The dorsal pathway is responsible for processing motion, whereas the ventral pathway is responsible for object recognition. Data from V1 is passed to V2, and then onto V4. V4 is crucial for object recognition (Bachatene et al., 2012).
Back to the brain activation of color-grapheme synaesthetes when encountering achromatic graphemes: their brain’s language regions are activated, but so is color region V4, which, in all humans, is stimulated more strongly by colorful stimuli rather than black-and-white ones. V4 is implicated in the ventral pathway of visual processing, which means it is crucial for object recognition.
But do synaesthetic individuals make the conscious decision that they want to ‘see’ these colors along with the letters? To consider that, we must understand that the ventral network – once again, the bottom-most pathway which passes information from V1 -> V2 -> V4 for object recognition – is known to utilize a bottom-up attentional process. What does this mean? Attention is a limited resource, and individuals must choose when and where to deploy it. Attention can be both “bottom-up” and “top-down”: stimuli in the world can ‘grab’ our attention and dominate our attentional spotlight from the bottom up, but we can also choose precisely where we want to deploy our attention and exert this control from the top down, selecting where we want to place that attentional spotlight. It is well-studied that object recognition is bottom-up, meaning that it is stimuli in the world that control our attention. This means that linguistic synaesthesia is a bottom-up process that is influenced by the stimuli available rather than a top-down process of placing our own cognitive contexts onto the situations at hand. It is automatic, rather than something synaesthetes actively strive for.
Another additional question: is it that the grapheme activates associated memories of the color – for example, if you played with a refrigerator magnet of a blue ‘d’ as a child, and upon seeing the grapheme, you are simply retrieving that memory? A timing study by Ramachandran and Hubbard suggested that, no, there is not enough time between the stimulus encounter and the activation of both the linguistic and the color regions for a memory to be retrieved. V4’s activation after the achromatic grapheme encounter occurred as early as 110 ms – the exact same time course as when colorful stimuli are encountered by the retina, actually visual stimuli. This means that reading an achromatic grapheme stimulates color perception the same way that real colors do, and there is not enough time elapsed for a memory to be retrieved.
pop-out effect
Even given this evidence, some still doubt the legitimacy of synaesthesia as a legitimate neural phenomenon, suggesting that these associations stem from important childhood memories that have become deeply ingrained in the child’s system. So, another study was done. Non-synaesthetic and synaesthetic subjects were shown a cluster of achromatic numbers; they were predominantly 5s, but there was a small 2 buried inside. The density of the small numbers and the overall shape of the large cluster made it extremely hard to pick out the minory numbers. Synaesthetes were able to identify these ‘hidden’ numbers with faster reaction times, seeing a pop-out effect, as compared to control subjects, because the colors were different, but the control subjects demonstrated much longer reaction times, showing the difficulty level of parsing out the numbers purely based on the numbers.
This supports the theory that synaesthesia is sensory, rather than cognitive or memory-based: rather than occurring during cognitive processing, the very encounter with the linguistic stimulus generates the color differentiation (Ramachandran and Hubbard).
numerical identity
But is it the visual look of the number, or the number’s identity? In a follow-up study to the previous one, Roman numerals and clusters of dots that could be subitized (or ‘added’) were also given to the synaesthetes, but they did not have the same effect, meaning that it is the visual grapheme of the letters and numbers that are associated with the colors (Ramachandran and Hubbard). The visual, or orthographic, forms of the graphemes trigger this color association automatically and sensorily. It’s a really fascinating effect, and one that many artists use to generate emotional art. Many experiences with poetry lead us to enter into the synaesthetic worlds of the poets, where senses are juxtaposed in unconventional ways.
Now we understand a little better where synaesthesia arises biologically, and that synaesthetes experience an automatic and parallel triggering of multiple sensory modalities. It is likely unrelated to childhood memories due to the time course of these activations, and they can pick out small numbers in a sea of other numbers due to visual pop-out of colors. So, how is it that these associations are acquired over time, and can they change over the course of an individual’s life?
grounded cognitive model
One possible way to consider the acquisition of linguistic synaesthesia first requires an understanding of the Grounded Cognitive Model of language acquisition.
Let’s look at David Kemmerer’s tenth chapter of Cognitive Neuroscience of Language. Here, he brings up the question of how it is that our abstract concepts of object nouns are acquired, developed, and made more refined over time. For example, how is it that we develop mental representations of words such as squirrel, raccoon, bicycle, and nose? When we read these words, what is attached to their lexical entries in the brain? (Synaesthesia creates connections between colors and words, but colors and graphemes, too. It is interesting – and somewhat inexplicable – how they create different ‘identities’ for graphemes, since they should not have different semantic meanings. For the purpose of this section, we will look primarily at color-word synaesthetes.)
Kemmerer proposes two models of object noun representation in the mind, and then proceeds to support one of them more fully. His first proposal is the Amodal Symbolic Model, which claims that concepts and word meanings are processed in a semantic system separate from the modality-specific systems for perception and action. For example, the concept encoded by the word banana is composed of amodal features such as [yellow], [long], and [curved]. These are features that we can observe when different bananas pass through our lives, but they are not necessarily related with how we engage or interact with the bananas.
The second model is the Grounded Cognition Model, where our comprehension of these concepts overlaps with the modalities for perception and action. Our understanding of banana then arises from how these objects are used, manipulated, and engaged within the external environment, such as [eat], [touch], and [peel] (Kemmerer 275). Studies were then performed to gauge which one was more accurate.
This figure from Kemmerer’s textbook illustrates what he means by the Grounded Cognition Model. Not only does this lexical entry contain semantic meanings such as visual elements, but reading it also activates modality-specific systems for perception and action that allow us to access the concept of a ‘banana.’
In the Grounded Cognition Model, visual, auditory, and tactile modalities are activated in a corresponding manner to motion-based and sense-based experiences of the object. It is like you are engaging with the object in memory. Kemmerer writes, “What this implies is that the meaning of an object noun like banana does not reside in any single place in the brain; instead, different fragments of this complex concept are scattered across different cortical regions according to the sensory or motor content of the type of information that is represented.”
It is not clear where synaesthesia is acquired from for most individuals. Many people have different associative maps, but it is known that they change over time. But based off of the GCM, it is likely that graphemes –– and words –– have ‘definitions’ that go beyond a pure, semantic definitions. Linguistic meaning is ‘scattered’ around the brain, and it is likely that each ‘lexical entry’ is attached to other senses in such a way. It is also interesting that the GCM posits that we primarily form these connections based on how we perceive and physically engage with the objects. This suggests that memory is required, but synaesthesia is proven not to rely on memory. However, as a concept, linguistic synaesthesia may be similar in structure to the GCM for object noun retention, where the true ‘meaning’ of a grapheme or word lies in different regions of the brain. This would explain previous results, where the activation of V4 was simultaneous to the activation of the language regions.
more on the immediacy of word-color associations
Passive and active color sensation can be dissociated into two primary regions of the brain: V4, as mentioned earlier, in the lingual gyrus of the occipital lobe as people view colored stimuli in passive color sensation, and V4ɑ, a region in the middle of the fusiform gyrus further downstream from V4 that is occurs during color comparison or analysis, engaged during active color sensation.
In Simmons et al. 2007, blood-oxygen-level imaging was applied to participants being asked to judge whether a first word such as eggplant matched a second color word such as purple. During these color property judgements, V4ɑ was activated, supporting the notion that words force people to automatically retrieve implicit conceptual features as if they were confronted with the colors themselves (Simmons et al. 2007). The color-recognizing regions of the brain were activated automatically by the words themselves. This demonstrates that language intrinsically has the capacity to evoke synaesthesia by triggering those modalities. Timing experiments were also carried out that demonstrated that the time delay was highly abbreviated, meaning that it was unlikely that the words were triggering past memories that then activated the modalities; rather, the effect was immediate.
[Blood-oxygen-level-imaging, or BOLD imaging, works by using a machine to track how much blood is flowing to one area of the brain. It is like one hyper-active region of the brain demanding more energy, so BOLD imaging shows which regions of the brain are activated.]
synaesthesia & aging
Synaesthesia is innate, and attaches multiple sensory modalities to the ‘definition’ of a grapheme or a word. We also know that we can acquire language over time, and so as we create new lexical entries for graphemes or words, how does that affect the mapping of other senses?
As an individual’s language develops and becomes more complex over time, synaesthesia may also shapeshift. Studies have shown that primary colors and their associations to graphemes remain stable during the process of aging, but showy and gaudy colors decline over time, while brown and paler shades occur more frequently in old age. Even though visual perception declines as age increases, synaesthesia is more related to retrieving from memory (Meier et al., 2014).
Infants exhibit synaesthesia due to lack of neural pruning and hyper-connectivity between regions that are spatially adjacent, and the hereditary nature suggests that the innate synaesthesia that infants begin their lives with may be influenced by connective weights existing in their parents strengthened over time via experience.
synaesthesia & art
Metaphors and similes comparing one object or idea to another are common to creative writing, and ekphrastic poetry relies on the translation of the experience of engaging with visual art into literary text. However, synaesthesia extends beyond the associated concepts that are inherent within one word, creating a tactile, aural, and otherwise sensory landscape. Looking at the experiences of creative writers in linguistic synaesthesia offers insight into the synaesthetic landscape in writer’s heads that allow them to create visions on the page that feel immersive, and emotional, to others. There is something, then, in the juxtaposition of sensory elements that results in emotional experiences for a reader. Perhaps it encourages them to make connections between concepts in new ways.
One writer who exhibits synaesthesia is Vladimir Nabokov. He described the “English a” as tinted like weathered wood, of the “French a” like polished ebony, of “o” like an ivory-backed hand mirror, of “e’s” and “i’s” as yellows. He realizes that “English u” differs; he describes what he calls its “alphabetical value” as “brassy with an olive sheen” (Reichard et al.). The correspondence of sounds to specific textures such as wood and ebony suggest that sounds have textures –– rough, smooth, round, small –– that are translated into synaesthesia.
The non-syntacticality of poetry relies on the ability of the everyday individual to form synaesthetic associations. Poetry juxtaposes familiar objects in unfamiliar settings, personifying that which is inanimate, giving texture, color, and sound to figures that typically lack them, to make expressive arguments about the poet’s attitude towards the world. Analyzing poetry that has had wide-ranging effects on public audiences demonstrates the capacity of the masses to decipher synaesthetic experiences, to take part in the recognition of the multitudinous associations between the senses.
gertrude stein
One poet whose work exemplifies synaesthesia is Gertrude Stein, whose literary style is unique. Rather than focusing upon plot, setting, or genre, factors which distinguish many other authors from the pool of other writers, Stein’s work flourishes in the language itself. In Natalie Cecire’s “Ways to Not Read Gertrude Stein,” she describes Stein’s style as “chaotic, unsystematic, and virtually impossible to read.” Other critics have noted that her work toes the fine line between “genius and fraud.” Indeed, her style is difficult to parse: grammatically, her work follows the rules necessary to be diagrammed cleanly on top of a tree, but lexically, the words often devolve into synaesthesia (Cecire).
For example, in her poem, “Tender Buttons,” which meanders through a list of household objects but gives them radically deconstructed descriptions that are impossible to paraphrase.
“Tender Buttons” opens with:
A carafe, that is a blind glass.
A kind in glass and a cousin, a spectacle and nothing strange a single hurt color and an arrangement in a system to pointing. All this and not ordinary, not unordered in not resembling. The difference is spreading.
By relying on synaesthetic associations rather than semantic meanings, Stein creates a new form of reading that is individual for every reader, gesturing to the individuality of the domestic vocabulary we have each created. Critics have described her work as “echolalia-like incantations, half-witted sounding catalogs of numbers,” and physics professor Langdon Brown noted that her work appeared to be “fished up from the unconscious… unshaped by intellect” (Cecire). Moving from line to line, she crafts meaningful compositions of the female unconscious and draws us deeper towards our own perspective of a traditional domestic milieu. Her broken language signifies an ability to experiment and break traditional logic in a way that challenges the reader and leaves one wondrous at their own potential for linguistic comprehension. She at once asserts the unreadability of her own work, but then provokes the reader to engage; the immediate incomprehensibility seems to suggest both an infinite distance, but also the closest intimacy one can achieve to unvarnished thought. Without traditional logic, how do people attain meaning from poetry? It is through synaesthesia –– the association of words with a network of senses, and how those networks can connect to one another to form shifting narratives.
Semantic priming is a phenomenon based on the observation that individuals have a faster response time comprehending words when they were previously presented with a related word, connected to the first word in a semantic network, as shown above. For example, if given the word rainforest, and then given a choice between the words tree and computer, they would be more likely to choose the word tree. Semantic priming effects can be moderated by individual reading skill and attentional control (Yap et al., 2016). For synaesthetes, it is possible that words and letters also gain colors that are linked to their maps, as Nabokov noted.
In Stein’s work, synaesthetically semantic networks of individual words build up a subconscious map drawing upon our universal understandings of domesticity, intimacy, and wifehood. The cognition of poetry, which utilizes non-traditional syntax and is often not possible to parse into a syntactic tree, works based on these overlaps of semantic and synaesthetic assumptions. Stein’s work draws us closer to the unconscious, to the interior, of the individual, attempting a radical form of intimacy which erases the barrier of the formality of English prose. Synaesthesia radically unites the senses and creates possibilities of texts like these.
The unification of the senses is not only an effect of art; it is perhaps one of the core experiences that it evokes, allowing the reader to transcend their body to experience new cross-modal connections of the senses. Synaesthetes experience this on an immediate level; although the language faculty has been doubly dissociated from visual and motor forms of perception, as Braille readers and deaf individuals can communicate linguistically and experience activation of the same language centers as hearing individuals, it seems to maintain its connectivities.
Synaesthesia, as described above, is a highly human and creative effort. It brings a reader into the rich folds of an associative mapping that can only be made by a human. I was curious: if I took all the words from Gertrude Stein’s famous poem, “Tender Buttons,” and input it into a machine that replicates her writing, will human readers be able to tell the difference if the sentences were written by a machine or a human?
There is something distinctly human about Stein’s vocabulary. A survey of undergraduate swere polled on whether or not they believed that language produced by Stein or the Gertrude-Stein-machine was human or not, and the result was: they could not tell the difference. This is critical: the synaesthetic lexicon of her work, juxtaposing different senses with objects and abstract concepts, was capable of creating a human, emotional effect – even when the syntax was completely disrupted and jumbled around.
There is something inherently human about synaesthesia and language that can be reproduced in a machine by the words used. Stein’s language itself is a fabric of associations mined from the domestic landscape that draw upon ties we have built up slowly and gradually over our lives. Yet she points to us to draw a coherent narrative out of them. In her work, storytelling incorporates our identities, too. And, in the end, neither a machine nor cognitive science even approximate a careful consideration of Stein’s language. I aimed to utilize these findings as the basis for a literary analysis.
¿linguistics?
Works Cited
Bachatene, Lyes & Bharmauria, Vishal & Molotchnikoff, Stephane. (2012). Adaptation and Neuronal Network in Visual Cortex. 10.5772/46011.
Brang D, Ramachandran VS. Survival of the synesthesia gene: why do people hear colors and taste words? PLoS Biol. 2011 Nov;9(11):e1001205. doi: 10.1371/journal.pbio.1001205. Epub 2011 Nov 22. PMID: 22131906; PMCID: PMC3222625.
Cecire, N. (2015). Ways of Not Reading Gertrude Stein. ELH, 82(1), 281–312. http://www.jstor.org/stable/24477816
Cytowic, Richard, and Jamie Ward. “Synesthesia and Language .” Research Gate, Elsevier, 2018, https://www.researchgate.net/publication/327108549_Synesthesia_and_Language.
Gladys A.Reichard, Roman Jakobson & Elizabeth Werth (1949) Language and Synesthesia, WORD, 5:2, 224-233, DOI: 10.1080/00437956.1949.11659507
Jones, Michael N., et al. “Individual Differences in Semantic Priming Performance.” Big Data in Cognitive Science, Routledge, London, New York, 2017.
Kemmerer, David. Cognitive Neuroscience of Language. Routledge, Taylor & Francis Group, 2023.
Meier B, Rothen N, Walter S. Developmental aspects of synaesthesia across the adult lifespan. Front Hum Neurosci. 2014 Mar 11;8:129. doi: 10.3389/fnhum.2014.00129. PMID: 24653689; PMCID: PMC3949321.
O’Grady, Cathleen. “Synaesthesia Could Help Us Understand How the Brain Processes Language .” The Guardian, Guardian News and Media, 26 Feb. 2016, https://www.theguardian.com/science/head-quarters/2016/feb/26/synaesthesia-could-help-us-understand-how-the-brain-processes-language.
Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia–a window into perception, thought and language. Journal of Consciousness Studies, 8(12), 3–34.
Simmons, William & Ramjee, Vimal & Beauchamp, Michael & McRae, Ken & Martin, Alex & Barsalou, Lawrence. (2007). A common neural substrate for perceiving and knowing about color. Neuropsychologia. 45. 2802-10. 10.1016/j.neuropsychologia.2007.05.002.
Smilek, Daniel, et al. “Synesthetic Color Experiences Influence Memory .” Sage Journals, https://journals.sagepub.com/doi/10.1111/1467-9280.00496.
Stein, Gertrude. “Tender Buttons .” Poets.org, Academy of American Poets, https://poets.org/poem/tender-buttons-objects.
Wassiliwizky E, Koelsch S, Wagner V, Jacobsen T, Menninghaus W. The emotional power of poetry: neural circuitry, psychophysiology and compositional principles. Soc Cogn Affect Neurosci. 2017 Aug 1;12(8):1229-1240. doi: 10.1093/scan/nsx069. PMID: 28460078; PMCID: PMC5597896.