American Synesthesia Association
Previous Conference Abstracts
The Twelfth National Conference of the American Synesthesia Association took place on October 6 through October 8, 2017 at Harvard University in Cambridge, Massachusetts.
Photo Credit Giselle Grenier, Entrance to Geo Hall
Photo Credit Carter Jones, Geo Hall
Photo Credit Carter Jones, Takao K. Hensch presenting
We are pleased to announce our Keynote Speaker was
Dr. Takao K. Hensch
Professor, Molecular and Cellular Biology
Professor, Neurology (Children's Hospital)
Center for Brain Science
Harvard University
Abstracts
Sérgio Basbaum, Ponticia Universidade Católica de São Paulo, Brazil
Sense and Meaning: Body Senses and Language in a Synesthetic Key
In 2010, I spent a year in post-doc research work supervised by Prof. Dr. Alfredo Pereira Jr., a leading Brazilian researcher in Consciousness studies. This resulted in an empirical work article about verbal descriptions of ambiguous images (Basbaum et al, 2012) and a lengthy research report with the grounds for a personal approach to the relations between perception and language, based on Merleau-Ponty’s Phenomenology of perception and several other references - from the neurobiology of Maturana and Varela to the philosophy of Vilém Flusser and the anthropology of the senses of Classen and Howes. Presented in the 2011 “Towards a science of consciousness” in Sweden, but still unpublished, this report turned out, however, to be a cornerstone to more than two decades of research work I've been doing about the senses. In 2015, when asked to teach classes on the “fundamentals of body communication”, in a body arts undergraduate course (theatre, dance, performance students) in Brazil, it became the basis of a methodology to discuss and organize the relations between body, perception, gesture, language and sense. Departing from Merleau-Ponty’s understanding of perception, contemporary works on synesthesia by Marks, Hubbard and others, and Horst Ruthroff's semantics of the body, the students were asked to develop works in which synesthetic correspondences between abstract graphemes, colors, and pseudo-words (in the “kiki-bouba” style) become predicates to gain insight and understanding on a broader meaning of body gestures. Through collaboration with choreographer and dancer Camila Venturelli, in 2016, this approach has been refined and applied to other groups of students in undergraduate and post graduate levels, turning into a seed for an original and revealing methodology for body and sense understanding and self-discovery, which combines senses and language in a new frame.
Greta Berman, The Juilliard School, New York City
There's Something About Synesthesia
It used to be common knowledge that synesthesia defied any systematic study due to its idiosyncratic nature; every synesthete had his or her own distinct sense associations.
This, of course, remains true to some extent. However, recent investigations of art works and musical compositions have also demonstrated astonishing commonalities.
Upon entering Pace Gallery’s exhibition of David Hockney’s iPad drawings last year, I found myself in a synesthetic “world”, one in which I was strongly reminded of the works of other known synesthetic artists: Joan Mitchell, Charles Burchfield, Carol Steen, and Marcia Smilack, as well as Munch, Van Gogh, and August Strindberg (the latter three possible synesthetes). It all had to do with questions of personification, movement, speed, and patterning - and, in Strindberg's case - his forests and cliffs.
Previous presentations by others and myself have stressed the presence of Kluverian forms in some of these artists' works. I have found similar shared characteristics in musical synesthesia. Musician synesthetes I have interviewed report seeing zigzags, repeated and circular radiating patterns, colors, and note placements that resemble what visual artists have described.
By means of analyzing specific paintings, musical compositions, and performance, this paper shall further explore commonalities among visual and musical synesthesia. It poses - and attempts to answer - the question of what it is that makes an artistic work look or sound synesthetic.
Christin Deja, Friedrich-Schiller University, Jena, Germany
Synaesthesia in the Literary Works of Wilhelm H. Wackenroder and Ludwig Tieck
I want to point out, how synesthesia as a phenomenon was literarily implemented in Heinrich Wilhelm Wackenroder's and Ludwig Tieck's composite works Outpourings of an Art-Loving Friar (1796) and Fantasies on Art (1798) and which indications for the main Ideas of the German Romanticism could be drawn from it.
Since the Outpourings and Fantasies are significant works of the German Romantics, this will contribute to the literary research on synesthesia in German Romanticism. I hope to give an innovating input by concentrating not on the notion of synesthesia as a metaphor (which indeed has been investigated extensively), but by considering synesthesia as a perceptual phenomenon inside the fictional text-world, that is: how its fictional character conceives synesthesia as his own sensual experience.
In a first part I will explicate, what was known about the phenomenon of synesthesia throughout the 18th century. From this, I will infer a special definition of synesthesia, based on this contemporary knowledge.
This definition of synesthesia I will take as basis, when investigating its literal realization, in the second and main part of the presentation. I will show how the character sees his synesthetic visions not only wedded with his main experience of music and of his emotions, but also interprets it religiously as mystic-epiphanic experience, far beyond the common human intellect and perception.
It will reveal that the synesthetic experience of an artist was a crucial circumstance of the romantic ideas on Art, Religion, and of the role of this artistic individual in it.
Carrie C Firman, Artist; Edgewood College, Madison, Wisconsin
Seeing the Sounds of Iceland: A Synaesthetic Exploration of My Temporary Home
Travel provides an opportunity to be out of one’s element–physically, culturally, and sensorially. While spending the summer of 2017 in Iceland, I collected sound recordings of the new soundscape I was living in: arctic terns and snipe in the air, rain on abundant metal rooftops, the ocean hitting cold black sand beaches, a myriad of tiny streams running from mountain to river, sheep bleating, the traditional three-stringed langspil, a cappella vocal folk songs, and more. These sounds were collected in both a tiny village in the far north and a densely populated suburb of the capital.
My new series of abstract animated digital artwork captures my synesthetic response to this Icelandic soundscape, recording shape, color, and movement. The aesthetic of a collective still image of this experience is informed by an Icelandic painter whose work I was drawn to as a visual artist and synesthete, Guomunda Andrésdóttir (1922-2002). Her abstract works show many characteristics of synesthetic art, especially similarity to the Kluver Form Constants and some direct title references to music. My presentation will show some artworks and research on Guomunda, who is not well known outside Iceland, as I share the process, intent, and final product of my project.
Photo Credit Carter Jones, Radhika Gosavi Presenting
Radhika S. Gosavi and Edward M. Hubbard, Department of Educational Psychology, University of Wisconsin–Madison
The Color of Memory: Synesthesia Enhances Multiple Stages of Memory
Radhika S. Gosavi1, Edward M. Hubbard1, 1Department of Educational Psychology, University of Wisconsin–Madison
In grapheme-color synesthesia, which is experienced by 1-2% of adults, synesthetes reliably and automatically experience specific colors when viewing black-and-white letters and numbers (Simner et al., 2006; 2009). Previous studies have identified synesthetes with spectacular memory and have found advantages for synesthetes compared to nonsynesthetes in long-term memory (Luria, 1968; Rothen et al., 2012; Smilek et al., 2002). Despite its well-documented impact on long-term memory, little is known about the impact of synesthesia on earlier stages of memory. To explore whether the synesthetic advantage in long-term memory could arise from advantages at earlier stages of memory, we tested 20 synesthetes and 20 nonsynesthetes to investigate the impact of synesthesia on iconic (Study 1), working (Study 2), and long-term (Study 3) memory. We find that synesthetes are significantly more accurate than nonsynesthetes across all three stages of memory. Critically, we found that this difference was greatest when participants were presented with higher item loads. These findings extend the current scientific knowledge by demonstrating that synesthetic advantages previously reported in long-term memory may indeed stem from earlier advantages in iconic and working memory. Furthermore, this information may shed light upon the involvement of multisensory integration in memory mechanisms as research suggests that recall is enhanced for objects with cross-modal rather than uni-modal components (Botta et al., 2011; Delogu et al., 2009; Quak, 2015). As applied to education, understanding the mechanisms of synesthetic memory could improve the lives of synesthetes and might be used to create enhanced learning methods for nonsynesthetes.
Alberta Ipser, University of Sussex, England
Standardizing Two Novel Consistency Tests for Lexical-Gustatory Synaesthesia
Alberta Ipser1, Julia Simner1, 1University of Sussex, England
In lexical-gustatory synaesthesia, the sight or sound of speech induces a sensation of taste and/or texture, subjectively located in the mouth, or automatically brings the thought of a specific taste to mind. For example, the word 'society' might taste like fried onions and ‘Ralph’ might taste of hamburgers. At present, lexical-gustatory synaesthesia is verified using the consistency with which word-taste associations are reported over long-term, repeated testing. Although it reliably dissociates between synaesthetes and non-synaesthetes, this method suffers from practical limitations (e.g. participant dropouts, lengthy data collection times). Furthermore, taste associations are provided as verbal descriptions, often resulting in subjective interpretation of consistency. The method is also based on the assumption that synaesthesia is fixed and that it does not change over the lifespan, but recently, we showed that saturation and consistency in grapheme-colour synaesthesia decreases as a function of age. In light of these findings and limitations of the existing testing method, we developed two novel 20-minute consistency tests for lexical-gustatory synaesthesia. The adaptable tests, which will be widely available for research use online, are the first to collect quantitative data on two different taste attributes: food categories and taste composition. We present comparisons of test performance from synaesthetes and non-synaesthetes, individually matched on age, gender and education-level, and discuss the practical, theoretical and research implications of our tests and findings.
Greg Jarvis, Musician, Composer; Durham College, Toronto, Canada
Tone Pallets And Other Tools of Synesthete Musicians
With synesthete musicians increasingly reflecting on the role synesthesia plays in their art, working from a personal ‘pallet’ of synesthetically pleasing stimuli is emerging as a commonality in their creative process. As a timbre-to-shape synesthete for whom all sounds trigger visions, Greg Jarvis will explain how he experiences music and how such ‘pallets’ influence the musical creations of himself and other synesthetes, including the Grammy-winning artist Lorde. Advantages that synesthesia provides musicians with will also be explored, including its influence on musical memory, its emotional ties to precision and perfection, its role as a creative stimulus, the impact of neural cross-connectivity in both creating and marketing music, and the driving quest for synesthetic euphoria
Imran Keshani, University of East London, England
Synaesthesia and Executive Functions: Exploring the Relationship between Inhibition and Grapheme-Colour Synaesthesia
Imran Keshani1, Mary Jane Spiller1, Clare Jonas1, Volker Thoma1, 1University of East London, England
Synaesthetic-experiences appear automatic, take time and effort to inhibit, but do not excessively disrupt the lives of synaesthetes. One explanation for this is that synaesthesia might be related to highly-developed executive functions; an attitudinal tendency for inhibition; or both. This study looked at whether grapheme-colour synaesthesia was related to: executive functions (specifically, performance on an inhibition and shifting measure); and an attitudinal proclivity for deliberate thinking, as measured on a short Need for Cognition scale (REI-NFC). The study also controlled inhibition-variables such as bilingualism. Participants (n=75) were not specifically recruited for having synaesthesia. To assess synaesthesia-like responses, participants completed a colour-picker task giving a pseudo-synaesthesia score. Results suggest that, when bilingualism was controlled for, colour-picking consistency for letters was a significant independent predictor of performance on the inhibition and shifting measures; and colour picking consistency for numbers was a significant independent predictor of performance on the inhibition but not the shifting measure, suggesting that trait-synaesthesia correlates positively with and can predict executive function ability. Additionally, performance on the inhibition measure, and scores on the REI-NFC were both found to be independent predictors of colour picking consistency scores for numbers. Results suggest that better inhibition abilities predict higher levels of trait-synaesthesia, whilst a greater proclivity for a need for cognition predicts lower levels of trait-synaesthesia, with numbers only. Results support the idea that it is through highly developed executive functions (especially in relation to inhibition), rather than an attitudinal tendency to inhibit, that synaesthetes effectively manage automatic-synaesthetic experiences.
Herve Pierre Lambert, Writer, Researcher; Paris, France
Synesthesia and Anthropology: Some Remarks
Alongside neuropsychological theories of synesthesia, anthropological approaches have been developed to study intersensoriality. We will, in particular, consider the works of Laurence Kirmayer, Constance Classen, David Howes, Lawrence Sullivan and focus on the following themes: The phenomenon of synesthesia in oral societies versus literate societies. The case of audio-olfactory synaesthesia, or ‘smell-hearing’: virtually unknown to neuropsychologists but known to anthropologists. What could a historical-anthropological theory of the phenomenon of synesthesia mean? In conclusion, I will comment on another current anthropological study about synesthesia: In Western society, how and why has synesthesia, previously associated with transgression and pathology, become a symbol of ‘connectivity’ in our Western late modernity?
Chang Hee Lee, Designer; Royal College of Art, London, England
Synaesthesia Materialization: Methods of Applying Synaesthesia as a Provocation for Generating Creative Ideas within the Context of Design
Synaesthesia has long been a captivating source for a range of disciplines from science to fine art practice. Its transcendental and intangible quality invites speculative thoughts and romantic aspirations. However, the application of such fascinating material has not been widely discussed in design research, and there is a particular lack of debate on the subject in design literature. Although researchers such as Giovanni Ancesch, Dina Ricco and Michael Haverkamp have discussed potentials of synaesthesia application, the large body of works in design research revolved around effective coordination of cross-sensory communication and optimization between product and users, rather than exploring synaesthesia experience itself in a design context.
This presentation attempts to share provocative properties of synaesthesia and its usefulness through exploring one of the author's projects, called Three Studies of Synaesthesia, and synaesthesia toolkits. This research tries to value synaesthesia as a source of inspiration and a provocative tool for generating creative ideas. The project Three Studies of Synaesthesia includes three different artifacts that present three methods of applying different properties of synaesthesia in a creative context. This presentation will introduce each method through exploring the project and the synaesthesia toolkits, and aims to generate opportunities and momentum in synaesthesia application research, primarily in a design context.
David Lewkowicz, Department of Communication Sciences and Disorders, Northeastern University, Boston, Massachusetts
To Be a Synesthete or Not: Can Infants Answer this Question?
Infants are born into a multisensory world with an immature, rapidly growing, and highly plastic nervous system. The neural plasticity makes them open to experience which, in fact, has major effects on how they end up processing multisensory inputs and how they acquire the ability to have unitary and meaningful multisensory experiences.
One of the still unanswered questions is whether initial neural immaturity and naïveté (i.e., lack of experience) result in a state of synesthetic “confusion” in early multisensory development or whether infants have orderly, non-synesthetic experiences that gradually become more adult-like and typical in terms of multisensory integration. One way to answer this question is by examining evidence from studies of infant response to multisensory objects and events to determine whether any of it might be interpreted as evidence of synesthesia-like responsiveness. In this talk, I will review evidence from such studies and show that infants enter the world with some rudimentary multisensory processing abilities and that those abilities improve as infants grow and acquire experience. Specifically, I will show that, initially, infants can perceive multisensory unity on the basis of low-level, physical perceptual attributes such as intensity and temporal synchrony and that as they grow and acquire experience with their world, they become capable of perceiving multisensory unity based on increasingly more complex perceptual attributes, including such attributes as phonetic identity, gender, affect, and linguistic prosody. In addition, I will show that infants’ usually restricted experience with native-face and native-speech inputs tunes infants' perception of multisensory unity in these domains, leading to native-input expertise and a decline in the ability to process non-native multisensory inputs. Finally, I will survey studies of possible synesthesia in infancy and show that some evidence is consistent with synesthetic-like perception but that other evidence is not. I will conclude by suggesting that the question of whether synesthesia might represent the initial state of multisensory organization in human development is still an open and fascinating one and that answering it requires additional research.
Jennifer Mankin, University of Sussex, England
When is “red” purple and “fire” blue? Conceptual influences on synaesthetic colours
Jennifer Mankin1, Julia Simner1, 1University of Sussex, England
Grapheme-colour synaesthesia is a promising but largely unexplored tool for understanding language. Our study investigates the colours that synaesthetes experience for two types of words. The first experiment looked at colour terms like “red” and “yellow”. The “alien colour effect” (Gray et al., 2002, 2007) occurs when there is a conflict between the typical colour of these colour terms and their synaesthetic colour (e.g. purple for “red” from a purple R).
We asked synaesthetes what colour they experienced for colour terms and similarly spelled words matched to the colour terms - for example, “red” and “reed”. Our results show that each synaesthete experienced the alien colour effect, but to different degrees. A multilevel model revealed that the synaesthetic colour changed depending on how frequently the colour term (e.g. “red”) appears in the language, and how vividly coloured the typical colour is. Our second experiment asked the same questions about words like “fire” and “mint”, which also have a strong colour associated with them. We again collected colours for these and matched control words, e.g. “fine” and “mind”. We found that the synaesthetic colour again systematically changed based on the frequency of the typically coloured word (e.g. how often do you see “fire”?) and the colour of the control word (e.g. how light and vivid was the synaesthetic colour of “fine”?). Our results illustrate that synaesthetic word colours display systematic conceptual and linguistic influences, which necessitate a more complex account of synaesthetic experiences.
Daphne Maurer and Julian Ghloum, McMaster University, Hamilton, Ontario, Canada
Face Processing in Synesthesia
Daphne Maurer1 and Julian Ghloum1, 1McMaster University, Hamilton, Ontario, Canada
One hypothesis about the origin of synesthesia is less pruning of the hyperconnectivity of the infant's cortex. Such pruning is said to underlie the phenomenon of perceptual narrowing, namely that over the course of the first year, infants become less sensitive to “non-native” contrasts such as the difference between two chimpanzee faces or two foreign speech sounds, while improving their sensitivity to “native“ contrasts, such as upright human faces. Previously, we presented evidence consistent with the pruning hypothesis: English-speaking adults with synesthesia were better able than controls to discriminate between chimp faces, inverted human faces, and two Hindi speech sounds. Here we explored whether synesthetic superiority for inverted human faces and chimp faces comes at the price of less expertise for upright human faces, as would be predicted by theories of how expertise develops.
In the first study, adults with and without synesthesia (n=20 per group) indicated which of two probe faces matched the preceding target face. The faces were filtered to emphasize coarse, mid-scale, or fine information (low, mid or high spatial frequencies). Adults with synesthesia were faster and more accurate than controls for both upright and inverted filtered faces. In the second study, adults with synesthesia (n=30) showed evidence of stronger holistic processing than did controls (n=30) for both upright human and chimp faces. Collectively, the results suggest that the preserved hyperconnectivity in the synesthetic brain can support superior processing of both native categories (e.g., upright human faces) and non-native categories (e.g., chimp faces and inverted human faces).
Helena Melero, Medical Image Analysis Laboratory, Universidad Rey Juan Carlos, Madrid, Spain
The Neuroanatomy of Olfactory Synesthesia–A Magnetic Resonance Imaging Study
Helena Melero1, Mario Gil-Correa1, Susana Borromeo1 and Norberto Malpica1, 1Medical Image Analysis Laboratory, Universidad Rey Juan Carlos
Odor-color synesthesia is a low-prevalence phenomenon whose neural underpinnings have not been explored yet. In this study, we investigate the neuroanatomical basis of olfactory synesthesias using Magnetic Resonance Imaging. CAT12, a toolbox implemented in SPM for volumetric parcellation of the brain, was used to measure gray matter volume in a group of 13 olfactory synesthetes and 16 controls. Mann-Whitney U test (synesthetes > controls; p > 0,05) revealed differences in gray matter volume in the left amygdala, the left posterior orbital gyrus and the left cerebellar vermal lobules VIII-X. These results confirm that the brain structure of synesthetes is different both in regions that belong to the specific sensory systems that give rise to the subject's synesthesia (i.e. olfactory areas) and in areas that participate in emotional regulation. This study did not find anatomical differences in the areas involved in the processing of the concurrent (i.e. color). These data reaffirm the idea that the brains of synesthetes are anatomically different, and provide additional evidence in favor of the Emotional Binding Theory (EBT) of synesthesia. Specifically, these data suggest that the common anatomical basis of different modalities of synesthesia may lie on the emotional regulation system, instead of being restricted to sensory and/or attentional/multisensory integration networks.
Louisa Rinaldi, University of Sussex, England
The Development of Synaesthesia in Childhood: Instruments to Identify and Evaluate
Louisa Rinaldi1, Julia Simner1, Rebecca Smees1, James Alvarez1, 1University of Sussex, Duncan Carmichael2, 2University of Edinburgh
We developed and validated tests for two types of synaesthesia in children: grapheme-colour synaesthesia and ordinal linguistic personification (OLP), defined respectively as colour and personality traits associated with graphemes (i.e. letters and numbers). Whilst there are widely used tests to identify grapheme-colour synaesthesia in adults, there is no standard test for grapheme-colour synaesthesia in children, and no test whatsoever for childhood OLP. To help parents and educators meet the specific needs of young synaesthetes, the MULTISENSE lab at Sussex University, funded by the European Research Council, has created a child-oriented version of the grapheme-colour test and a new test for OLP.
We tested three thousand five hundred children aged 6 to 10 on measures of academic achievement, creativity, wellbeing, and personality, as well as on our two synaesthesia tests. Using a modified consistency measure, we identified potential grapheme-colour and OLP synaesthetes. We then returned 8 months later and tested these potential synaesthetes a second time along with a group of age and gender-matched controls to verify which children genuinely experienced synaesthesia. Having identified genuine synaesthetes, we can investigate how they may differ from non-synaesthete children on our measures and how these two types of synaesthesia develop in childhood. This validation of our tests also allows us to define cut-off points for each age group such that synaesthetes can be identified in a single-session test. These new tests are sensitive and easily administered tools for parents, teachers, and researchers to identify synaesthesia in children and better understand their experiences.
Svetlana Rudenko, Concert Pianist; School of Computer Science, Trinity College Dublin, Ireland; Department of Fine Arts, University of Granada, Spain
Cross–Sensory Visualization of Musical Texture
Music is initially associated with our sense of hearing and we know that it has a strong power to stimulate our brain in terms of feelings, memories, or even to make our body to move rhythmically. But some people can experience music in additional ways- not just heard but seen, felt, smelt. This cross-modal perception, or synesthesia, allows them to perceive a particular stimulus with multiple responses, such as the shape or touch of sound, aroma of timbre, etc.
Artists use multi-sensory experiences from synesthesia to capture sound impressions in artistic forms. Musical texture of some composers, such as Scriabin, Messiaen, Liszt, Rachmaninoff (synesthetes/pseudo synesthetes), are organized in such a way that makes an image/archetype both visually and audibly recognizable. Music analysis could be the source for developing methodology of symbolic artistic visualizations. Scriabin as a performing pianist, stretched the piano to new limits of tone and touch. His performing tempo fluctuates continuously but the average tempo coincides with metronome indication: ‘he kept, side by side, two timelines’ (Leikin, 2011).
I argue that visualization of musical texture has a great influence on sound perception: timing of the musical phrases, dynamics of rhythm, harmony ‘tastes’ and the whole interpretation of the composition. My digital project, 4D visualization of musical texture, demonstrates volumetric properties of sound as a terrain ’‘landscape’ in musical-space synesthesia perception.
Photo Credit Carter Jones, Joel Salinas presenting
Joel Salinas, MD, MBA, MSc, Harvard Medical School and Massachusetts General Hospital, Cambridge, Massachusetts
Is Synesthesia a Gift, a Curse, or a Quirk?
Does synesthesia and synesthesia research have any practical applications in medicine? Should synesthesia ever be considered a medical diagnosis? And, why is it that some people with synesthesia experience their conscious sensory perception as a profound blessing, while others consider it a profound impairment?
As a synesthete, as a neurologist who specializes in behavioral neurology and neuropsychiatry, and as a clinician-scientist in epidemiology, I will examine these questions through objective and subjective lenses. Objectively, I will refer to relevant empirical evidence—prioritizing evidence with the most scientific rigor and validity—and suggest future directions for research. Meanwhile, subjectively I will share my own synesthetic experiences, draw on the anecdotes and personal reports from other synesthetes, and delve into the unexpected ways that synesthetes and non-synesthetes can learn from each other today—how the neuro-typical and neuro-atypical can help each other gain a better understanding of what it means to be human, to think, to feel, to be.
Krish Sathian, Pennsylvania State College of Medicine, Hershey, Pennsylvania
Neural Processing of Sound Symbolic Crossmodal Correspondences
Krish Sathian1, 1Departments of Neurology, Neural & Behavioral Sciences, and Psychology, Pennsylvania State University College of Medicine, Hershey, PA; Lynne Nygaard2, Kelly McCormick2, 2Department of Psychology, Emory University, Atlanta, Georgia; Margaret Martinez3, Sara List3, Simon Lacey3, Randall Stilla3, 3Department of Neurology, Emory University, Atlanta, Georgia
Crossmodal correspondences (CCs) occur between a variety of sensory stimuli. Sound symbolism refers to CCs between certain sounds and visual shapes; e.g., the auditory pseudowords “lomo” and “kike” (/lomo/ke/) are associated with rounded and spiky shapes, respectively. We previously compared the sensitivity of synesthetes and non-synesthetes to CCs, using the implicit association test. Synesthetes exhibited stronger congruency effects than non-synesthetes for sound-symbolic CCs, but not for more basic CCs (Lacey et al., European Journal of Neuroscience, 2016). Thus, at least for synesthetes, sound-symbolic CCs appear to differ from basic CCs.
Here we focus on our study of the neural basis of sound-symbolic CCs (the auditory pseudowords “lomo” and “kike” and visual shapes that were rounded or spiky) using functional magnetic resonance imaging. Stronger responses were found for incongruent compared to congruent audiovisual stimuli, when the auditory stimuli were attended. These incongruency effects were found in and around the right intraparietal sulcus and in left frontoparietal areas. The right-sided effects co-localized with incongruency effects for audiovisual temporal synchrony and with magnitude estimation effects; the left-sided effects co-localized with stronger responses for auditory pseudowords than meaningful sentences. These findings suggest that sound symbolic CC processing may involve mechanisms related to both multisensory integration and phonological decoding.
Ferrinne Spector, Edgewood College, Madison, Wisconsin
Visual Representation of Musical Sounds - Translational Elements across Domains
The purpose of this research is to expand upon a body of work examining music visualizations created by audio-visual synesthetes in response to musical excerpts (ASA, 2015). In the current research, we examined music visualizations created by non-synesthetes (n = 40) while listening to five of the original musical excerpts that varied on pitch (high/low), articulation (staccato, legato), and timbre. Participants used crayons and white construction paper to draw associations representing the music. Using systematic inter-rater coding, we found a) the presence of form constants in most abstract visualizations, b) the consistent presence of more shapes than are currently recognized within the dictionary of form constants, and c) the auditory components of pitch and articulation consistently lead to the presence of specific visual elements. Using these music visualizations across three experiments, non-synesthetic adults (n=40) listened to the musical excerpts and made a forced-choice between a congruent or incongruent visualizations. The congruent visualizations were created by either a non-synesthete (Experiments 1 & 2) or an audio-visual synesthete (Experiment 3) in response to the target music clip. The incongruent visualizations were created in response to a different clip of music by either the same non-synesthete (Experiment 1), a different non-synesthete (Experiment 2), or a different audio-visual synesthete (Experiment 3). When visualizations were created by non-synaesthetes, participants consistently chose the congruent visualization (all ps < .001). These results highlight the need to expand our dictionary of form constants, provide insight into the consistent translation of information across the visual and auditory domains and point to interesting differences between synesthetes and non-synesthetes in the translation of music to images.
Mary Jane Spiller, University of East London, England
Mental Imagery and Synaesthesia: Mental Imagery Ability Predicts Performance on a Letter-Colour Picking Consistency Task
Mary Jane Spiller1, Lee Hackner1, Peter Finton McCullagh1, Volker Thoma1, Clare Jonas1, 1University of East London, England
Research suggests that synaesthetes report more vivid mental imagery than non-synaesthetes. The current study further explores this idea by looking at mental imagery ability using both self-report and behavioural measures in a sample of participants not specifically recruited for having synaesthesia. Participants (n = 75) completed a battery of tasks, and we were interested in whether participants scores on mental imagery self-report scales (the Subject Use of Imagery Scale (SUIS) and the Vividness of Visual Imagery Quota (VVIQ)) and performance on a mental imagery task and mental rotation task, could be used to predict their consistency on a grapheme colour-picking task. We found taken together, the VVIQ and mental imagery task performance, along with participants' numeracy ability, significantly predicted performance on the colour picking task for letters but not numbers, whilst the use of imagery scale and mental rotation performance did not. These findings are important as they corroborate previous research suggesting a link between synaesthesia and mental imagery, but also highlight the complexity of the relationship, and the significant role of other correlates such as numeracy.
Photo Credit Carter Jones, Carol Steen presenting
Carol Steen, Artist; Touro College and University System, New York City
Synesthetic and Hypnagogic Visions: a Comparison
I have been a synesthete all my life and thought I understood my synesthetic perceptions rather well when, suddenly, in 2013, I began to see a new kind of internal image: hypnagogic visions. These visions, a normal state of consciousness, are known to occur somewhere between wakefulness and sleep and are reported by 1/3 of the general population.
They appeared to me as another source of photisms, in addition to my usual synesthetic experiences. But when I described these new visions to some researchers there was a wide range of reactions, from concern, to acknowledgement of similar experiences. While hypnagogic visions are different in some ways from my synesthesia, which I previously described in my paper, “Visions Shared: A Firsthand Look into Synesthesia and Art”, I find they share some commonalities. However, there is scarcely any discussion of a possible relationship between synesthesia and hypnagogic imagery. I intend to begin to bridge this gap.
In my paper I will explore both the visual similarities and differences in my synesthetic photisms and hypnagogic visions, discuss when my hypnagogic images first appeared, and where I see them now. I will compare the triggers or lack of them, range of colors, lines, shapes, geometric ornamentations, and movements I see. I will compare my creative approaches to working with both sources of imagery, and mention seeing usually distinctly separate synesthetic photisms and hypnagogic visions occur in the same experience.
Ella Striem-Amit, Department of Psychology, Harvard University, Cambridge, Massachusetts
What can we learn about the senses and the brain from people born blind, deaf or without hands?
How important is the sense from which information arrives to the brain? How does being born blind, deaf, or without arms, affect brain structure and function?
I present studies conducted on a group of congenitally blind people using sensory substitution (a transformation of images into sounds) and on a different cohort, of people born without hands, which show that no single sensory or motor system is critically required for association visual cortex organization. Findings from people born deaf support the extension of this principle to the auditory cortex. These data suggest that cortex organization is highly dependent on innate (prenatal) constraints, such that brain regions preferences are based on sensory-independent computations and domains of knowledge.
These findings will be discussed with reference to the model of a-modal brain structure and its limitations regarding developmental critical periods and brain plasticity.
Ninghui Xiong, Artist; Airbus Creative Thinking Pilot Project, Beijing, China
Music visualization: Study of Musical-Space Synaesthesia Hyperstructure
How can one transfer music to visual art? As an artist, the author uses his synaesthetic experiences to visualize classical music. This has become the main part of his art language. In this paper, the author presents his visual artworks in forms other than video in order to demonstrate how music translation from sound to visuals might be done. He will discuss two series of paintings that document his visual progress from two different pieces of music: “Flowing Water” by Chinese Qin Music, and Britten’s “Storm” from his Sea Interludes”. He will explain the multiple synaesthetic experiences he has for these pieces. In addition, he will discuss the visuals he sees from listening to “Chopin’s Ballade No. 1”, and “Ravel’s Bolero”. He will show a collage of images that appear alongside their scores which, for him, make a complete picture when the music ends.
In 2015, the author first proposed charts to visualize musical structure and put them into practice, “Music Synaesthesia Fishbone Chart” and “Music Synaesthesia Spider Ch”. Later, these methods were applied to large musical pieces and a series of paintings was produced. In doing this, the author has combined several charts in order to seek the inner relationship of each part of music and thus transferring them organically one after the other into a musical-space synaesthesia hyperstructure. Such methodology can be applied to various areas, for example in motion video, new media art, and programs designed for music performance (opera, dance, fashion demo and advertisement etc.).
