The proliferation of COVID-19 fake news on social media poses a severe threat to the health information ecosystem. We show that affective computing can make significant contributions to combat this infodemic. Given that fake news is often presented with emotional appeals, we propose a new perspective on the role of emotion in the attitudes, perceptions, and behaviors of the dissemination of information. We study emotions in conjunction with fake news, and explore different aspects of their interaction. To process both emotion and ‘falsehood’ based on the same set of data, we auto-tag emotions on existing COVID-19 fake news datasets following an established emotion taxonomy. More specifically, based on the distribution of seven basic emotions (e.g. Happiness, Like, Fear, Sadness, Surprise, Disgust, Anger ), we find across domains and styles that COVID-19 fake news is dominated by emotions of Fear (e.g., of coronavirus), and Disgust (e.g., of social conflicts). In addition, the framing of fake news in terms of gain-versus-loss reveals a close correlation between emotions, perceptions, and collective human reactions. Our analysis confirms the significant role of emotion Fear in the spreading of the fake news, especially when contextualized in the loss frame. Our study points to a future direction of incorporating emotion footprints in models of automatic fake news detection, and establishes an affective computing approach to information quality in general and fake news detection in particular.

Comprehension of metaphorical expressions differs with their degree of novelty. Conventional metaphors are typically comprehended as easily as literal sentences, while novel metaphors are responded to less quickly than their conventional counterparts. However, the influence of metaphor signals on the interpretability and acceptability of sentences with metaphors, especially their potential interaction with novelty, remains an open question. We conducted six online experiments among 1,694 native speakers of American English to examine how interpretability and acceptability ratings of individually presented sentences were affected by metaphor novelty and different types of metaphor signals. Across all six experiments, we consistently found that novel metaphors decreased the interpretability and acceptability of sentences compared to both conventional metaphors and literal controls. Signals, on the contrary, did not impact the interpretability or acceptability of the sentences. Moreover, only in experiment 3b did we find an interaction between metaphor type and signals. Specifically, when a metaphor was marked by double signals (i.e., both lexical signals and a typographical signal were added around the metaphorical keywords) vs. no signals, acceptability of novel metaphors increased, but acceptability of conventional metaphors decreased. We hypothesize that the double signaling of novel metaphors marks their novelty, making them more acceptable. By contrast, the double signaling of conventional metaphors may have been perceived as redundant, leading to a lower acceptability.

Nouns in human languages mostly profile concrete and abstract entities. But how much eventive information can be found in nouns? Will such eventive information found in sensory nouns have anything to do with the cognitive representation of the basic human senses? Importantly, is there any ontological and/or cognitive motivation that can account for this noun-verb dichotomy via body-and-world interactions? This study presents the first comprehensive investigation of sensory nouns in Mandarin Chinese, examining their qualia structures formalised in the Generative Lexicon Theory, as well as the time-dependent (endurant-perdurant) properties encoded in their sensory modalities. This study fills the gap in sensorial studies by highlighting the pivotal position of nouns in sensory experiences and provides insights into the interactions between perception, cognition, and language. Further, it establishes, for the first time, the cognitive motivation of the categorial noun-verb bifurcation without presupposing any a priori knowledge of grammatical categories.

Linguistic synesthesia links two concepts from two distinct sensory domains and creates conceptual conflicts at the level of embodied cognition. Previous studies focused on constraints on the directionality of synesthetic mapping as a way to establish the conceptual hierarchy among the five senses (i.e., vision, hearing, taste, smell, and touch). This study goes beyond examining the directionality of conventionalized synesthetic terms by adopting a Conceptual Metaphor Theory approach (i.e., the Conceptual Mapping Model) to test if conventional synesthetic directionality still holds when it comes to novel metaphorical expressions. The subjects, 308 native English speakers, are asked to judge the degree of commonness, appropriateness, understandability, and figurativeness in order to measure the degree of comprehensibility of novel synesthetic metaphors. Our findings demonstrate that novel synesthetic metaphors that follow conventional directionality are considered more common, more appropriate, and easier to comprehend than those that violate conventional mapping principles; they are also judged as more literal than those that do not follow conventional directionality. This study explores linguistic synesthesia from the perspective of comprehension of novel synesthetic metaphors, posits a pivotal position for mapping principles in synesthetic directionality, and supports an embodied account of linguistic synesthesia.

Short Descriptions

The shift towards the development of ‘new literacy skills’, specifically ‘multimodal literacy,’ has gained significant attention in contemporary education due to the emergence of new technologies and communication channels. Multimodal literacy extends beyond traditional reading and writing skills, encompassing a broad range of modes encountered in various texts, including words, images, sounds in printed and electronic media, and face-to-face interactions (e.g., Jewitt & Kress, 2003; Lim, 2021; van Leeuwen, 2017). Multimodality has become an integral component of the multiliteracies, leading educational systems worldwide to incorporate it into their language curricula and assessments. For instance, the Hong Kong Education Bureau has highlighted the development of multimodal literacy as a key focus in the English Language Education Key Area Curriculum Guide (2017) for secondary school students. The recognition of the significance of multimodal literacy in preparing students for the modern communication environment has increased, but teachers still face challenges in establishing a systematic and structured multimodal environment in the classroom that can effectively enhance students’ language skills and knowledge development. Previous studies reported that teachers feel uncertainty or unprepared for multimodal pedagogies as they lack the relevant skills to deliver multimodal practices in their classes (e.g., Ajayi, 2010, Li, 2020). To align with current educational trends, it is crucial to investigate multimodal practices in the language classroom to raise awareness of the importance of multimodal practices as well as to equip language practitioners with comprehensive multimodal literacy. This project investigates the implementation and effectiveness of multimodal practices in teaching Putonghua in Hong Kong kindergartens. Kindergartens play a critical role in laying the foundation for language learning, and incorporating multimodal literacy practices at an early age can significantly benefit students’ language development. In addition, the extensive use of multimodal means in early childhood education offers invaluable insights into effective pedagogical practices. Main objectives of this project include: 1) To explore teachers’ perceptions of multimodal pedagogies and teaching multiliteracies in the kindergarten classroom, gaining insights into their beliefs, attitudes, and understanding of this concept. This object aims to provide a comprehensive understanding of how teachers perceive the integration of multimodal practices in their instructional strategies. 2) To examine the actual classroom practices of teachers in implementing multimodal pedagogies, with a focus on observing how they integrate different modes (e.g. linguistic, visual, auditory, gestures) and resources (e.g. pictures, songs, digital resources) to enhance children’s learning experiences. This objective aims to capture the practical application of multimodal pedagogies and identify the strategies employed by teachers to create a multimodal learning environment. 3) To assess the impact of teachers’ multimodal pedagogies on the development of (multi)literacies in children acquiring Putonghua. This objective involves evaluating the progress of children’s listening, speaking, and reading skills, as well as analysing their overall multiliteracy development within the context of multimodal instruction.

Possible Benefits

While this project primarily focuses on early childhood education, the pedagogical approach employed in language classrooms holds relevance for language practitioners in university settings as well. The framework of multimodal literacy, explored within the project, exhibits a level of replicability that can be adapted to suit the needs of language practitioners working with university-level students.

Deliverables

The deliverables mainly include in-service and pre-service kindergarten teachers. We will provide in-service and pre-service kindergarten teachers with video lessons and training workshops. Journal publications and conference papers are expected after the completion of the project. The findings of this project will contribute to the existing body of knowledge on multimodal literacy and its application in language education. The outcomes will provide insights for educators, curriculum developers, and policymakers in designing effective pedagogical strategies to empower students to effectively communicate and interact in the diverse and technologically advanced society of Hong Kong.

The semantic meaning of the Chinese word 杠 gàng ‘wooden bar at the front of bed,’ has experienced a long derivational process driven by cognitive mechanisms such as metaphor and metonymy. In this study, we elaborated on the extended meanings of 杠 gàng based on the dictionaries and the corpus data. We have found a tendency of grammaticalization, denominalization, and deterioration of affective connotations for 杠 gàng. Further, we summarized five representative meanings of 杠 gàng and created image schemas respectively to discuss how the new meanings are generated and associated through the cognitive mechanism. With a radial framework of the semantic change of 杠 gàng presented in the end, we reiterate a continuum of metaphor and metonymy through realizations of image schemas and further highlight the interrelations between the cognitive mechanisms. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

The inherent semantic relatedness and closeness of near-synonyms pose difficulties to second language (L2) learners in comprehending and applying lexical knowledge in real situations. Previous studies have shown a sophisticated corpus-based method of distinguishing Chinese near-synonyms from a more ‘theoretical-based’ approach, the application of corpora in learning near-synonyms in an L2 classroom, however, is underexplored. This study reports both the ‘theoretical-based’ and ‘pedagogical-application’ of using corpora in studying ‘business’ related near-synonyms, and a significant gap between the ‘theory’ and the ‘application’ is identified. Our findings not only affirm the ‘theoretical-based’ method in capturing subtle nuances of near-synonyms but suggest implications of ‘pedagogical-application’ in teaching and learning near-synonyms with corpora. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

Human sensations and emotions are our primary embodied feelings in experiencing the outside world. The two systems are closely intertwined and jointly contribute to cognitive processes such as language use. However, how the two systems interact as manifested in our languages is still not well understood. This paper utilizes perceptual strengths and affective ratings to delve into the interaction between specific sensory modalities and emotional valence-arousal in Chinese. We found that smell and interoception, considered the two sensations directly linked to emotional processing, are more emotional and can elicit higher arousal levels than words associated with other senses. This study demonstrates the relevance and significance of the relationship between sensorimotor and affective information. It further sheds light on the embodied effect and associated emotional implications in the Chinese language. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

This paper introduces a linked data approach to grammar. In particular, we explicate the linked data design of A Student Grammar of Chinese (Zhu & Huang, 2022) to make Chinese grammar accessible to students of different backgrounds and levels. The main linked data design of this student grammar includes: boxouts for mouseover hyperlink effects in print, narrative building through examples, and word-to-grammar links to allow easy grasp of grammar.

Embodiment has been the tenet of several linguistic theories accounting for how language conceptualizes cognitive and bodily experiences. Studies on linguistic synesthesia and sensory lexicon strengthened the embodiment account by showing that the mapping patterns amongst sense modalities likewise exhibited a tendency from the more embodied to the less embodied. This paper reports a corpus-based study of gustatory vocabulary in Mandarin Chinese to explore the interaction between embodied senses and conceptual embodiment. We first observed that the perception of là 辣 'spicy 'and má 麻 'numbing ' was chemesthesis derived from a chemical reaction from the body. In addition, the concept of taste was found capable of being depicted by variegated non-taste lexical items from less embodied sensory domains. This study posits that gustatory properties as abstract cognitive categories are likely to be derived from more embodied senses, yet when the quality of gustatory sensation is the focus, less embodied senses can be adopted to modify it. Corroborated with other recent studies, this study underlines and clarifies the role of embodiment as a versatile tool of linguistic conceptualization among multiple conceptual layers instead of being a fixed set of conceptual objects to select from.

Metaphors are often used for presenting government policy to the general public in news, but the degree to which metaphors affect evaluation of such policies is not well understood. We conducted three between-subjects experiments (Nexperiment-1 = 331; Nexperiment-2 = 301; Nexperiment-3 = 608), in which participants read news items about foreign policies. News items contained either (a) novel metaphors, (b) conventional metaphors, or (c) literal controls. Results demonstrated that novel metaphors increased cognitive text perceptions, which led participants to evaluate proposed policies more favorably in a longer passage (Experiment 1) but not in a shorter passage with a larger percentage of metaphors (Experiments 2 and 3). By contrast, Experiments 2 and 3 showed a sequential indirect effect of novel metaphors (vs. controls) through perceived novelty and affective text perceptions on policy support. These results demonstrate that novel metaphors are helpful to readers processing texts about new topics as they draw attention to the language with their novelty, but remain familiar enough to generate positive affect.

Sensorimotor information is vital to the conceptual representation of our knowledge system. This study collects perceptual and action ratings for 664 disyllabic nouns among 438 native speakers and creates the first and largest dataset of sensorimotor norms for nouns in Chinese. Using aggregated semantic covariates, including concreteness ratings from a concreteness rating study, as well as the reaction times and error rates from a lexical decision study, our current work demonstrates the strengths of sensory modalities and action effectors in Chinese nouns and explores the contributions of embodied experiences in reflecting orthographic representations and semantic processing in the Chinese language. This study contributes valuable data sources to the study of Chinese lexical processing and highlights the importance of sensorimotor information and embodied manifestations in the semantic representations of concepts. Our results also support the language universal that orthographic awareness in lexical processing and reading supersedes phonological awareness.

Infertility is a prevalent medical condition that affects millions of individuals globally. Infertile patients may face multiple cycles of infertility treatments (e.g., in vitro fertilization, IVF) as well as psychological consequences such as anxiety, depression, and stress during the treatment process (Eugster & Vingerhoets, 1999; Maroufizadeh et al., 2019). Moreover, infertile women, particularly in developing countries such as China, often face stigmatization due to the traditional gender perception and social-cultural influences (Fu et al., 2015). Nowadays, infertile patients frequently resort to online support groups (OSGs) to seek information and emotional support (Erculj et al., 2021; Lee, 2017), although the effectiveness and outcomes of these OSGs vary (Lawlor & Kirakowski, 2014; Naslund et al., 2016). This paper investigates online discussions among infertile patients in a pregnancy-related forum in China and examines to what extent the OSGs provide emotional support. A topic modelling algorithm (Latent Dirichlet Allocation, LDA) was used and found that the predominant topics discussed among forum users include Family Relationship, Surgery, Therapy, and Affection. Using an automated lexical tool (Linguistic Inquiry Word Count, LIWC), we further compared the linguistic and psychological cues in the posts and comments. We found that pronouns and words related to positive emotions and insights are more prominent in comments; while terms concerning sad feelings, family, and power are more salient in posts, indicating that those posting comments are, in fact, trying to help alleviate emotional pain. This paper provides the first linguistic-psychological exploration of the online discussions among infertile patients in China and further adds to the literature the role OSGs play in helping people cope with stigmatized health conditions.

The genitive marker 的 li55 in Chongqing Mandarin can also function as a coordinating conjunction. This function develops from the usage of li55 to link numbers or quantities in calculations. Several restrictions are found on the coordinator li55, e.g., conjuncts must be nominal and shall be all the members of a definite set; li55 must be used between every two conjuncts and can only be used in informal registers. Similar coordinate function of genitive markers can also be found in other Sinitic languages while rarely seen in other language families, which merits further typological investigations. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

The possessive marker 的 li55 in Chongqing Mandarin can also function as a coordinator. This function develops from the usage of 的 li55 to link numbers or amounts in calculation. Several restrictions are found on the coordinating function of 的 li55, reflecting its original usage: the conjuncts must be nominal; 的 li55 must be used between every two conjuncts; it can only be used in informal registers; the conjuncts must be all the members of a definite set. The coordinating function of possessive markers is also found in other Sinitic languages, while rarely seen in other language families, meriting further typological investigations.

Human sensory and emotional systems contribute to cognitive processing in reflecting internal bodily experiences and representing the external world. A proliferation of behavioral and neurological studies has tapped into the effects of sensorimotor and affective information in semantic processing (e.g., Newcombe et al., 2012; Pulvermüller, 2005). Despite that considerable evidence of recruiting sensorimotor mechanisms in language comprehension has been posited, the interaction between sensorimotor and affective systems coded in semantic processing is still an underexplored topic. It is also important to note that emotions are weighed differently across sensory modalities—taste and smell are claimed to be the most “emotional senses” (Mantel et al., 2021). This neurological finding is further attested in the English lexical repository, given that taste and smell lexicons were found to contain more emotional contents than the lexicons of other senses (Winter, 2016). This study aggregated perceptual strengths across six sensory channels (i.e., vision, hearing, taste, smell, touch, and interoception) (Zhong et al., 2022) into affective ratings (i.e., valence and arousal) (Xu et al., 2021) to investigate the interaction between sensory modalities and emotional valence-arousal dimensions in Chinese. Our findings suggested that smell and interoception, considered the two sensations directly linked to emotional processing, are more emotional and can elicit higher arousal levels than words associated with other senses. This study demonstrates the differentiation of emotional information across different sensory modalities and provides further insights into the interplay between sensation and emotion as manifested in the language.

Improving the understandability of health information can significantly increase the cost-effectiveness and efficiency of health education programs for vulnerable populations. There is a pressing need to develop clinically informed computerized tools to enable rapid, reliable assessment of the linguistic understandability of specialized health and medical education resources. This paper fills a critical gap in current patient-oriented health resource development, which requires reliable and accurate evaluation instruments to increase the efficiency and cost-effectiveness of health education resource evaluation.

Linguistic synaesthesia involves conceptual conflicts created by two concepts from two distinct sensory domains. In previous studies, synaesthetic directionality is of pivotal interest. This study goes beyond examining the conventional synaesthetic directionality of five traditional senses by implementing the experimental method and adopting metaphor comprehension theory (i.e., Conceptual Mapping Model in particular) to explore how people comprehend novel synaesthetic metaphors. We used four measurements, including degree of commonness, appropriateness, understandability, and figurativeness, to judge people’s comprehension over two main types of novel synaesthetic metaphors (presented as adjective-noun pairs): novel synaesthetic metaphors that follow conventional synaesthetic mappings and novel synaesthetic metaphors that violate conventional synaesthetic mappings. The empirical findings demonstrated that novel synaesthetic metaphors that follow conventional directionality are more common expressions, more appropriate usages, and much easier to comprehend than those that violate conventional mapping principles; those that follow conventional mapping principles are also judged as more literal than those do not follow conventional directionality. The current study supports Conceptual Metaphor Model’s claim that mapping principles are the underlying reasons for the source-target domain pairings in conceptual metaphors, and further sheds light on theoretical claims about the systematicity of conceptual mappings for linguistic synaesthesia.

Most previous research has investigated how embodied cognition captures concrete notions (e.g. money), but the role sensory modalities play in more abstract concepts (e.g. time) lacks empirical research—in particular, how abstractness is grounded in perceptual experiences. In this paper, a sensorimotor strength rating study (also known as modality exclusivity norms) is conducted, to ascertain the sensory-perceptual information encoded in both concrete and abstract nouns in Mandarin Chinese. The preliminary results suggest that a sensation denoting one’s internal bodily feelings—interoception—captures more abstract information than the five basic human senses, and that the abstract concepts perceived predominantly by interoception mainly comprise OBJECT EVALUATION, MENTAL, THINKING, TIME, and SPACE as their ontological domains. This study affirms the embodied grounding of the concrete and abstract concepts, and further sheds light on the grounded account of mind-body-interactions.

The sensory lexicon plays a pivotal role in bridging our cognitive system to the physical world. In this role, it has been the focus of recent interdisciplinary investigations on cognition, language, culture, and their interactions. Recent studies on linguistic synesthesia and sensory modality exclusivity showed unequivocally that cross-modality usages of sensory words are the norm rather than the exception. Given the dominance of cross-modality uses, the null hypothesis that the five senses are separate but equal modules merits a closer examination. In this paper, we focus on the gustatory quality of sweetness because of its universal appeal as well as the well-attested cultural influence on the gustatory lexicon. Based on an analysis of online food reviews containing descriptions of desserts, we show that mouthfeel, a multisensory concept, is strongly preferred over sweetness. Mouthfeel is associated with words from all the sensory domains, including both sensory and abstract (e.g., mental state) concepts. The highly non-exclusive characteristic of gustatory sensation suggests that it might be the most connected sensory modality, and the cross-modality expressions indicating personal preferences further imply the subjective propensity of the gustatory sense. Our study adds to the existing literature the interrelationship among sensory modalities through language use, and further sheds light on the interactions between language, cognition, and culture.

Hearing or auditory sense has particularly strong temporality and dynamicity among the five sense modalities. Taking an ontological and cognitive perspective, this study examines hearing nouns in terms of their qualia values and eventive natures utilizing Generative Lexicon Theory and the basic ontological concept of endurant and perdurant. It is shown that linguistic representation of auditory perception related items shares strong perdurant properties. This is manifested by large proportion of event nouns, deverbal nominals and coerced event episode interpretation of hearing nouns. In addition, interpretation of classifiers of the default hearing nouns, 聲音 sheng1yin1 ‘sound’, further supports the eventive nature of the auditory modality. A sound referring noun phrase typically has an eventive reading even when it is enumerated with a classifier. In this context, the meaning refers to the frequency of sound-making events instead of the counting of sound content.

This article aims to analyse the four major sense organs of human beings, viz., 眼 (yǎn, eyes), 耳 (ěr, ears), 口/嘴 (kǒu/zuǐ, mouth) and 鼻 (bí, nose), in Chinese compound words with the combination of Generative Lexicon Theory and Embodied Cognition. It was shown that Embodied Cognition gives us an idea of the locus of the source domain in figurative use of organ-related words. Meanwhile, qualia structure in Generative Lexicon Theory, in particular, can be used to examine which sense of the word is activated when combining with other morphemes in a compound word. Moreover, the study found that the involved qualia roles vary in different syntactic structures and metaphorization of the compound words, which further demonstrates different lexical compositionality and productivity of the four basic sense organ words.

How human beings perceive the world through our five main physiological senses and how such perception is encoded in the human languages are intriguing yet challenging in linguistic studies. Considering uneven studies of Chinese sensory language compared to other Indo-European languages, this study takes the initiative to replicate Strik Lievers and Winter (2018)’s perspective of investigating sensory lexemes distribution across lexical categories, to examine the relationship between Mandarin Chinese sensory lexicon in terms of their parts-of-speech and the five senses. A list of 1,369 sensory related lexemes with annotated lexical categories and sensory modalities were compiled. The quantitative method was used to explore the correlation between sensory modalities and part-of-speech distributions. Chi-squared test yielded a significant result with the p-value<0.05, suggesting a strong relationship between these two categories and the existence of asymmetrical distributions in lexical categories among different senses. In addition, correspondence analysis was conducted, and the result suggests vision is the only sense associated with a high proportion of verbs, taste and touch are more associated to adjectives, while smell and hearing are found preferring nouns. But given there exists a large amount of “deverbal” nouns in auditory sense (occupying 58% of all the nouns), it suggests hearing tends to be more ‘verby’ instead of ‘nouny’. In view of the time-dependency features associated with different lexical categories (see Givón, 1979; 2001[1984]), this study goes further to discuss the ‘time-variant’ in each sense. We adopt Huang (2015; 2016)’s ontological endurant/perdurant dichotomy in which a concept which can be defined independent of time is endurant, and a concept which must be defined dependent of time is perdurant. Since some formal syntacticians suggested lexical categories could be decomposed to +N/+V features, i.e., nouns as [+N, -V], verbs as [-N, +V] and adjectives as [+N, +V] (Haegeman 1994, p.146), Huang (2016) claims that +N feature stands for endurant properties, and +V feature stands for perdurant properties. By decomposing words and linking such features to endurant/perdurant dichotomy, it is possible for us to suggest a ‘sensory endurant/perdurant scale’, on which smell is seen as the most endurant sense, touch and taste are in the middle, while hearing and vision are viewed as the most perdurant modalities.

Description of flavours of desserts often involves a rich range of vocabulary. This paper investigates the real-life language describing tastes of desserts in Mandarin Chinese, extracting data from Dazhong Dianping, the most popular restaurant review website in China. Using the Sketch Engine as the primary tool to extract collocations, we found that ‘mouthfeel’ and ‘personality’, instead of direct descriptions of TASTE or SMELL, are the most dominant expressions. In particular, more than one hundred ‘mouthfeel’ words are identified, with strong tendency of positive polarity. The majority of ‘mouthfeel’ terms are tactile (sense of touch) in nature, which shall be considered as synaesthetic metaphors to depict TASTE. Moreover, these ‘mouthfeel’ words often collocate with words connoting pleasant personality, especially in terms of warm social interactions. In summary, due to its intensional rather than physical telicity, description of desserts shows significant cultural variations. On the one hand, the preferred words still retain the same telic purpose, i.e., to please the mouth; on the other hand, instead of using expressions conveying intensional bodily pleasure (sensuality/sexuality), Chinese focus both on the parochial bodily experience (pleasing the mouth) as well as the social-interactional (pleasant personality) to describe desserts.